In 1902, whilst a student at Georgetown University, Duffey ran a world record of 9.6 seconds for the 100 yards. Although equaled in 1906 by Dan Kelly, it would be 27 years before it was finally bettered, by Eddie Tolan.[5]
In 1905 it was rumored that he was to marry the actress Mabel Hite and, as she was a divorcee, he would meet Pope Pius X in order to attain special dispensation to wed. The rumor proved false.[6] Duffey finally married in 1911, to long-time friend Helen Louise Daley.[7]
In 1905 he confessed that he had been accepting sponsor money since 1898, and the AAU ordered all of his records expunged. In 1908 he attempted to form the National Protective Athletic Association (NPAA) to challenge the AAU.[8] Around this time Duffey was also involved in promoting professional athletes on the east coast racing circuit. One of those he worked with was the future Mercersburg Academy coach Jimmy Curran.[9]