Nelson began his play at Saint Joseph’s University in the 2000–01 season. He had a breakout freshman season in which he was named unanimous National Freshman of the Year. During his junior season in 2002–03, he averaged 19.7 points per game, 5.1 rebounds per game, and 4.7 assists per game. He declared for the 2003 NBA draft, but later decided to remain for his senior season.
Nelson led the Saint Joseph’s Hawks to a 27–0 regular season record in 2003–04. The Hawks’ first loss came in the Atlantic 10 Tournament to Xavier. Nelson and junior guard Delonte West formed what was largely considered the nation’s best backcourt, helping the Hawks earn a #1 seed in the NCAA Tournament. They advanced to the Elite Eight and were within seconds of the Final Four before Oklahoma State Cowboys’ John Lucas III hit a three-pointer with only a few seconds remaining (after the make, Nelson dribbled up the court and had a chance to tie the game, but his 15-foot shot fell short). Saint Joseph’s finished with a 30–2 record, the best in the university’s history. Nelson averaged 20.6 points, 5.3 assists, and 2.9 steals per game. He received the Lowe’s Senior CLASS Award his final year, recognizing him as the nation’s top senior men’s basketball player. He left the Hawks as the best player in the program’s history, as its all-time leader in scoring (2094 points), assists (714), and steals (256). Nelson’s number was retired by the university on April 23, 2004.
Because of his extraordinary accomplishments as a senior, Nelson won the 2004 Wooden Award, the 2004 Naismith Award, the 2004 Bob Cousy Award, the Rupp Trophy, the Oscar Robertson Trophy and many more accolades, including being featured on the cover of Sports Illustrated. Nelson was the first Atlantic 10 athlete to be on the cover of the magazine since Mark Macon in 1988.