Rozehnal began his career at SK Sigma Olomouc, where his performances earned him a place in the Czech Republic under-21 team. In 2003 he was signed by Belgian club Club Brugge. In 2004 Rozehnal played for the Czech Republic at the European Championship where the team made it to the semi-finals before being knocked out by Greece. He won the Belgian Cup in his first season with Brugge and the Belgian League in 2005. He signed for Paris Saint-Germain in June 2005.

Rozehnal played in all three of Czech Republic's matches at the 2006 FIFA World Cup and is the 33rd most capped player in Czech history. At Paris Saint Germain in the 2006–07 season, he was named as the club's Player of the Year. His form resulted in reported interest from Borussia Dortmund, Newcastle United and Sevilla. Sources close to the defender confirmed that he would favour a move to Newcastle.

On 22 June 2007 it was confirmed by the player's agent that a "definite agreement" had been reached for the player to join Newcastle United. He underwent a medical check on June 25 and Newcastle confirmed the transfer had been completed on 29 June for a fee of £2.9 million.

On 11 August Rozehnal made his debut for Newcastle against Bolton Wanderers.

Rozehnal left Newcastle for Lazio, on loan until the end of the 2007–08 season, on 31 January 2008, despite having only signed in June 2007. After only playing 7 times for Lazio during his loan spell, it remained unclear if they wanted to sign him on a permanent deal. On 9 June 2008, Lazio officially announced they had signed Rozehnal on a permanent basis, with Newcastle recouping the full £2.9 million which they initially paid for the defender.

After only one year in Italy, Rozehnal left S.S. Lazio, on 29 July 2009 for German top club Hamburger SV, signing a contract until 30 June 2012. After a poor season with several costly errors by Rozehnal, he was removed from the first team and asked to find a new club. On 31 August 2010, HSV confirmed, that Rozehnal would leave for French side Lille OSC on a year long, free loan, with the German club still paying part of his salary. While at Lille, his side reached the 2011 Coupe de France Final where he faced former side PSG. Lille prevailed 1-0 in the final, claiming their first Coupe de France trophy since 1955. After 2010-2011 season with Lille, Rozehnal signed for Lille on a permanent basis.