I was having the same problem on a Debian system. I was getting sound from MP3s and other audio files from my hard disk (after I fixed the darn default volume=0 mixer setting

), but the CD player played but produced no sound. I just found this thread through a Yahoo search. Anyway, the answer was in the link supplied by Celettu above.
In short, most older systems have a direct audio cable from the CD to the soundcard (transmits sound already converted to analog audio), in addition to the IDE cable. Some newer systems don't have the audio cable, and the sound has to be transmitted over the IDE cable (transmits the digital data). If the software is expecting the analog cable, and your system doesn't have it, it will appear as if the CD is working, but you will get no sound.
The solution is probably simpler than adding a cable. You may just need to set an option on your CD playing software. For example, I am using the XMMS player. I had to right click on the player and select "options" "preferences" from the menu. This opens a box where I can select the "Audio plugins" tab, select the "CD Audio Player" plugin, and click "configure". There is a toggle button there that lets you choose between "analog" and "digital audio extraction". The default, "analog" wasn't working on my system. When I switched it to digital, it worked.
I would guess that some older CD playing software might not have this type of option. But most newer software will.