The file /dev/modem is linked to the file which runs com port 3(/dev/ttyS3). Is this the com port which the modem is installed to? Your bios which be able to to tell you the com port it is installed on, and its irq and I/O number (looks like 00xd00). If these do not match what the file ttyS3 is set for (check using: setserial -agv /dev/ttyS3) then you can use setserial to set them. If you need to change them then I can tell you later.
Have a look at what I have said and write back saying the results from what I have suggested, which distribution you are running and how you use the modem, i.e what program you use to dial out(seyon,minicom,kppp, etc..).
I assume you are using windows to surf the internet, so windows will also have the information you need if you cant get it from the bios.
Dont worry about having problems with your modem, I have had loads of probs with my softmodem, but after tweaking it can dial out in Linux