You can try creating a symlink to your modem device to /dev/modem with a command like this:
ln -s /dev/<actual-modem> /dev/modem
But what kind of modem is this? Usually by default if Linux detects the modem, it will create this device for you and link it, etc. I have a feeling you got a winmodem and it might not work without proper drivers if any exist and some configuring on your part, etc.
Check
www.linmodems.org for more details if you do have a winmodem.