There is no way that the RH8 kernel is going to handle a usb modem connection. I had an older Motorola Surfboard 5100 back in the day and the consensus was that you couldn't get it working in usb mode on RH8, mandrake 8.2, 9.0, etc or anything else that came out about that time. The drivers simply were not in the kernel or otherwise available. If you can run in ethernet mode(I could on my old surfboard), do so; it's a no brainer setup in linux that way, even on older distros of RH8 vintage.
You have a couple options if you want to use usb. First, you can try and find the necessary drivers on the net and compile the driver modules for your kernel. That's not an easy task for a newb. Second, get a more modern distro which is more likely to have usb support for your modem. Ubuntu will send you an install cd of their most recent release free of charge, postage free:
However, if you can go ethernet, that is the way to go. Ethernet over usb is a problem is both windows and linux. Straight ethernet is the way to go.