Hmmm... DO you have a standard install and automount enabled? I'm also running Fedora Core 4 (x86 32-bit, though) and when I insert my USB thumbdrive (pendrive, whatever) I get a USB removeable storage media icon on my desktop (under Gnome, never tried it under KDE) and that's it. Right click and unmount to make sure everythin is actually written/modified before I pull it out.
I wonder if you do not have automount enabled, or if FC4 64 does not recognize a USB drive by default. At any rate, this is an automount issue. That should help you look in the right direction. Start googling for related automount issues and I'm sure you'll find something.