Hmmm, here a few suggestions of mine:
If you have a connection to high speed internet, you could download a boot floppy for a specific distro and download packages through ftp. then once the distro is installed you won't need the floppy drive anymore.
Arch linux provides this technique for installing it's distro, but it not might be the best to start off with, especially if your still new to linux.
I know that Debian also offers the floppy-ftp technique, and it's a bit easier than arch to install. They also have a lot of documentation. Although general debian software is quite old, they don't always keep their stable relase up-to-date with the latest packages available.
I don't know if Ubuntu offers the choice, it's a distro based on Debian so it might.
Otherwise if you don't have high speed internet, you can find an iso somewhere, copy it onto your hdd using your usb caddy, load a floppy and make beleive that the iso is an actual CD drive. Gentoo has a section on alternate install methods, and they have quite a lot of different installation varieties, maybe they have one that uses this technique.
Well, those are a few of my suggestions. Maybe someone around has better ideas.