Before you invest too much time in the project, verify whether or not your computer is capable of booting from USB (most older computers can't). (EDIT) I just re-read your original post; sounds like this isn't an issue in your case, sorry.
Headrift is correct: boot your other computer with the Live CD of your chosen distro and use it to install to the USB drive.
There is one thing to be careful of. Depending on the distro, it may default to installing the Grub bootloader on the internal hard drive, rather than the USB drive. (Ubuntu for example is notorious for this.) The last thing you want to do is overwrite your Windows MBR and be unable to boot without the USB drive plugged in. So make sure in the install options you are putting Grub on the correct drive. You may also need to edit the Grub options because the drive will probably be hd0 on the Optiplex and hd1 on the other computer. If you aren't sure how to do this, your safest option is to physically disconnect the internal hard drive on the Windows computer prior to beginning the experiment.