OK, we see that your external drive is /dev/sdb, so you can install Grub on it with
sudo grub-install /dev/sdb
After that you have to restore the Windows bootloader. To do that you have to boot from your Windows DVD and go to the recovery-console. Launch the command
and you should be able to boot Windows without plugin in the USB drive.
To choose Ubuntu at startup just plug in the USB device and choose it from the boot-menu after starting the machine. On my HP I get the boot menu with pressing F10 at startup, but this may differ for you.