Welcome to LQ mikiefitz456!
your probably not running in persistent mode
how did you install ubuntu to usb??
did you use usb-creator or what?
you gotta make a persistent ext2 partition on your usb
and label it casper-rw with gparted or whatever
so, sdb1 is fat32?
and sdb2 would be ext2, labeled casper-rw
see here
https://wiki.ubuntu.com/LiveUsbPendrivePersistent