I asked related questions in my recent thread:
I don't think I really got answers to some parts of that I would have expected to be answered in this forum. But despite that, I think some of what was said there would help p3aul in this thread.
Originally Posted by p3aul
use the space left over to install apps.
That aspect is much much simpler if you treat the USB as if it were a hard drive and do an ordinary install of Linux onto it.
You can use the left over space to install apps, even if you install a bootable .ISO file onto the USB (treat the USB as if it were a liveCD, rather than a hard drive). But that way adds extra layers of distribution specific complexity onto the task of using the extra space to install apps. Using the USB as if it were a hard drive means installing apps happens the ordinary way (just slower).
I still don't know what part of the performance deficit I saw in setting up a USB as a hard drive rather than liveCD was inherent in that approach vs. due to other details. But when booted in a better computer, even the USB as hard drive version wasn't all that bad performance. In theory file caching should make almost everything fast the second time you use it after booting. In practice, second use of each program was far faster than first use, but still not as fast as you would expect given enough ram that nothing ever needed to be dropped from cache once loaded.
For the best performing USB based system, I think you would want to install everything on a hard drive first, including installing and updating apps and whatever else needs to be customized, then convert it all to a squashfs and turn that into a bootable USB. I know the basics, but have not gone through the exercise to nail down the details and see the final performance. (The whole idea is based on CPU cost for decompressing being much faster than USB access speeds, so it will depend on your specific hardware).
When I installed Centos 7 from a bootable USB copy of the install DVD onto a second USB stick (treated as a hard drive), the installer was confused (as you would expect) about the drive number/name that the new install would have when finally booted. It really should have used UUID rather than device name in /boot/grub2/grub.cfg. It usually does and I don't really know why it didn't for me. But if you understand such issues, it is easy to check /etc/fstab and /boot/grub2/grub.cfg while the stick is still mounted in another Linux and make sure everything uses UUID. The next kernel update will overwrite /boot/grub2/grub.cfg, but in Centos 7 that used UUID anyway. /boot/grub2/grub.cfg only needed to be manually fxed once.
In /etc/fdisk make sure /tmp is a tmpfs, not an ordinary directory in the USB file system. Some of the installs I did got this right automatically. Others did not. When you are booted on a slow USB stick (pretending to be a hard drive) and web browsing, you really really don't want /tmp on the USB.
If you decide against removing the hard drive (for safety) during the install, pay close attention to any choice in the installer on where to put the first part of grub. When I did Centos 7, I never spotted that choice but somehow it did the right thing (maybe there is no choice or maybe the default is right). When I earlier did a similar install with Centos 6, the choice would have been easy to get past without noticing and the default was horribly wrong (would have connected the mbr of the hard drive to the directory on the usb, so neither could be used without the other). If there is a choice, make sure the first part of grub goes on the usb, not in the mbr of the hard drive.