I think you are going to find that running from a USB drive is even slower than a live CD. This is because transfer rates over the USB bus are alot slower than over the IDE bus from a CD drive.Plus, if using a FLASH drive it's not going to last very long, especially if you set up a swap drive on it. Even if you use the 'noatime' option with filesystems which support it, the drive will only last for a few months. It is possible to setup a USB drive so that it will last longer, but it basically means running like a liveCD from the drive. This means that you run in read-only mode and use the same memory-consuming techniques as liveCD's use.
Some time ago I began work on a distro designed from the ground up to do just what you want. It boots up in read-only mode, but without loading everything into RAM. It then uses tmpfs to allow 'normal' running in read-write mode. But in order to save any changes you make you have to remount read-write on the fly and sync the file system on the device with what's in tmpfs. All-in-all it's a pretty complex way to run.
You can however run a normal installation from a real USB hard drive (not FLASH), but it will still be slow because of the transfer rates. Every program will be slow to start the first time. After that they will run nicely though since they are cached in memory by the kernel. Running from a real USB hard drive is still slower than running from a FLASH device because of the disk access times, but at least you can run a normally installed and configured system.
You could use a special filesystem called jffs2 or jffs3, but the partitions take a very long time to mount. This is normally used for devices which run an embedded OS where the device is not re-booted very often, or ever.
The problem with FLASH devices is that data is written block-wise. This means that when only one byte of data is changed a whole block of data must be re-written. And the number of times a particular block can be re-written is limited to somewhere between 10,000 and 1 million times. This may sound like a lot, but it is not -especially, as mentioned, for swap space.
The system I have been working on needs a little less than 256MB for a nice GUI desktop with a few dozen programs. But everything is light-weight software and some of the key components are loded into RAM in order to reduce the starup time for the basic desktop. Still, as soon a you start a browser, for instance, it takes about twice the normal time for it to start. Afterwards it runs normally, as mentioned above. If you were to try running KDE that way, I imagine that it would 4-5 minutes for the desktop to load. This is (mostly) not dependent on how fast your processor is nor how mauch RAM you have. The bottle-neck is the transfer rate over the USB bus.
If you really want to have a complete system 'to-go' which runs fairly normally, you need to put it on a real USB hard-drive.