Debian has a net installer that is a little under 250mb at
http://cdimage.debian.org/debian-cd/...64-netinst.iso
XFCE can be installed as the desktop.
If you're looking for a live image, I think most live Linux distros tend to go for something lighter even than XFCE, but Slitaz is designed such that you can easily roll your own version of it, so you might be able to build your own.
http://www.slitaz.org
Otherwise, Slax is a great little system that happens to ship with KDE in under 200mb. Has lots of packages and is easy to use. I have it on my keychain and use it often.