I'm surprised your existing PC will not boot from USB. I thought VIA boards had this for a long time. You've checked the BIOS menu I guess. As an alternative you could get an IDE to Compact Flash adapter (easy to find on eBay). The CF card looks just like an IDE drive to the system. Have to open the case tho'.
You can buy a new 1.5GHz VIA cpu/mboard for $US60:
http://www.linuxdevices.com/news/NS5305482907.html
and put it in any micro-ATX case. I don't think this board is mini-ITX format like the one you have. Intel also make cheap m/boards with soldered-on Celeron processors. Look for models D201GLY and D201GLY2, but check for hardware compatibility first.