If you are running the default GUI then it is a good idea to install something lighter, Ubuntu bring Gnome as default which is fat no doubt and memory killer. There are many other GUIs that you can try, Fluxbox is a popular one I prefer Blackbox, Wmii, DWM, they are simpler but way lighter than Gnome, Using one of these will drop the memory usage at least by half and probably less if pick really lightweight ones. Check this: http://xwinman.org/
There are many things you can do to reduce memory usage, some googling and you'll find easily.
Like disabling some daemons that are not in use. Keep one thing in mind you don't want to load in your truck more weight than it really can carry, it might run for while but it is gonna broke.
Meaning don't fry your motherboard giving it more work than it can handle
PS. please post your hardware CPU, Video, etc.