Same problem even with the correct driver
I'm using Kubuntu, and I'm having similar problems. The cpu usage doesn't go that high, but remains around 40%, with X.org being responsible for somewhat more than 20%. Things were worse before, but this cpu usage is still much higher than it was before yesterday. Yesterday restarting X still worked, as cpu usage droped down to 8%, but today this doesn't work anymore. Reading this thread, I've looked for the correct driver and I've noticed that it wasn't installed, although it was in xorg.conf. I've installed it using adept_manager and things got slightly better, but it's not the ideal yet. I don't know how to check if this is due to logging activity or anything else, because the only crazy app in 'top' is Xorg.