G'day, my understanding is that QEMU only provides emulation of the video card, basically to capture graphics calls and rewrite them for X. Thus, QEMU won't let you directly access your video card.
Why don't you want to run the ATI drivers? What do you suspect them of doing to your system?
For better performance of Windows XP within QEMU, check out the Wikibook on QEMU:
http://en.wikibooks.org/wiki/QEMU/Wi...sing_the_image