Yum only removes the package if it installed it i believe (at least it does for me)
you can uninstall RPMs with
Code:
rpm -e foo (the application name, not the package filename with issue/version)
This won't handle it's dependencies like yum does though
Infact this is a good quick reference for RPM commands
http://www.faqs.org/docs/securing/chap3sec20.html
No idea what the rest of the book contains, could prove useful too (who doesnt want to optimise and secue their box?

)
I pressume you are removing GCC to install an older version, or it is borked?