So is my initrd bootimage messed up or is this all caused by removing fc3 glibc packages ?
Yeah, well, glibc is pretty important for the health of your system as I am sure you have found. You can expect that not much of anything will work.
I suggest downloading FC4 cdroms (or dvd) and choosing the 'upgrade' install option. Hopefully this will save you from having to do a full reinstall.