If I use yum update or yum update <package>, it will download the full latest version of the package to update. It appears so, as some packages are average 15MB download.

When it untars them to install them, what happens to the tar.gz files afterwards? Are they still there on my PC? Where and how do I clean them up / remove them to free space?

I believe they are removed as I never found any remnants of yum based updates...

I could be wrong...

some are in /var/cache/yum/REPO_NAME/packages.

Hmm, I couldn't figure out how to read du -h or df -h properly. So I used Konqueror instead. And from the properties of the folders:

/var/cache/yum == 401.8MB
/var/cache/ == 405.7MB

Wow, shouldn't we clear this stuff? Is it okay to clear the cache? Such as doing a massive rm -Rf * ?

---
/var/cache/yum == 401.8MB
/var/cache/ == 405.7MB
---
I think this means the cache dir (which includes cache/yum), so the size is really in cache/yum.
I keep the updates so that if I need to re-install etc or get another machine up to the same state, I already have the rpms and hdr files on disk.
I made this easier for myself by changing the cache directory to a different partition (that doesn't get erased on install). I also copy this to other machines that have the same version installed.

However, I do tidy up the cache rpm by using gftp to compare the cache/yum folder to fedora/redhat update ftp site. Then I delete any files that are shown as differing (highlighted) on my disk (these are eg old versions of packages - kernel / mozilla etc are large, so this saves lots of space).

You can clean up the yum crumbs with either:

yum clean packages

or

yum clean all