I think maybe what c_cinq meant was where to place them before installing them or compiling them. That was one of my questions at first, too. There aren't many directories that are writable by regular users; you often can't install source or rpms or compile things in, say, /usr/src or /usr/local/src. You can change permissions on those to make it okay for regular users to download and save stuff there, but that may not be too safe.
Anyhow, what I have is a /pub directory (which is actually its own partition, and has lots of room) where I can save stuff I download (as normal user), save rpms or tar files, etc. I also have a /pub/src directory where I do compiling of the source for programs I download. You could make /src, /download, and /rpms directories for each kind of file, if you want. I'd recommend doing something like that. If you're the only user on your system, you could just as easily put all that stuff in your home directory, but for example with rpms, you have to be root to install them, and I like having them in a nice global place. It's really up to you; put them wherever is comfortable for you.