"I'm wondering, how would I go about installing 3rd party software on it? Do I have to compile stuff? And if so, how do I do it?"
There are different ways to install software depending on how the developer packaged it. You can install rpm packages with command line rpm or with kpackage.
You install tarballs by compiling them. You do have gcc installed don't you?
First you unpack the tar ball using the tar command, typically something like:
tar -xzf /pathname/package.tar.gz
Then you cd to the directory created by unpacking the tarball.
Then you usually go through the following command sequence:
If you want to change the tarball into a rpm you use checkinstall:
and then use rpm to install the resulting rpm package.
Here is checkinstall:
Be prepared. Create a LifeBoat CD.