ok. Big explaining to do here

(not that i'm bothered

)
in short, / contains EVERYTHING you have on your computer, even devices (yes, your soundcard is there also! in /dev/ folder. And even the state of your processor is written in /proc/) interesting, isn't it?
In long, there is a bit more detailed explanation.
Here is what a ls command throws out on my machine.
Code:
atom@saturn / $ ls --all -1
.
..
bin
boot
dev
etc
home
lib
media
mnt
opt
proc
root
sbin
sys
tmp
usr
var
atom@saturn / $
Ok, so here's the explanation.
ls --all -1 The command to list the files in directory, list all of them and list them in a column, rather than using less space and listing them in paralel.
. and .. they both lead to /, since this is the topmost directory in the filesystem
bin there are most of the commands here. Since the commands are standard, there is no need to back them up, they get included with every distro
boot the kernel is in here, just as is the boot loader. you might want to back that up, especially if it works

.
dev no files here, just devices. (i'm actually wrong here, because everything on unix is a file, even your graphics card, but for simplicity...)
etc this is an important place to start backing up if you have made any changes to the configuration of your sistem. All configuration files should be kept here, in one centralised place.
home this direcory contains all user's files, personal profiles, histories, program options, configuration, etc. This is the only place that is a must to backup
lib the meaning of this directory is somewhat unclear to me, but it's name tells it keeps system libraries. Never backed it up and i'm still alive.
media i've left this one in on purpose, even though it's non-standard. It contains all the files my users share (presentations, music...)
mnt nothing to back up here, contains the external devices you have mounted as filesystems (cdroms, windows partitions, floppies...)
opt optional... software. No need to back up IMHO, but it might be a good idea if you have anything you have dug up over the centuries saved there. (i'd have that in my home anyway)
proc this is a filesystem that is truly imaginary. It has the current computer state saved in there and it has absolutely no value to back up
root the root user's home directory. You should back that up or have your account deleted by the angry administrator... wait, that's you
sbin the root user's special chest of tools. No need to backup
sys don't really know what's in there, but i guess that's because only programs need it, and not you. no backup.
tmp like the name tells, temporary files. No need to backup imho.
usr programs. the /usr most closely resembles the c:\Program Files on windows. Unless you are a psycho and want to preserve every single file on your comp you do not back that up. You reinstall the updated programs.
var if you are running a server it might be a good idea to back this up, because apache, mail servers and most of the others keep their files here. If you are running a strictly office workstation there might be no need to back it up.
I'm not 100% in all of these, but I stand behind my word. This is how i would have done my backup (and have done) and it never failed yet. Some users might have different experiences but this is mostly it.
Hope some of this comes in useful,
Gasper