Here's what you need to do to get started.
Generate your public/private key pair.
Determine the key-id for your key-pair and use it to publish the public key to a keyserver.
gpg --send-keys DEADBEEF
If you would like to send an encrypted message to another user. You first need to import their public key into your keyring.
Import keys from a keyserver:
gpg --search-keys email@example.com
Encrypting a file for user2:
gpg -ae secretfile.txt
This would output the encrypted file secretfile.txt.asc
Most email clients allow you to specify the key-id to use for encrypting and/or signing email. I also remember having tried a FireFox plugin(FireGPG) that worked with Gmail.
With regards to backup, I would suggest making a copy of both your public/private keys as follows:
gpg -a --export-secret-keys firstname.lastname@example.org > secretkey.asc
gpg -a --export email@example.com > publickey.asc