I'm fairly new to this so I hope someone will correct me if I make a mistake...
You should generate your key pair on a machine that you trust. It's really easy using the ssh-keygen commands on a linux machine or a Max running OSX. Once you create this pair, it's important to know which key is public and which is *private*. You can show the public key to the whole world -- put it up on http://keyserver.ubuntu.com/
if you like. Publish it on the front page of the New York Times. A public key is the one you hand to someone and you say "if you want to talk to me, use this key to encrypt your messages to me."
Your private key is SECRET and you should NEVER SHOW IT TO ANYONE. When people send you data encrypted with your public key, this is the only way you can (reasonably) decrypt it. If you are smart, you will keep your private key encrypted with a password -- ssh-keygen makes this easy because it prompts you for a password. Any application (like ssh or sftp or apache etc) that wants to use your private key for the purpose of encrypting or decrypting data should prompt you for this password so it can extract your private key from the password-encrypted text file you keep lying around on your hard disk. Having to supply a password to use one's own private key can be a hassle in certain circumstances -- e.g., when apache boots up on your server and there's no one around to enter the passphrase -- and in those circumstances, the private key is often stored unencrypted.
Bruce Schneier's book "Applied Cryptography
" is a masterpiece of a book that describes this stuff very patiently and clearly.