Currently I have one physical server at home, running Debian and sitting behind my router and cable modem. I set up dynamic dns with FreeDNS. I can ssh into my home server using my public IP and all is fine and dandy.

Now I want to add a second physical server connected to the same router. From the way I'm understanding it, my public IP is an address on my router not on my current physical server. So when I add a second physical machine how do I determine which one I am SSH to if they hypothetically will have the same public IP through my router?

If I am way off, please help me understand. Thanks!

you could either set up your router so that it port-forwards ssh traffic to pc1 via port 22 and pc2 via port 2222 (for example)

what i do is ssh to my fedora server (port forwarded from router) and once i'm inside if i need to ssh to my xbmc machine/nas/fone/... i would issue the command like:
ssh user@192.168.1.5

No, you're pretty much spot on.

The IP addresses on your home LAN are private, non-routable addresses. Due to a shortage of public IP addresses, ISPs typically allocate only a single IP address to each customer, and the router has to "hide" outbound connections behind that one address using Network Address Translation (NAT).

Since there's only one public IP involved, when you SSH to that IP address from the outside, you actually end up connecting to the router. However, it is possible to have the router forward such requests to a private address on the inside rather than replying to the request, which is why when you SSH to the public IP, your internal server responds. At some point you (or someone else) must have configured this forwarding on the router, otherwise it wouldn't have worked.

As for forwarding an incoming connection request to a second server, that's only possible if the request is somehow different. The router must be able to differentiate between requests that are to be forwarded to the first server and requests that should go to the second server.

When it comes to SSH you're in luck, as SSH uses TCP as a transport protocol, and TCP has port numbers. The standard TCP port number for SSH is 22, and currently you have port 22 on the router forwarded to port 22 on the first server. What you can do, is forward a different port on the router to port 22 on the second server. You would have to explicitly tell the SSH client to use a non-standard port when you connect from the outside, but it would work.

There are 65536 ports in total, and you can basically just pick a number. Port 0 is "reserved" and 1-1023 are considered "privileged" (that is, a lot of them are assigned to common services), but the rest is mostly free game. The procedure for configuring port forwarding on a router varies somewhat depending on make and model, but a quick Google search ought to point you in the right direction.

Thanks both of you for the education! I am still learning as I go and as needed ;-)

I didn't think about ssh into my existing server and then ssh to the new server using the private ip 192.168.x.x but that sounds like the easiest way to do that. I actually have a couple of virtual servers running and that's how I access those. I am not really using the virtual machines for anything right now so I haven't ssh to those in a while hence the reason I didn't even think about doing it that way.

I have RSA keys set up on the existing physical server. If I wanted to set up the second server on another port I could still use RSA keys and just add -p command when I'm ssh in , correct? And then add port forward on the router.

While this will certainly work. It is bad practice to leave an SSH port open to the world.

There are plenty of VPN solutions that will get you into your private network more securely than that. Some even have 2 factor authentication.

I run OpenVPN. It's my staple. But there are plenty of other solutions if you'd like to go down that path.