I believe you are speaking of X11 Forwarding.
There's a command called xhost that's controls which hosts are allowed to connect to your server. Have a quick read.
And when you want to connect to a server you need to set your DISPLAY environment variable correctly.
For example, when I ssh into my other box, and type
echo $DISPLAY
it tells me
localhost:11.0
Now the Xwindows server running on that box is 0.0, so 11.0 must be a connection to another server, namely the one running on the box I'm sitting at.
When I first set it up I had to tell it the ip address to use, so I ssh in to the other box and typed
export DISPLAY=192.168.1.66:0
and then ran an X program from that box. It appeared on my monitor and worked. Then I guess it remembered and I never had to do it again.
I'm sure someone can give you a more technical explanation, but I hope this gets you started.