All you need to is make sure sshd is running on the server. Open a terminal on another machine and type: ssh <username>@<hostname or ip> hit enter and input the password when requested. If you change the default port from 22 you will need to specify the port to connect to: ssh <username>@<hostname or ip>:<port number>.

Ok, having a desktop might not impact on your server that much (if at all) but what would you do if for some inexplicable reason vnc couldn't display a desktop?

Yes you could still return to using the desktop if you wanted to. It would be down to you which method of connection you use at the time.