You need the mysql client library installed on any client host
A minor clarification -
If your mysql client app and mysql server are on different hosts, they communicate with each other over TCP/IP (using sockets).
However, you can't just open a socket and start "doing mysql". You generally need the mysql client library installed and configured to communicate between your app and the server.
The *library* will typically be the one who opens the sockets, and who translates your API calls into mysql protocol. You talk to the library; the library "does" the protocol translation and manages the networking transport. This is how PHP works, this is how Perl/DBD works ... and this is generally how any of your applications should work.
So you should plan on locating, installing and configuring an appropriate client library on your client host.
Hope that helps .. PSM