socket: All ports in use
 

On my AIX box, when I try to "rlogin remotehost" I receive
" socket: All ports in use "
I guess it's a problem with the number of allowed outoing ports.
Where can I increase this number of ports ?
There is no "/etc/xinetd.conf" file in AIX, where are these things set up?
I need to perform 1000 simultaneouis rlogin's, and currently I cannot go above 509.

Hi -

Maybe you're running out of ephemeral ports.

The OS only has a limited range of TCP/IP port numbers it can assign to new connections, and TCP/IP requires a "TIME_WAIT" period before it can recycle an old socket number for a new connection.

You can use the AIX "no" command to try different values of tcp_ephemeral_low, tcp_ephemeral_high, and tcp_timewait.

On other versions of Unix (but not AIX), I would also check kernel parameters like "MAXFD" (maximum #/open file descriptors).

IMHO .. PSM

Node1 />no -a |grep ephem
tcp_ephemeral_high = 65535
tcp_ephemeral_low = 32768
udp_ephemeral_high = 65535
udp_ephemeral_low = 32768
Node1 />
tcp_ephemeral_high = 65535 looks rather high. Do you mean that 500 rlogin connections use 65000 ports ?

No, that doesn't mean that 500 rlogin connections use 65000 ports.

Yes, that *does* mean that those 500 rlogin connections might well be exhausting one or another limited system resource (like, for example, free ephemeral port descriptors).

Your homework assignment is to do some more research to figure out what resource might be the culprit.

Here's one link that may (or may not) be related to your specific problem:

http://linux.about.com/library/cmd/b..._rresvport.htm
<= The problem might have to do with the function "rresvport"; used by "rcmd", which is in turn closely related to "rlogin"

"netstat -a|wc -l", "netstat -a|less", "lsof" and (of course!) "errpt -a|less" are some of the commands that might be useful to you to get more information about the problem.

You can get lsof binaries for AIX here:
http://www.mirrors.wiretapped.net/se.../binaries/aix/

Good luck!

PS:
And by all means, look at your time_wait settings, too!

The reason for this behaviour is quite simple.

rsh/rlogin will only use trusted/privileged ports for outgoing communication, by that meaning ports with a number below 1024. In a typical configuration ports 513-1023 will be used, but other ranges may apply for your environment.

There is simply no way to sustain 1000 rsh connections from a single host without using unprivileged ports, which again would leave you wide open to a number of attacks.

The solution is to use pdsh or ssh, or both.

ssh does not require privileged ports to stay secure and is not affected by such limitations. You may however meet limitations on other system resources when running a large number of parallel sessions.

pdsh uses a sliding window to limit the number of parallell threads while running commands on a large number of remote hosts.

Additionally, pdsh can be configured to use ssh, allowing for a more secure approach to managing your clusters.