Hi folks,

I know this is going to sound stupid at first, but please bear with me and I'll explain why I want to do this particular stupid-sounding thing.

We have a Beowulf cluster, currently running RH 8.0. We're about to do some major reconfiguration, including changing to RHEL4. The choice of distro is pretty much set in stone as it is approved and supported by a couple of commercial apps that we need to run, so please don't bother with any religious distro wars even if you would be so inclined otherwise.

Anyway, in preparation for the changeover, I expect that we will see some "anomalies" so I am setting up a 3-node test cluster just to make sure that I can have everything ready to run on the real cluster by the time we make the changes there and I can minimize downtime. I plan to keep this test cluster up for a couple of months and then wipe all of the boxes.

The real cluster is isolated from the rest of the world and only the head node is accessible, and that only by ssh, sftp, scp. That is, no other protocols are allowed to initiate incoming connections. Outgoing, the only additional protocols allowed are ntp and dns, and those only to our servers.

The test cluster is just 3 old boxes that I was able to scrounge from our spares/discards pile and put into a rack. They are all connected to the router in the rack. I was not able to isolate them like the real cluster. I want to make the test cluster appear to act as much like the real cluster as possible.

Now, we finally reach the stupid question part: Within the real cluster, users are allowed to use rsh/rlogin to other nodes and lots of software and scripts are set up to do that. In that cluster environment it's secure enough as the packets never escape from the rack into any larger network. In the test cluster, the "compute nodes" are visible, at least to the rest of the campus, though not to the internet at large. I'm still not planning to allow the users I'll be asking to test their apps to log in directly to the compute nodes, but the nodes are "visible". (The network segment that they are on is restricted to the secured computer room, for what that's worth.)

What I plan to do is add a suitable entry or entries to the iptables on the 3 nodes (head and 2 "compute" nodes) so that users can rsh between them. I have already edited hosts.deny to deny all access to everybody and hosts.allow to allow rsh/rlogin access to the other nodes. I have also edited /etc/hosts.equiv. Next, I need to poke a hole into the firewalls of all 3 nodes so that packets can actually reach the other nodes. For purposes of discussion, assume that the head node (of the test cluster) is 10.192.255.1 and the other two are 10.192.255.2 and 10.192.255.3.

I would expect a rule much like the rule that allows ssh to initiate connections. I plan to copy /etc/sysconfig/iptables to a test file, e.g., /etc/sysconfig/iptables.test and edit that so I can use iptables-restore < iptables.test and then eventually use iptables-save to keep what I want. The current /etc/sysconfig/iptables contains the line:
so I think that the following should be appropriate
with corresponding rules for the other two potential source nodes. The relevant port appears to be 513, or at least that's what appears in netstat when I make an rsh connection. I'd rather not use a netmasked source because there are other boxes on 10.192.255.*, even though they are all windows boxes in a secured computer room. Thinking about it, I suppose the risk of using "-s 10.192.255.0/24" may not be too great. What do you all think about that?

I know it's too late to make this long story short, but at last we've reached the end: 1) Is this really a suitable solution?, 2) Is my proposed rule correct? and 3) How stupid is it really? Am I paranoid enough?

Thanks,
Paul

this kinda comment is really uncalled for IMHO... i've read your question (all ten paragraphs of it) and there really wasn't any need to even bring up the distro topic at all... you could have just gotten into the rsh/iptables topic directly... anyways, here's my :

1) sounds good to me...

2) yeah, it's correct, as long as you are positive about the port number...

3) this couldn't possibly be considered paranoia, this is very basic iptables stuff... having said that, if you'd like your rule to be a little more specific, i'd suggest throwing in another match into the rule... like perhaps a MAC address, for example:where XX:XX:XX:XX:XX:XX is the mac address of the nic with ip 10.192.255.2...

BTW, yes i think it's better if you don't use the subnet match, it's better to be as specific as possible...

Thanks, win,

I am sorry for the length of my initial post, but in my experience it's awfully easy to leave out what turns out to be critical information, and I am, as should be obvious, relatively inexperienced with iptables -- most of my experience has tended to needing only pretty generic firewalls. I also expected that if I didn't explain what I was doing and why, the most likely response would have been along the lines of "don't use rsh, use ssh," which would be the proper answer 99% of the time, so I figured that if I was going to ask something that would probably look like a stupid question, it would be a good idea to explain why I wanted to do it.

Anyway, thanks very much for taking the time to answer my questions and for suggesting an improvement. I plan to take your advice.

Thanks,
Paul

Update:

I did take win's suggestion and, with a bit more floundering around, I can now rsh/rlogin within my test cluster and I also have nfs working, though it took a while before I found out that with the version of nfsd and related daemons that I have, I also have to list both nfsd AND mountd in /etc/hosts.allow. But it's looking much better now and the only contact that anyone outside the cluster is allowed is by ssh.

Thanks,
Paul