I'm looking for good HPC cluster software to run on debian. Most of the stuff I've found hasn't had any updates in two years. Can anyone make a suggestion? My users are pretty adamant about a debian based OS but, I'd be willing to consider other options as long as it's not RHEL. (though I should mention I'm not a big fan of CentOS or Fedora.)

What do you mean by “HPC cluster software”? Just a queuingsystem to distribute the jobs of the users to free nodes?

I'm trying to setup a diskless beowulf cluster. I had planned on using slurm for queuing jobs. It's my first stab at a cluster for my bioinformatics group. I was going to use Caos NSA and Perceus but, it seems that Perceus is no longer supported and all the download links are broken. I finally got my servers to boot from KestrelHPC but, they lock up and become unresponsive once the Dracut image loads. Essentially, I would like to have diskless nodes where I can assign the hard drives to be fast r/w swap space and be able to easily edit/configure/install to the images that will be pushed to the nodes. I hope I described that right. I'm new to clustering.

You might looks at something like ROCKS ... I think it's pretty well supported. I'm not sure how well it can handle NFS root filesystems, but I recall hearing at one point that it could handle them.

How adamant are you about the whole diskless thing? What I do on my HPC setup (several hundred nodes) is to have local OS installs (done via kickstart since I work in Red Hat space; FAI can be used for Debian based distros) and then I use puppet to manage configuration etc. to ensure consistency. TORQUE is used for batch queuing with the Maui scheduler (because we've used them for years and know how they work), but SLURM is a good choice as well. The OS takes up only a small part of the disk (a few gigs, since the installs are pretty stripped down), and the rest is used for scratch space.

To be honest, I find it easier to take a stock distro and convert it to my needs rather than use specialized HPC software like Perceus, ROCKS, etc. It's a bit more work at the outset, but you get much more control of what the final product looks like. If you want to go diskless or NFS root, this might be a good way to go since Googling around should give you lots of good info.

wow. thanks for the awesome response! I've been googling like crazy but, the options for ubuntu and a diskless HPC cluster (or any cluster for that matter) seem to be limited. Unfortunately, I've been instructed to use ubuntu so I don't have a lot of other options. I might be able to get away with debian but I'm not sure. The whole idea was to be able to just plugin a new server, set it to PXE boot, and let it run. Seemed like a good idea at the time but, it's turning out to be harder than I expected.

You might also look at Warewulf then. I've never used it, but it seems to do a lot of what you want in terms of setting up a stateless cluster.

yeah, I looked at that one but, they don't have any documentation on stateless provisioning for Debian Linux.