server "hardening" - users accidentally locking cluster master node
Linux - SecurityThis forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
server "hardening" - users accidentally locking cluster master node
All,
I run a compute cluster with only a few users. Occasionally a user will accidently run a job on the master node that runs out RAM/swaps then hanges up for a while.
In /etc/security/limits.conf I have set memlock to 7.5GB (master has 8GB RAM) and maybe that is what lets the machine come back rather than hanging completely?
Is this the right setting to physocally limit a single user from asking for more RAM than the system has and bringing down the system? Should I set this to 2GB or so or is there something else I can do??
System is running 64-bit Fedora core 6. (Quite old now but I leave the OS alone until the hardware is replaced)
The 7.5 GB was there for another reason but I was asking whether it helped prevent total melt down of the system?
My secondary question was whether using the same method, but reducing the 7.5GB to say 2GB would prevent a user starting an application then reading in massive data files way beyong the memory on the system? (either not understanding what they are doing, or accidently starting a local job that was intented to run on many cluster nodes.
The 7.5 GB was there for another reason but I was asking whether it helped prevent total melt down of the system?
Probably. Thing is: if there are enough processes eating up memory that are not affected by pam_limits(8) - i.e. daemons and other processes that don't have PAM support baked in or aren't configured correctly - you could still technically bring down your system. It sounds like your main concern is rogue/errant users, though.
Quote:
Originally Posted by mckenzig
My secondary question was whether using the same method, but reducing the 7.5GB to say 2GB would prevent a user starting an application then reading in massive data files way beyong the memory on the system? (either not understanding what they are doing, or accidently starting a local job that was intented to run on many cluster nodes.
Sure, assuming we're still talking about your system with 8GB RAM. 2GB would be a more sane restriction, IMO, depending on what the users are legitimately supposed to be doing. (As with all things: test this out to be sure pam_limits(8) is behaving the way you'd expect after making any changes.)
My main concern was whether the memlock setting was doing what I thought from what you say it seems that it is. The setting of an appropriate value is the trick.
Too low could be overly restrictive of legitimate usage and too high still allows the system to lock up with accumulated other memory usage adding to the trouble maker process.
I think I might go with 4GB next time I can squeeze in a reboot and do some testing to check that it stops me using more than that.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.