On a website I support (unpaid, a volunteer for an environmental organization) webcams upload images via ftp. I wrote a small app that detects every new image using inotify; it creates a thumbnail of the new image and archives the old image. At the moment we have 3 webcams so I run 3 instances of it. It has worked well for 5 years.

The host recently moved us to a new set of IPs, allegedly for security purposes. At the same time it changed the volume structure of the hard drive. At the same time it changed the owner and group of the httpd processes without telling us. (I suspect it restored the defaults.)

This forced us to figure out why webcams and the thumbnail app and some other external users weren't working.

After I sorted out the ownership problem a weird thing happens. When I run more than one instance of the thumbnail app all instances stop working, nothing gets written to the log.

I can't do anything as that user, not even ls or pwd; when I try I get

"bash: fork: Resource temporarily unavailable"

multiple times.

When I run 3 instances of the app I get a bunch of those then:

"bash: cannot make pipe for command substitution: Too many open files "

lsof shows only 324 files open for that user (I run as root to do this), only a fraction of them related to the thumbnailer.

Any idea what this is?

Sounds like the limits (see also ulimit) have been reverted.
Unfortunately, on Linux the defaults are set in various places eg /etc/security/limits.conf, /etc/security/limits.d (dir) and can be overridden by setting the same vars in the various login files eg /etc/profile, /etc/bashrc, & personal versions thereof.
See the discussion http://www.linuxquestions.org/questi...e-from-866550/ where I found out the hard way...
See cmd defn http://linux.die.net/man/1/bash

1 members found this post helpful.

Yep: that was it. Annoyingly it's set in 3 places: /etc/profile, /etc/bashrc, /etc/profile.d/limits.sh. I had to change all 3 of them.

Thanks.

I'd definitely complain to your host.
Changing IPs is one thing (although I fail to see the security reason), but messing with your OS (or apps/data) is definitely not on

I suppose it will encourage you to setup a comprehensive backup procedure & use it