"Unable to fork: Cannot allocate memory" even 10GB RAM free
 

Hello,

on a dedicated server i am using a bash script like this: https://github.com/slrslr/linux-bash...slogscanner.sh

I believe it not returned any errors "Unable to fork: Cannot allocate memory" when it was executed.
But now, when i execute it, i see this "Cannot allocate memory" error.

I do free -m before it and after it and i see there is plenty of free RAM (when i not count cache):

https://s20.postimg.org/bcqsjr2a5/fr...nable_fork.gif

it is not first time i am having trouble with this error, so it probably is not just "bad" commands executed from the bash script.

Please what should i do to discover the cause, why this happens?

you can put set -xv at the beginning of the script. And you need to give more info, otherwise we will not be able to help you. At least the complete error message, the real command which failed...
I would say it is not the bash script itself, but a special command inside...

Is it 64-bit installation?

You say "dedicated server"...what ELSE is running on the server? As pan64 said (and others have also, over the years you've been here), you need to provide details.

Secondly, what have you actually done/tried to see what's going on? What (if ANY), troubleshooting have you done? Did you try running the commands in that script one at a time, to see if any of them cause the problem? Watch memory when you start it up? ANYTHING???

Seems odd that after three years asking such questions, you can't think of any ideas on how to troubleshoot anything:
http://www.linuxquestions.org/questi...ng-4175486469/
http://www.linuxquestions.org/questi...ic-4175496634/
http://www.linuxquestions.org/questi...5D-4175503589/
http://www.linuxquestions.org/questi...ad-4175500161/

What if you run out of memory while running the script? And after it fails, memory is free again.

I think you need to add debug to that script and diagnose it a bit further.

How can i add debug to find the cause? I used "set -ex" in the script and executed it with "2>/dev/null" suffix + without it and i not spotted any fork error like mentioned, but it appears only when executed like ./scriptname or /bin/sh scriptname etc.

I provide usually all details that seems to be important for me, i you need more, tell me which commands to execute.

yes, CentOS 6.x

I just execute the script (as i mentioned above in this post of mine), i provided link to the script details in my first post. And i provided only error that appears as a result of the execution (the Unable to fork one).

Thank You all

----
PS: here are some details that may/may not be related:
# cat /proc/sys/kernel/pid_max && cat /proc/sys/kernel/threads-max
1048576
383064

# ulimit -a

core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 191532
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 191532
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

I'm assuming typo in that you used "-vx" to enable more debug as opposed to -ex which I've just not heard of before.

I'd consider a couple of things:

Not use the 2>/dev/null suffix, I think that will just output to nowhere.

Instead use something like:This will output stdout and stderr into the output.log file, so you can see all information in this log file.

After each of your loops in that script, put in a pause to allow the script to stop where you can check to see how much system resources are used by each phase. The problem is, either one or more of those loops may be causing this, an outcome such as an attempt to perform the loop internal operation on a non-found result, or the final command to create the web page may be the problem. But it's worth checking it all step by step.Note that all of these operations you can do at the command line. You can perform each of those loops to verify that they work, and then perform the final web page command to verify that, that works. I'd verify each of the loops and get to a point where I was sure the script would run up to the last point, and then maybe exit and do the web page command manually to see if that is the problem.

Just a few suggestions, I do not believe this will be an easy debug process, especially if the problem occurs and then doesn't occur. I'm not sure if it happens everytime you run this or sporadically.

Good luck!

You were told how to do this in post #2, the very first reply, where you were also asked for the exact error message. You were also asked in post #4 to run the commands in the script one at a time, to see what the results were. Did you do either?
Yet you didn't tell us what version/distro of Linux, running on what kind of hardware what architecture, what (if ANY) troubleshooting you've done so far, what else is running on that server, how many users, does that script execute through CRON or do you run it manually, etc. In short, next to ZERO that is useful in helping diagnose any problem.

Again, after three years, you should be able to do basic troubleshooting. Such details are covered in the "Question Guidelines" link in my posting signature, that you have been directed to numerous times.
No, you said "I am using a bash script like this", we had to assume that was the script you downloaded, although what you wrote implied you were using a SIMILAR script.

As rtmistler pointed out, you're not getting errors by redirecting output to /dev/null, because obviously that goes NOWHERE. How long have you been running this script, and what is the purpose of running it? Has it EVER worked successfully?

Again, try just running the commands of that script one at a time, manually, from the command line, and see where the error is coming from.

I suspect a lack of disk space!!!

@rtmistler

Thank You for ideas, you were right in the resulting logfile i found that error to be actully output of one of the VPSs, not dedi itself. The VPS had not enough ram previously and was stalled/crashed.

Also good idea regarding pausing the script and then monitoring RAM usage etc.

@dave@burn-it.co.uk

Thx for the idea

---
---> SOLVED

Glad you solved it. :)

In Thread Tools you can choose to mark the thread as Solved. It helps future seekers to know it was resolved.

-RT