LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 05-30-2016, 04:54 AM   #1
postcd
Member
 
Registered: Oct 2013
Posts: 448

Rep: Reputation: Disabled
"Unable to fork: Cannot allocate memory" even 10GB RAM free


Hello,

on a dedicated server i am using a bash script like this: https://github.com/slrslr/linux-bash...slogscanner.sh

I believe it not returned any errors "Unable to fork: Cannot allocate memory" when it was executed.
But now, when i execute it, i see this "Cannot allocate memory" error.

I do free -m before it and after it and i see there is plenty of free RAM (when i not count cache):



it is not first time i am having trouble with this error, so it probably is not just "bad" commands executed from the bash script.

Please what should i do to discover the cause, why this happens?
 
Old 05-30-2016, 06:12 AM   #2
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 11,127

Rep: Reputation: 3322Reputation: 3322Reputation: 3322Reputation: 3322Reputation: 3322Reputation: 3322Reputation: 3322Reputation: 3322Reputation: 3322Reputation: 3322Reputation: 3322
you can put set -xv at the beginning of the script. And you need to give more info, otherwise we will not be able to help you. At least the complete error message, the real command which failed...
I would say it is not the bash script itself, but a special command inside...
 
1 members found this post helpful.
Old 05-30-2016, 10:13 AM   #3
Emerson
LQ Guru
 
Registered: Nov 2004
Location: Saint Amant, Acadiana
Distribution: Gentoo ~arch
Posts: 6,311

Rep: Reputation: Disabled
Is it 64-bit installation?
 
Old 05-30-2016, 10:33 AM   #4
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 20,399

Rep: Reputation: 4958Reputation: 4958Reputation: 4958Reputation: 4958Reputation: 4958Reputation: 4958Reputation: 4958Reputation: 4958Reputation: 4958Reputation: 4958Reputation: 4958
Quote:
Originally Posted by postcd View Post
Hello,
on a dedicated server i am using a bash script like this: https://github.com/slrslr/linux-bash...slogscanner.sh

I believe it not returned any errors "Unable to fork: Cannot allocate memory" when it was executed. But now, when i execute it, i see this "Cannot allocate memory" error. I do free -m before it and after it and i see there is plenty of free RAM (when i not count cache):

it is not first time i am having trouble with this error, so it probably is not just "bad" commands executed from the bash script.

Please what should i do to discover the cause, why this happens?
You say "dedicated server"...what ELSE is running on the server? As pan64 said (and others have also, over the years you've been here), you need to provide details.

Secondly, what have you actually done/tried to see what's going on? What (if ANY), troubleshooting have you done? Did you try running the commands in that script one at a time, to see if any of them cause the problem? Watch memory when you start it up? ANYTHING???

Seems odd that after three years asking such questions, you can't think of any ideas on how to troubleshoot anything:
http://www.linuxquestions.org/questi...ng-4175486469/
http://www.linuxquestions.org/questi...ic-4175496634/
http://www.linuxquestions.org/questi...5D-4175503589/
http://www.linuxquestions.org/questi...ad-4175500161/

Last edited by TB0ne; 05-30-2016 at 04:23 PM.
 
Old 05-30-2016, 11:00 AM   #5
rtmistler
Moderator
 
Registered: Mar 2011
Location: MA, USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 6,892
Blog Entries: 12

Rep: Reputation: 2474Reputation: 2474Reputation: 2474Reputation: 2474Reputation: 2474Reputation: 2474Reputation: 2474Reputation: 2474Reputation: 2474Reputation: 2474Reputation: 2474
What if you run out of memory while running the script? And after it fails, memory is free again.

I think you need to add debug to that script and diagnose it a bit further.
 
Old 05-31-2016, 02:41 AM   #6
postcd
Member
 
Registered: Oct 2013
Posts: 448

Original Poster
Rep: Reputation: Disabled
Thumbs up

Quote:
Originally Posted by rtmistler View Post
What if you run out of memory while running the script? And after it fails, memory is free again.
I think you need to add debug to that script and diagnose it a bit further.
How can i add debug to find the cause? I used "set -ex" in the script and executed it with "2>/dev/null" suffix + without it and i not spotted any fork error like mentioned, but it appears only when executed like ./scriptname or /bin/sh scriptname etc.

Quote:
Originally Posted by TB0ne View Post
you need to provide details.
I provide usually all details that seems to be important for me, i you need more, tell me which commands to execute.

Quote:
Originally Posted by Emerson View Post
Is it 64-bit installation?
yes, CentOS 6.x

Quote:
Originally Posted by pan64 View Post
At least the complete error message, the real command which failed...
I just execute the script (as i mentioned above in this post of mine), i provided link to the script details in my first post. And i provided only error that appears as a result of the execution (the Unable to fork one).

Thank You all

----
PS: here are some details that may/may not be related:
# cat /proc/sys/kernel/pid_max && cat /proc/sys/kernel/threads-max
1048576
383064

# ulimit -a

core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 191532
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 191532
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

Last edited by postcd; 05-31-2016 at 02:47 AM.
 
Old 05-31-2016, 06:56 AM   #7
rtmistler
Moderator
 
Registered: Mar 2011
Location: MA, USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 6,892
Blog Entries: 12

Rep: Reputation: 2474Reputation: 2474Reputation: 2474Reputation: 2474Reputation: 2474Reputation: 2474Reputation: 2474Reputation: 2474Reputation: 2474Reputation: 2474Reputation: 2474
Quote:
Originally Posted by postcd View Post
How can i add debug to find the cause? I used "set -ex" in the script and executed it with "2>/dev/null" suffix + without it and i not spotted any fork error like mentioned, but it appears only when executed like ./scriptname or /bin/sh scriptname etc.
I'm assuming typo in that you used "-vx" to enable more debug as opposed to -ex which I've just not heard of before.

I'd consider a couple of things:

Not use the 2>/dev/null suffix, I think that will just output to nowhere.

Instead use something like:
Code:
./vpslogscanner.sh >> output.log 2>&1
This will output stdout and stderr into the output.log file, so you can see all information in this log file.

After each of your loops in that script, put in a pause to allow the script to stop where you can check to see how much system resources are used by each phase. The problem is, either one or more of those loops may be causing this, an outcome such as an attempt to perform the loop internal operation on a non-found result, or the final command to create the web page may be the problem. But it's worth checking it all step by step.
Code:
read -p "Press [ENTER] to continue ..."
Note that all of these operations you can do at the command line. You can perform each of those loops to verify that they work, and then perform the final web page command to verify that, that works. I'd verify each of the loops and get to a point where I was sure the script would run up to the last point, and then maybe exit and do the web page command manually to see if that is the problem.

Just a few suggestions, I do not believe this will be an easy debug process, especially if the problem occurs and then doesn't occur. I'm not sure if it happens everytime you run this or sporadically.

Good luck!
 
Old 05-31-2016, 08:36 AM   #8
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 20,399

Rep: Reputation: 4958Reputation: 4958Reputation: 4958Reputation: 4958Reputation: 4958Reputation: 4958Reputation: 4958Reputation: 4958Reputation: 4958Reputation: 4958Reputation: 4958
Quote:
Originally Posted by postcd View Post
How can i add debug to find the cause? I used "set -ex" in the script and executed it with "2>/dev/null" suffix + without it and i not spotted any fork error like mentioned, but it appears only when executed like ./scriptname or /bin/sh scriptname etc.
You were told how to do this in post #2, the very first reply, where you were also asked for the exact error message. You were also asked in post #4 to run the commands in the script one at a time, to see what the results were. Did you do either?
Quote:
I provide usually all details that seems to be important for me, i you need more, tell me which commands to execute.
Yet you didn't tell us what version/distro of Linux, running on what kind of hardware what architecture, what (if ANY) troubleshooting you've done so far, what else is running on that server, how many users, does that script execute through CRON or do you run it manually, etc. In short, next to ZERO that is useful in helping diagnose any problem.

Again, after three years, you should be able to do basic troubleshooting. Such details are covered in the "Question Guidelines" link in my posting signature, that you have been directed to numerous times.
Quote:
yes, CentOS 6.x

I just execute the script (as i mentioned above in this post of mine), i provided link to the script details in my first post. And i provided only error that appears as a result of the execution (the Unable to fork one).
No, you said "I am using a bash script like this", we had to assume that was the script you downloaded, although what you wrote implied you were using a SIMILAR script.

As rtmistler pointed out, you're not getting errors by redirecting output to /dev/null, because obviously that goes NOWHERE. How long have you been running this script, and what is the purpose of running it? Has it EVER worked successfully?

Again, try just running the commands of that script one at a time, manually, from the command line, and see where the error is coming from.
 
Old 06-04-2016, 07:39 AM   #9
dave@burn-it.co.uk
Member
 
Registered: Sep 2011
Distribution: Puppy
Posts: 601

Rep: Reputation: 171Reputation: 171
I suspect a lack of disk space!!!
 
Old 06-05-2016, 08:08 AM   #10
postcd
Member
 
Registered: Oct 2013
Posts: 448

Original Poster
Rep: Reputation: Disabled
@rtmistler

Thank You for ideas, you were right in the resulting logfile i found that error to be actully output of one of the VPSs, not dedi itself. The VPS had not enough ram previously and was stalled/crashed.

Also good idea regarding pausing the script and then monitoring RAM usage etc.

@dave@burn-it.co.uk


Thx for the idea

---
---> SOLVED
 
Old 06-05-2016, 12:03 PM   #11
rtmistler
Moderator
 
Registered: Mar 2011
Location: MA, USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 6,892
Blog Entries: 12

Rep: Reputation: 2474Reputation: 2474Reputation: 2474Reputation: 2474Reputation: 2474Reputation: 2474Reputation: 2474Reputation: 2474Reputation: 2474Reputation: 2474Reputation: 2474
Quote:
Originally Posted by postcd View Post
@rtmistler

Thank You for ideas, you were right in the resulting logfile i found that error to be actully output of one of the VPSs, not dedi itself. The VPS had not enough ram previously and was stalled/crashed.

Also good idea regarding pausing the script and then monitoring RAM usage etc.

@dave@burn-it.co.uk


Thx for the idea

---
---> SOLVED
Glad you solved it.

In Thread Tools you can choose to mark the thread as Solved. It helps future seekers to know it was resolved.

-RT
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Centos 7 unstable with Nouveau graphics card driver "cannot allocate memory" needsleep Linux - Hardware 1 12-31-2014 08:35 AM
Problem with GIMP: error "failed to fork - cannot allocate memory" heffo_j Linux - Software 5 05-24-2009 12:19 AM
"bash: fork: Cannot allocate memory" error guarriman Linux - General 1 04-05-2007 12:46 PM
"Failed to allocate mem resource" with Broadcom BCM5704 and 8GB of RAM Xaox Linux - Hardware 5 02-15-2006 03:52 PM
ndiswrapper troubles: "Failed to allocate DMA coherent memory." Oodini Linux - Wireless Networking 1 01-12-2005 12:20 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 03:42 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration