LinuxQuestions.org
Latest LQ Deal: Linux Power User Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 08-31-2010, 08:15 AM   #1
TheIndependentAquarius
Senior Member
 
Registered: Dec 2008
Posts: 4,680
Blog Entries: 29

Rep: Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917
Angry Too many forks, pid got used, we're confused


OpenSuse 11.2
LXDE


A very strange thing happened just now.

I was in the GUI mode.
The processes started by me on my computer were:
1. OpenOffice word.
2. Firefox.
3. Konsole.
4. A Pdf and a Folder.


I started feeling slowly that the system was getting slow in responding.

I did:
Code:
anisha@linux-uitj:~> ps -el
...
...
0 S  1000   300 32767  0  80   0 -  3106 pipe_w ?        00:00:00 sh            
0 S  1000   301 32767  0  80   0 -  1228 pipe_w ?        00:00:00 sed           
1 S  1000   304   300  0  80   0 -  3106 wait   ?        00:00:00 sh            
0 S  1000   305   304  0  80   0 -  3106 pipe_w ?        00:00:00 sh            
0 S  1000   307   304  0  80   0 -  1228 pipe_w ?        00:00:00 sed           
1 S  1000   309   305  0  80   0 -  3106 wait   ?        00:00:00 sh            
0 S  1000   310   309  0  80   0 -  3106 pipe_w ?        00:00:00 sh                                     
0 S  1000   312   309  0  80   0 -  1228 pipe_w ?        00:00:00 sed                                    
1 S  1000   314   310  0  80   0 -  3106 wait   ?        00:00:00 sh                                     
0 S  1000   315   314  0  80   0 -  3106 pipe_w ?        00:00:00 sh                                     
1 S  1000   318   315  0  80   0 -  3106 wait   ?        00:00:00 sh                                     
0 S  1000   319   314  0  80   0 -  1228 pipe_w ?        00:00:00 sed                                    
0 S  1000   320   318  0  80   0 -  3106 pipe_w ?        00:00:00 sh                                     
0 S  1000   322   318  0  80   0 -  1228 pipe_w ?        00:00:00 sed                                    
1 S  1000   324   320  0  80   0 -  3106 wait   ?        00:00:00 sh                                     
0 S  1000   325   324  0  80   0 -  3106 pipe_w ?        00:00:00 sh
I have presented a short list here, actual list was very long !!!!

I pressed Ctrl+Alt+Backspace to get out !!!!

It too 3 minutes and I noticed a message on the non-GUI mode :
Quote:
Too many forks, pid got used, we're confused
What must be the cause, any ideas ???????????
 
Old 08-31-2010, 08:40 AM   #2
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389
Hi,

Looking at the mother and child pids in the output you posted it looks like and endless loop was started that ate all the available pid numbers the system has.

WARNING! The following example may crash your computer if executed.

:(){ :|:& };:

The above is a known Bash fork() bomb that can produce something similar.

Most of the time it is very hard to do anything useful after something like this happens (you won't be able to get any commands to execute).

Not much to go on, but maybe it clarifies the behaviour a bit.
 
1 members found this post helpful.
Old 08-31-2010, 09:02 AM   #3
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,354
Blog Entries: 55

Rep: Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541
BTW, posting something like '/bin/ps U 1000 xfo pid,ppid,uid,cmd --sort=ppid' (or 'pstree -Apuh `id -gn`') would provide more process info making things easier to read.
 
Old 08-31-2010, 09:17 AM   #4
TheIndependentAquarius
Senior Member
 
Registered: Dec 2008
Posts: 4,680
Blog Entries: 29

Original Poster
Rep: Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917
Quote:
Originally Posted by unSpawn View Post
BTW, posting something like '/bin/ps U 1000 xfo pid,ppid,uid,cmd --sort=ppid' (or 'pstree -Apuh `id -gn`') would provide more process info making things easier to read.
Thanks for replying !

But during that incident I could barely manage to save this output on a text file, I couldn't do anything more as mouse clicks were not responding !!

I had to hard restart the computer, so I couldn't execute pstree !
 
Old 08-31-2010, 09:21 AM   #5
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389
Quote:
Originally Posted by anishakaul View Post
But during that incident I could barely manage to save this output on a text file, I couldn't do anything more as mouse clicks were not responding !!

I had to hard restart the computer, so I couldn't execute pstree !
That is a classic symptom of what I described in my previous post, be glad you could at least do that command and save the output
 
Old 08-31-2010, 09:28 AM   #6
TheIndependentAquarius
Senior Member
 
Registered: Dec 2008
Posts: 4,680
Blog Entries: 29

Original Poster
Rep: Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917
Quote:
Originally Posted by druuna View Post
WARNING! The following example may crash your computer if executed.

:(){ :|:& };:

The above is a known Bash fork() bomb that can produce something similar.
Thanks for replying !

I was confused and worried because I had not manually typed any such command !!

I should have checked history command !! but how could I ???
 
Old 08-31-2010, 09:39 AM   #7
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389
Hi,

Unless you remove your history during shut-down/boot it is still there the next time you boot up your machine......

The bash fork bomb I mentioned is just an example (it is fast! and you can probably do nothing of use any more within seconds if not faster) and I'm pretty sure that specific example wasn't the cause.
 
1 members found this post helpful.
Old 08-31-2010, 09:44 AM   #8
TheIndependentAquarius
Senior Member
 
Registered: Dec 2008
Posts: 4,680
Blog Entries: 29

Original Poster
Rep: Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917
Quote:
Originally Posted by druuna View Post
Unless you remove your history during shut-down/boot it is still there the next time you boot up your machine......
Yes, how could I forget that Perhaps because of the goose bumps I am still having !
 
Old 08-31-2010, 09:59 AM   #9
TheIndependentAquarius
Senior Member
 
Registered: Dec 2008
Posts: 4,680
Blog Entries: 29

Original Poster
Rep: Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917
Quote:
Originally Posted by druuna View Post
:(){ :|:& };:

The above is a known Bash fork() bomb that can produce something similar.
Right now I am at home not in the office, out of curiosity, I just tried the above command thinking I stop it with ctrl+c

But the command was over-smart it ran in the background and didn't give me even a chance to type the kill command
 
Old 08-31-2010, 10:07 AM   #10
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389
Hi,

Have a look here Understanding Bash fork() bomb

You're not up against 1 process. 1 becomes 2 -> 4 -> 8 -> 16 etc
 
1 members found this post helpful.
Old 09-01-2010, 03:21 AM   #11
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,354
Blog Entries: 55

Rep: Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541
Quote:
Originally Posted by anishakaul View Post
during that incident I could barely manage to save this output on a text file, I couldn't do anything more as mouse clicks were not responding !!
I had to hard restart the computer, so I couldn't execute pstree !
Understandable, yes. I'm only saying that with the right information it would be easier to find out what on your system caused it. Right now the only thing one can say is that applying ulimits is always good. You did apply those, right?
 
1 members found this post helpful.
Old 09-01-2010, 05:04 AM   #12
TheIndependentAquarius
Senior Member
 
Registered: Dec 2008
Posts: 4,680
Blog Entries: 29

Original Poster
Rep: Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917
Quote:
Originally Posted by druuna View Post
I'm pretty sure that specific example wasn't the cause.
I checked the history command, there was nothing suspicious.

Then I suddenly remembered that there was one more process opened by me and that was KDevelop ! I ran commands to compile our project there. There is a file named "Libtool" which needs to be modified each time ./configure runs other wise the system hangs !!

That time I must not have done the same resulting in system hanging.

I confirmed this behavior today by deliberately missing the above step in compilation and it resulted in the same behavior as yesterday !

And that fork bomb explanation was nice in the link, thanks for that.

Quote:
Originally Posted by unSpawn View Post
Right now the only thing one can say is that applying ulimits is always good. You did apply those, right?
What are ulimits ? I really don't know ! Perhaps I should RTFM.
 
Old 09-01-2010, 05:42 AM   #13
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389
Hi,

@anishakaul

Here you are: ulimit and sysctl - Linux Howto's

Hope this helps.
 
Old 09-01-2010, 05:48 AM   #14
TheIndependentAquarius
Senior Member
 
Registered: Dec 2008
Posts: 4,680
Blog Entries: 29

Original Poster
Rep: Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917
Quote:
Originally Posted by druuna View Post
Here you are: ulimit and sysctl - Linux Howto's

Hope this helps.


You just pointed me to the link I was already reading, thanks anyway !

I tried this:
Code:
anisha@linux-uitj:~> ulimit -u 30
bash: fork: retry: Resource temporarily unavailable
bash: fork: retry: Resource temporarily unavailable
bash: fork: retry: Resource temporarily unavailable
bash: fork: retry: Resource temporarily unavailable
^C^C^C

^Cbash: fork: Resource temporarily unavailable

bash: fork: retry: Resource temporarily unavailable

bash: fork: retry: Resource temporarily unavailable
^Cbash: fork: retry: Resource temporarily unavailable
What does this mean ? and the Ctrl+c was not working !

The following also resulted in the same ! I changed 30 to 3 here !
Code:
anisha@linux-uitj:~> ulimit -u 3
bash: fork: retry: Resource temporarily unavailable
bash: fork: retry: Resource temporarily unavailable
bash: fork: retry: Resource temporarily unavailable
^C^Cbash: fork: retry: Resource temporarily unavailable

Last edited by TheIndependentAquarius; 09-01-2010 at 05:51 AM.
 
Old 09-01-2010, 06:12 AM   #15
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389
Hi,

This tool (ulimit) will not necessarily stop the initial process, it will limit the use of resource for a user or a specific terminal.

The message you get (also mentioned in the link I gave) tells you that a limit is reached. And even though that specific terminal might become unresponsive, other terminals and the machine itself are not compromised by, in this example, a bash fork bomb.

You could compare it with making a building fire proof: You can have a blazing fire in one room (which is destroyed, most of the time) but the adjacent rooms and building itself survive.

If I set max user processes to 30 and execute the fork bomb I see this:
Code:
$ ulimit -u 30
$ :(){ :|:& };:
bash: fork: Resource temporarily unavailable
bash: fork: Resource temporarily unavailable
[1] 17255
[1]+  Terminated              : | :
And, in this case, the process is gone as well. I do believe the termination depends a bit on which process creates the fork(s). I'm not sure, but I would understand if the following scenario where true: Process A forks a sub-process B. Process B starts forking multiple processes (loop, intentional or not). Ulimit kills Process B and its children when reaching the limit, but process A remains.

Hope this helps.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: A Tale of Two Forks LXer Syndicated Linux News 0 11-26-2008 05:41 AM
What is the more easier way to check the pid and kill the pid cmx08 Linux - General 5 09-09-2008 10:57 PM
kill pid.... won't work with 'pid' variable given.. sachitha Programming 6 03-06-2006 07:48 PM
/var/run/[XXX].pid - Tcl pid code liguorir Linux - Software 1 05-20-2004 10:32 PM
ERROR: Couldn't write pid to pid file lawrencegoodman Linux - Newbie 2 02-13-2004 08:05 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 07:06 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration