LinuxQuestions.org
Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 08-22-2011, 05:39 PM   #1
locus84
LQ Newbie
 
Registered: Apr 2011
Posts: 9

Rep: Reputation: 0
Unhappy How to access overloaded linux machine


When my linux server is overloaded how can I fix the issue if I can't run any monitoring program like top or iostat?

I tried running 'nice -19 top' and waited for an hour for top to load and when it didn't load after an hour, I was forced to restart the machine.

Is there a way to make sure there is always some CPU/IO/Memory bandwidth left for monitoring programs to run?
 
Old 08-22-2011, 05:59 PM   #2
pljvaldez
LQ Guru
 
Registered: Dec 2005
Location: Somewhere on the String
Distribution: Debian Wheezy (x86)
Posts: 6,094

Rep: Reputation: 271Reputation: 271Reputation: 271
If it's a home server, usually what I try to do is telinit 1 to end all non-vital processes and return to single user mode. Then start diagnosing the issue. My problem has usually been disk space related. My issues were related to /var/log filling up. So I finally just moved it to it's own partition. Now when it fills, I get errors, but nothing catastrophic.

If it was a production server, I don't know what your procedures would be (since I'm not in the IT industry).
 
Old 08-22-2011, 06:17 PM   #3
locus84
LQ Newbie
 
Registered: Apr 2011
Posts: 9

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by pljvaldez View Post
If it's a home server, usually what I try to do is telinit 1 to end all non-vital processes and return to single user mode. Then start diagnosing the issue. My problem has usually been disk space related. My issues were related to /var/log filling up. So I finally just moved it to it's own partition. Now when it fills, I get errors, but nothing catastrophic.

If it was a production server, I don't know what your procedures would be (since I'm not in the IT industry).
The problem is that it's a production server that hosts remote X desktop connections. Usually the problem is that somebody tries to view a huge compressed file through GUI, or tries to read a huge text file using an editor that has a GUI, or do some other stupid unpredictable thing that eats up the whole memory or the IO bandwidth.
 
Old 08-22-2011, 06:29 PM   #4
pljvaldez
LQ Guru
 
Registered: Dec 2005
Location: Somewhere on the String
Distribution: Debian Wheezy (x86)
Posts: 6,094

Rep: Reputation: 271Reputation: 271Reputation: 271
Maybe you could create a script that runs at boot time with a high nice value that would run in the background and automatically kill processes that exceed X amount of bandwidth or memory? Or at least start top at boot in the background and write values to a file once in a while???

EDIT: I just blundered across this which may let you set some conditions for forced exit of processes.
 
Old 08-22-2011, 07:03 PM   #5
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,258

Rep: Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328
This may be useful http://linux.die.net/man/5/limits.conf.

As implied above, you'll need to set some monitoring up to run before it happens if its that bad.
You might get better response if you can login at the physical console; in fact if its in a locked room/cage, leave the console up so you can just start typing.
 
Old 08-23-2011, 11:40 AM   #6
locus84
LQ Newbie
 
Registered: Apr 2011
Posts: 9

Original Poster
Rep: Reputation: 0
I guess the best way would be to have a monitoring program running with high priority at all time. thx guys.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] How do I Remote access a Linux machine from a other Linux/Windows machine unkn(0)wn Linux - Newbie 6 08-27-2011 08:35 AM
[SOLVED] How can i access Windows machine through my Linux machine? manickaselvam Linux - Newbie 5 08-04-2010 12:24 AM
How to access files on XP machine from Linux machine john test Linux - Newbie 8 03-09-2009 11:43 AM
Firewall being overloaded by linux box - any ideas? wdawson Linux - Networking 2 12-22-2005 09:14 PM
how to access internet from a linux machine via a linux machine as gateway b0nd Linux - Newbie 5 03-21-2005 12:59 PM


All times are GMT -5. The time now is 12:41 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration