LinuxQuestions.org
Review your favorite Linux distribution.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 08-14-2018, 01:45 AM   #1
saiwang
LQ Newbie
 
Registered: Aug 2018
Posts: 4

Rep: Reputation: Disabled
why data was not updated in /var/log/sa/sadd file suddenly


I was told by my server admin that the server CPU reached 100% yesterday evening, today I checked the sa file of yeserday under /var/log/sa/, and I found there was no data suddenly in the file when CPU issue happened and sar data resumed after this server was reboot

May I get to know why no sar data updated for more than 1 hour until the server reboot? does Linus stop updating sar file when CPU is high? Thanks.

sar data in the file:

12:40:02 PM all 7.27 0.00 1.98 0.04 0.00 90.71
12:50:01 PM all 7.61 0.00 2.00 0.04 0.00 90.35
01:00:01 PM all 7.39 0.00 1.72 0.03 0.00 90.86
01:10:01 PM all 8.11 0.00 2.51 0.03 0.00 89.35
01:20:01 PM all 7.97 0.00 2.38 0.04 0.00 89.61
01:30:01 PM all 7.44 0.00 1.89 0.04 0.00 90.63
01:40:01 PM all 8.39 0.00 2.63 0.09 0.00 88.89
01:50:01 PM all 7.81 0.00 2.17 0.07 0.00 89.95
02:00:01 PM all 7.74 0.00 2.23 0.04 0.00 89.99
Average: all 7.36 0.00 1.80 0.04 0.00 90.81

03:10:26 PM LINUX RESTART

03:20:01 PM CPU %user %nice %system %iowait %steal %idle
03:30:01 PM all 2.36 0.00 1.91 0.09 0.00 95.64
03:40:01 PM all 1.91 0.00 1.20 0.14 0.00 96.75
03:50:01 PM all 1.82 0.00 1.08 0.04 0.00 97.05
04:00:01 PM all 1.85 0.00 1.15 3.18 0.00 93.82
04:10:01 PM all 1.85 0.00 1.02 7.17 0.00 89.96
04:20:02 PM all 1.56 0.00 0.77 0.03 0.00 97.64

Last edited by saiwang; 08-14-2018 at 01:46 AM.
 
Old 08-14-2018, 07:58 AM   #2
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,831
Blog Entries: 15

Rep: Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669
Most likely because whatever made your CPU go to 100% prevent sar from being able to run.

When you say you rebooted do you mean you did a warm reboot or did you have to power off the system because a warm boot couldn't be done?

It happens on occasion that so many resources are used the system will be totally unresponsive requiring a hard boot (power cycle). In such occasions the same thing that prevents you from doing other tasks will prevent system jobs from running.

Another possibility is that the filesystem was full and rebooting cleaned up some temporary file that had filled it up.

Nothing in the output you put here shows any huge decrease of CPU idle by 2 PM. Presumably at next 2:10 PM sample whatever happened that sent your CPU to 100% had already occurred preventing that sample. Look at the rest of your sadd to see if you can determine other issues (e.g. memory or file table) growth that was increasing rapidly. The CPU going to a 100% may have been a symptom of the other issue.
 
Old 08-28-2018, 01:23 AM   #3
saiwang
LQ Newbie
 
Registered: Aug 2018
Posts: 4

Original Poster
Rep: Reputation: Disabled
thank you

Quote:
Originally Posted by MensaWater View Post
Most likely because whatever made your CPU go to 100% prevent sar from being able to run.

When you say you rebooted do you mean you did a warm reboot or did you have to power off the system because a warm boot couldn't be done?

It happens on occasion that so many resources are used the system will be totally unresponsive requiring a hard boot (power cycle). In such occasions the same thing that prevents you from doing other tasks will prevent system jobs from running.

Another possibility is that the filesystem was full and rebooting cleaned up some temporary file that had filled it up.

Nothing in the output you put here shows any huge decrease of CPU idle by 2 PM. Presumably at next 2:10 PM sample whatever happened that sent your CPU to 100% had already occurred preventing that sample. Look at the rest of your sadd to see if you can determine other issues (e.g. memory or file table) growth that was increasing rapidly. The CPU going to a 100% may have been a symptom of the other issue.
Yes, I had to power off the system because the VM was not responsible. I also checked network, memory, fd and so on, everything was normal before the incident, it happened suddenly.
my server did not run on a physical machine but a VM managed by VMWare, maybe it is VM issue.
By the way, would you tell me how to check how much memory that Linux kernel is using? is it equal to a sum up of all output of /proc/slabinfo?
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
filesystem /var file system suddenly utilizes 100% _mz Linux - Enterprise 7 07-12-2013 05:50 PM
/var/adm/sa does compressing sadd files prathamesh7478 Linux - Newbie 2 02-28-2013 07:53 PM
setting file permissions for file /var/log/Xorg.0.log mfb Linux - Security 1 07-07-2009 01:41 PM
when starting bind /var/log/messages serial isnt updated matters Slackware 15 12-19-2008 06:00 AM
Can Samhain log my entries in /var/log/secure and /var/log/mesage to a central server abefroman Linux - Software 2 04-13-2008 04:13 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 03:01 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration