LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 01-11-2019, 08:38 AM   #1
ahramkoo
LQ Newbie
 
Registered: Jan 2019
Posts: 3

Rep: Reputation: Disabled
After e2fsck, CPU wait I/O are increaing


Hi
TESTDB1 has DB storage and backup storage.
I add backup disk to backup filesystem.
I don't know why, but server want to e2fsck.
OS version is Red Hat Enterprise Linux Server release 6.4 (Santiago).


[root@TESTDB1 bin]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda3 58G 19G 36G 35% /
tmpfs 32G 371M 32G 2% /dev/shm
/dev/sda1 194M 58M 127M 32% /boot
/dev/sda2 97G 3.4G 88G 4% /data001
/dev/sda6 105G 20G 80G 20% /data002
/dev/mapper/UNT319_VG01-engnlv01
50G 38G 9.3G 81% /engn001
/dev/mapper/UNT319_VG01-logslv01
50G 5.5G 42G 12% /logs001
/dev/mapper/UNT319_VG01-bkuplv01
99G 45G 49G 49% /bkup001
/dev/mapper/SIMPANA_VG-SIMPANA_LV
7.3T 6.7T 170G 98% /simpana_bk
[root@TESTDB1 bin]$ umount /simpana_bk
[root@TESTDB1 bin]$ resize2fs /dev/mapper/SIMPANA_VG-SIMPANA_LV
resize2fs 1.41.12 (17-May-2010)
Please run 'e2fsck -f /dev/mapper/SIMPANA_VG-SIMPANA_LV' first.

[root@TESTDB1 bin]$ e2fsck -f /dev/mapper/SIMPANA_VG-SIMPANA_LV
e2fsck 1.41.12 (17-May-2010)
Pass 1: Checking inodes, blocks, and sizes


Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information

/dev/mapper/SIMPANA_VG-SIMPANA_LV: 13585/491593728 files (2.9% non-contiguous), 1823669136/1966371840 blocks
[root@TESTDB1 bin]$ resize2fs /dev/mapper/SIMPANA_VG-SIMPANA_LV
resize2fs 1.41.12 (17-May-2010)
Resizing the filesystem on /dev/mapper/SIMPANA_VG-SIMPANA_LV to 2097443840 (4k) blocks.
The filesystem on /dev/mapper/SIMPANA_VG-SIMPANA_LV is now 2097443840 blocks long.
[root@TESTDB1 bin]$ mount /simpana_bk
[root@TESTDB1 bin]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda3 58G 19G 36G 35% /
tmpfs 32G 371M 32G 2% /dev/shm
/dev/sda1 194M 58M 127M 32% /boot
/dev/sda2 97G 3.4G 88G 4% /data001
/dev/sda6 105G 20G 80G 20% /data002
/dev/mapper/UNT319_VG01-engnlv01
50G 38G 9.3G 81% /engn001
/dev/mapper/UNT319_VG01-logslv01
50G 5.5G 42G 12% /logs001
/dev/mapper/UNT319_VG01-bkuplv01
99G 45G 49G 49% /bkup001
/dev/mapper/SIMPANA_VG-SIMPANA_LV
7.7T 6.7T 637G 92% /simpana_bk



After resizing filesystem, CPU wait I/O is 30% increasing and sda5(swap) busy % keep 100% utilization during 12 hours
(Before those utilization are lower than 10%)
After was instance restart,strangely... I/O and swap busy(%) are 0%

e2fsck has bug or not???
 
Old 01-11-2019, 01:41 PM   #2
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,831
Blog Entries: 15

Rep: Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669
Is the new disk you added to Volume Group, SIMPANA_VG, the same kind as the existing ones? e.g. If the existing ones are SAN disks is the new one from the same SAN disk array and presented via the same SAN switches? Dissimilar disks could cause issues as the entire VG may only run at the speed of its slowest disk.

What is your entry for simpana_bk in /etc/fstab? Is this an ext2, ext3 or ext4 disk there? ext2 is not journaled so shouldn't really be used. ext3 had odd performance issues in some situations (such as deleting all files or running fsck). Ideally you'd want to run ext4 as it performs very well on RHEL6/RHEL7.

Are any (or all of the disks) in the VG under multipath control (e.g. native Linux multipathd, EMC PowerPath)?
 
Old 01-11-2019, 09:36 PM   #3
berndbausch
LQ Addict
 
Registered: Nov 2013
Location: Tokyo
Distribution: Mostly Ubuntu and Centos
Posts: 6,316

Rep: Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002
Quote:
Originally Posted by ahramkoo View Post
After was instance restart,strangely... I/O and swap busy(%) are 0%
So ... what is your problem?
Quote:
e2fsck has bug or not???
I am sure it has many bugs, but since it didn't change anything on your filesystem, I have strong doubts that the temporary increase in CPU wait time was related.

Let me speculate: Since the volume size has increased, filesystem or LVM code (is this a RAID volume?) saw an opportunity to rebalance some data, which caused a temporary increase in disk access.

Of course, this is just guessing. Since your system looks good now, it's impossible to investigate what happened before the reboot.

Next time you post something, please put commands and their input and output between code tags for better readability.

Last edited by berndbausch; 01-11-2019 at 09:37 PM. Reason: added much needed lecture about code tags
 
Old 01-12-2019, 12:13 AM   #4
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: Rocky Linux
Posts: 4,783

Rep: Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214
Sounds like it might have been the ext4 lazy init proceeding in the background when the expanded filesystem was first mounted. I'm not certain this happens for a resize (as opposed to creating a new filesystem), but it sounds likely. Examining the filesystem flags while the unusual activity was happening would be the only way to be sure.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
After e2fsck, CPU wait I/O are increaing T.T ahramkoo Linux - Server 3 01-11-2019 08:10 PM
How to get physical CPU count, cores per CPU, and CPU type on RHEL and CentOS anon091 Linux - Server 7 01-08-2015 10:57 AM
[SOLVED] High CPU load, but low CPU usage (high idle CPU) baffy Linux - Newbie 5 03-13-2013 09:24 AM
Cpu fan stuck to the cpu, how do I get the cpu out? abefroman Linux - Hardware 16 09-04-2009 12:47 AM
I download then wait and download and wait... jsheffie SUSE / openSUSE 1 11-04-2005 04:43 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 08:51 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration