LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software > Linux - Kernel
User Name
Password
Linux - Kernel This forum is for all discussion relating to the Linux kernel.

Notices


Reply
  Search this Thread
Old 11-26-2017, 01:49 AM   #1
anlob
LQ Newbie
 
Registered: Nov 2017
Location: berlin
Distribution: debian
Posts: 2

Rep: Reputation: Disabled
Kernel trace log while having heavy disk load


Hi, found some kernel trace logs in /var/log/messages while having heavy disk load.

Excerpt:
Code:
Nov 25 17:10:45 matrix kernel: [341230.560096] Chrome_SyncThre D    0 15480  14106 0x00000000
Nov 25 17:10:45 matrix kernel: [341230.560101]  ffff90aaa27d4800 ffff90a9440d6000 ffff90a940000300 ffff90aabbc18240
Nov 25 17:10:45 matrix kernel: [341230.560105]  ffff90a9548cd1c0 ffff9c778462fe60 ffffffff982038e3 ffff90aab119f8a0
Nov 25 17:10:45 matrix kernel: [341230.560109]  0000000000000246 ffff90aabbc18240 ffff90aab119f888 ffff90a940000300
Nov 25 17:10:45 matrix kernel: [341230.560113] Call Trace:
Nov 25 17:10:45 matrix kernel: [341230.560121]  [<ffffffff982038e3>] ? __schedule+0x233/0x6d0
Nov 25 17:10:45 matrix kernel: [341230.560124]  [<ffffffff98203db2>] ? schedule+0x32/0x80
Nov 25 17:10:45 matrix kernel: [341230.560136]  [<ffffffffc039a683>] ? jbd2_log_wait_commit+0x93/0x110 [jbd2]
Nov 25 17:10:45 matrix kernel: [341230.560140]  [<ffffffff97cb8e80>] ? prepare_to_wait_event+0xf0/0xf0
Nov 25 17:10:45 matrix kernel: [341230.560168]  [<ffffffffc03c0cb5>] ? ext4_sync_file+0x1a5/0x380 [ext4]
Nov 25 17:10:45 matrix kernel: [341230.560172]  [<ffffffff97e36da8>] ? do_fsync+0x38/0x60
Nov 25 17:10:45 matrix kernel: [341230.560174]  [<ffffffff97e36fdf>] ? SyS_fdatasync+0xf/0x20
Nov 25 17:10:45 matrix kernel: [341230.560178]  [<ffffffff982085bb>] ? system_call_fast_compare_end+0xc/0x9b
Nov 25 17:10:45 matrix kernel: [341230.560188] TaskSchedulerBa D    0 18159  14106 0x00000000
Nov 25 17:10:45 matrix kernel: [341230.560191]  ffff90aaa27d4800 ffff90aaa27d4800 ffff90a9428542c0 ffff90aabbc18240
Nov 25 17:10:45 matrix kernel: [341230.560195]  ffff90a9415da380 ffff9c778107fe60 ffffffff982038e3 ffff90aab119f8a0
Nov 25 17:10:45 matrix kernel: [341230.560198]  0000000000000246 ffff90aabbc18240 ffff90aab119f888 ffff90a9428542c0
Nov 25 17:10:45 matrix kernel: [341230.560202] Call Trace:
Nov 25 17:10:45 matrix kernel: [341230.560205]  [<ffffffff982038e3>] ? __schedule+0x233/0x6d0
Nov 25 17:10:45 matrix kernel: [341230.560207]  [<ffffffff98203db2>] ? schedule+0x32/0x80
Nov 25 17:10:45 matrix kernel: [341230.560214]  [<ffffffffc039a683>] ? jbd2_log_wait_commit+0x93/0x110 [jbd2]
Nov 25 17:10:45 matrix kernel: [341230.560216]  [<ffffffff97cb8e80>] ? prepare_to_wait_event+0xf0/0xf0
Nov 25 17:10:45 matrix kernel: [341230.560232]  [<ffffffffc03c0cb5>] ? ext4_sync_file+0x1a5/0x380 [ext4]
Nov 25 17:10:45 matrix kernel: [341230.560234]  [<ffffffff97e36da8>] ? do_fsync+0x38/0x60
Nov 25 17:10:45 matrix kernel: [341230.560237]  [<ffffffff97e36fdf>] ? SyS_fdatasync+0xf/0x20
Nov 25 17:10:45 matrix kernel: [341230.560240]  [<ffffffff982085bb>] ? system_call_fast_compare_end+0xc/0x9b
Nov 25 17:10:45 matrix kernel: [341230.560249] TaskSchedulerFo D    0 19115  14106 0x00000000
Nov 25 17:10:45 matrix kernel: [341230.560251]  ffff90aaa27d4800 0000000000000000 ffff90aa9f4bee00 ffff90aabbc18240
Nov 25 17:10:45 matrix kernel: [341230.560266]  ffffffff9880e500 ffff9c7783d1fe60 ffffffff982038e3 ffff90aab119f8a0
Nov 25 17:10:45 matrix kernel: [341230.560268]  0000000000000246 ffff90aabbc18240 ffff90aab119f888 ffff90aa9f4bee00
Nov 25 17:10:45 matrix kernel: [341230.560271] Call Trace:
Nov 25 17:10:45 matrix kernel: [341230.560274]  [<ffffffff982038e3>] ? __schedule+0x233/0x6d0
Nov 25 17:10:45 matrix kernel: [341230.560276]  [<ffffffff98203db2>] ? schedule+0x32/0x80
Nov 25 17:10:45 matrix kernel: [341230.560282]  [<ffffffffc039a683>] ? jbd2_log_wait_commit+0x93/0x110 [jbd2]
Nov 25 17:10:45 matrix kernel: [341230.560284]  [<ffffffff97cb8e80>] ? prepare_to_wait_event+0xf0/0xf0
Nov 25 17:10:45 matrix kernel: [341230.560297]  [<ffffffffc03c0cb5>] ? ext4_sync_file+0x1a5/0x380 [ext4]
Nov 25 17:10:45 matrix kernel: [341230.560299]  [<ffffffff97e36da8>] ? do_fsync+0x38/0x60
Nov 25 17:10:45 matrix kernel: [341230.560302]  [<ffffffff97e36fdf>] ? SyS_fdatasync+0xf/0x20
Nov 25 17:10:45 matrix kernel: [341230.560304]  [<ffffffff982085bb>] ? system_call_fast_compare_end+0xc/0x9b
No accompanying text message / explanation.
Currently I am just guessing, whit it does mean.
Does anybody know, what happened here?

Thanks & regards,
Andreas
 
Old 11-26-2017, 03:21 PM   #2
business_kid
LQ Guru
 
Registered: Jan 2006
Location: Ireland
Distribution: Slackware & Android
Posts: 8,720

Rep: Reputation: 887Reputation: 887Reputation: 887Reputation: 887Reputation: 887Reputation: 887Reputation: 887
Hello and welcome to LQ.

Code:
Currently I am just guessing, whit it does mean. Does anybody know, what happened here?
No, but allow me to join in guessing. It looks like the hard disk(s) got backed up - not swapfile, but trying to write to an ext4 partition.
JBD = Journalling Block Device
EXT4 = hard disk journalling filesystem.

More than that, it's hard to say. If there's a serious operation being performed, I'd be thinking 'get an ssd in there.' If it's gaming on your home box, ignore it. You sounded so serious, I took it it was a serious operation.
If it's a server, you may consider a raid filesystem. One of them (raid 0 or 1) stripes the data between 2 drives, optionally using a third for indexing. With ssds, you've basically caught up with the motherboard bus speed if you do that. Regular backups are of course ESSENTIAL.
 
Old 11-27-2017, 08:08 AM   #3
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 8,706
Blog Entries: 4

Rep: Reputation: 3030Reputation: 3030Reputation: 3030Reputation: 3030Reputation: 3030Reputation: 3030Reputation: 3030Reputation: 3030Reputation: 3030Reputation: 3030Reputation: 3030
You also don't tell us what version of Linux [kernel ...] you are running. Could it be possible that there's some sort of problem with the ext4 journal? Try running [font=courier]fsck[/i]font] on this drive. This doesn't look like an "Oops!" but rather some kind of voluntary traceback.

I/O will never be "heavy" from the scheduler's point-of-view, but perhaps there is an underlying filesystem problem here . . . you're "filling something up" (much hand-waving here ...) to the point where you're "running into" some problem on the drive's filesystem data structures.

Last edited by sundialsvcs; 11-27-2017 at 08:09 AM.
 
Old 11-27-2017, 06:09 PM   #4
anlob
LQ Newbie
 
Registered: Nov 2017
Location: berlin
Distribution: debian
Posts: 2

Original Poster
Rep: Reputation: Disabled
All participating processes did not report any IO error. So my guess is, that the kernel detected a race problem (ie. deadlock) which might got solved by timeout. But this is, as I said a guess. I think this kind of trace log is well known to those, who have often to do with kernel problems..
 
Old 11-27-2017, 06:40 PM   #5
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 16,001

Rep: Reputation: 2219Reputation: 2219Reputation: 2219Reputation: 2219Reputation: 2219Reputation: 2219Reputation: 2219Reputation: 2219Reputation: 2219Reputation: 2219Reputation: 2219
Possibly (probably ?) a kernel debug option turned on. On my Fedora system, there are 217 kernel config options containing the word "debug", of which 60 are enabled. Lots of possibilities for unexpected messages.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Apache heavy CPU load bretticus Linux - Networking 0 06-11-2005 03:49 PM
azureus - heavy harddrive load sonaatti Linux - Software 2 04-11-2005 06:59 AM
Xfree heavy load NetScripter Linux - Software 4 01-25-2004 05:27 AM
System very choppy while under heavy i/o load devhen Linux - General 3 09-30-2003 05:25 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software > Linux - Kernel

All times are GMT -5. The time now is 09:09 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration