LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware
User Name
Password
Slackware This Forum is for the discussion of Slackware Linux.

Notices


Reply
  Search this Thread
Old 01-07-2019, 06:12 PM   #16
khronosschoty
Member
 
Registered: Jul 2008
Distribution: Slackware
Posts: 374
Blog Entries: 2

Rep: Reputation: 171Reputation: 171

Quote:
Originally Posted by ZhaoLin1457 View Post
The issue is not about a whatever filesystem, but is a known misbehavior of the Linux kernel and certain I/O schedulers, specially of CFQ used by default also by Slackware.

Some links discussing about that:
https://blog.vacs.fr/vacs/blogs/post...-are-performed
https://bugs.launchpad.net/ubuntu/+s...ux/+bug/131094
https://www.reddit.com/r/archlinux/c...izing_desktop/
https://blog.codeship.com/linux-io-scheduler-tuning/

From what I read, the consensus is that is much better to use Deadline for SSDs and BFQ for the mechanical hard drives. And the Block Multi-Queue.

Personally, I have this added on "/etc/rc.d/rc.modules.local" - because the lack of a better earlier "local" script
Code:
# Setup the DeadLine I/O scheduler for SSDs.
echo deadline | /bin/tee /sys/block/sda/queue/scheduler 1> /dev/null 2> /dev/null

# Setup the BFQ I/O scheduler for mechanical hard drives.
/sbin/modprobe bfq

#echo bfq | /bin/tee /sys/block/sd*/queue/scheduler 1> /dev/null 2> /dev/null

echo bfq | /bin/tee /sys/block/sdb/queue/scheduler 1> /dev/null 2> /dev/null
echo bfq | /bin/tee /sys/block/sdc/queue/scheduler 1> /dev/null 2> /dev/null
and in the kernel command line
Code:
scsi_mod.use_blk_mq=1
Quote:
Originally Posted by bassmadrigal View Post
You can do this using udev rules stored in /etc/udev/rules.d and you can specify whether a disk is rotational (HDD) or not (SSD). I have the following to set my SSD to deadline.

Code:
jbhansen@craven-moorhead:~$cat /etc/udev/rules.d/55-ssd-scheduler.rules
ACTION=="add|change", KERNEL=="sd[a-z]", ATTR{queue/rotational}=="0", ATTR{queue/scheduler}="deadline"
You could modify it to set BFQ for your HDDs (the number at the beginning would dictate when it will be run... 55 seemed to be a common number for setting the scheduler -- then the file just needs to end in .rules for udev to pick it up). This would ensure that all HDDs and all SSDs will use your preferred scheduler even if you add or remove drives or whether the drive designators change.

Code:
# SSDs
ACTION=="add|change", KERNEL=="sd[a-z]", ATTR{queue/rotational}=="0", ATTR{queue/scheduler}="deadline"

# HDDs
ACTION=="add|change", KERNEL=="sd[a-z]", ATTR{queue/rotational}=="1", ATTR{queue/scheduler}="bfq"
Then to load the bfq module, you can add it to /etc/rc.d/rc.modules.local
Great tips here. Thank you!
 
Old 01-08-2019, 12:40 PM   #17
rogan
Member
 
Registered: Aug 2004
Distribution: Slackware
Posts: 102

Original Poster
Rep: Reputation: 22
bassmadrigal: I have done some more testing on 14.2 using kernels 4.4.157 and 4.19.13 on the system.
4.19.13 really gives a massive performance hit on my AMD machine. It's not as pronounced on 14.2, xfs is still
usable here. This might be attributable to different compilers on current and 14.2 for example.
Numbers:
System kernel 4.19.13: defconfig kernel compilation on a xfs file system while doing background file transfers: 9 min 50 sec
System kernel 4.19.13: defconfig kernel compilation on a xfs file system without doing background file transfers: 2 min 40 sec
System kernel 4.4.157: defconfig kernel compilation on a xfs file system while doing background file transfers: 2 min 55 sec
System kernel 4.4.157: defconfig kernel compilation on a xfs file system without doing background file transfers: 2 min 45 sec
System kernel 4.19.13: defconfig kernel compilation on a ext4 file system while doing background file transfers: 4 min 28 sec
System kernel 4.19.13: defconfig kernel compilation on a ext4 file system without doing background file transfers: 2 min 40 sec
System kernel 4.4.157: defconfig kernel compilation on a ext4 file system while doing background file transfers: 3 min 5 sec
System kernel 4.4.157: defconfig kernel compilation on a ext4 file system without doing background file transfers: 2 min 40 sec

I have wasted far too much time on this and wont' dive deeper into this crap.
Im calling it solved: Linux is succumbing to severe late stage featuritis.
 
1 members found this post helpful.
Old 01-08-2019, 05:00 PM   #18
bassmadrigal
LQ Guru
 
Registered: Nov 2003
Location: West Jordan, UT, USA
Distribution: Slackware
Posts: 5,632

Rep: Reputation: 3389Reputation: 3389Reputation: 3389Reputation: 3389Reputation: 3389Reputation: 3389Reputation: 3389Reputation: 3389Reputation: 3389Reputation: 3389Reputation: 3389
Maybe I've just never noticed it. I can try compiling a kernel with and without USB transfers and time them for comparison.

As for succumbing to too many features, I doubt that is the case since it is such a drastic difference between xfs and ext4. If they were similar, I'd be more inclined to agree. Any chance you noticed this in any other kernels, like 4.14, to see if we can pinpoint when the issue occurred?
 
Old 01-11-2019, 02:51 PM   #19
Martinus2u
Member
 
Registered: Apr 2010
Distribution: Slackware
Posts: 413

Rep: Reputation: 72
Quote:
Originally Posted by ZhaoLin1457 View Post
From what I read, the consensus is that is much better to use Deadline for SSDs and BFQ for the mechanical hard drives. And the Block Multi-Queue.
I believe, in the past (prior to merging it into the kernel) BFQ used to be superior to CFQ in the case of SSDs, too. At least the benchmarks posted by the creator suggested so.

It should be noted that there are some people who say BFQ is not as good as it used to be since it was incorporated into the kernel as part of block multi-queue, mostly because of inefficiencies of the block multi-queue part itself. However, I cannot provide personal experience with that.
 
1 members found this post helpful.
Old 01-13-2019, 01:30 PM   #20
rogan
Member
 
Registered: Aug 2004
Distribution: Slackware
Posts: 102

Original Poster
Rep: Reputation: 22
The problem I encountered is courtesy of "Writeback Throttling".

A fantastic feature designed to make sure that you cannot anymore
perform two separate io-heavy processes simultaneously, without throttling
them almost to halt, even if they are on completey different disks.
This used to be possible. On my OpenBSD systems I can do this.
I disabled this "feature" (recompile) and I got my fast multiprocess
multiuser system back again -regardless of file system choice.

Given the lack of reports of people having these problems I must have
either pretty unique hardware (5 disk system on an old 8320 amd) or
very strange habits (compiling while background copying).
Go figure....
 
2 members found this post helpful.
Old 01-13-2019, 03:49 PM   #21
Petri Kaukasoina
Member
 
Registered: Mar 2007
Posts: 364

Rep: Reputation: 213Reputation: 213Reputation: 213
Quote:
Originally Posted by rogan View Post
The problem in itself is not solved and any kind of fiddling with schedulers or some such
will not improve anything noticeable in my can.
Quote:
Originally Posted by rogan View Post
The problem I encountered is courtesy of "Writeback Throttling".

I disabled this "feature" (recompile) and I got my fast multiprocess
multiuser system back again -regardless of file system choice.
You never told which I/O scheduler you use. Writeback throttling is disabled when cfq or bfq is used, at least in linux-4.19.
 
Old 01-13-2019, 11:28 PM   #22
rogan
Member
 
Registered: Aug 2004
Distribution: Slackware
Posts: 102

Original Poster
Rep: Reputation: 22
I use defaults (cfq).
I know that cfq or bfq are supposed to disable WT. Patched back in 2017 I believe.

After some more testing I'm back to square one.
Still only 2 solutions:
1: Use btrfs (or reiserfs).
2: Revert to 4.14 series. I haven't tested extensively on the 4.14 series
but I never noticed any problems as extreme as on current with 4.19 and xfs.

Last edited by rogan; 01-15-2019 at 06:35 AM.
 
Old 01-14-2019, 04:19 PM   #23
GazL
Senior Member
 
Registered: May 2008
Posts: 4,811
Blog Entries: 14

Rep: Reputation: Disabled
For CFQ I've always done my IO heavy workloads/batch compiles etc from a niced shell, started using this alias:

alias nicesh='chrt -i 0 ionice -c 3 ${SHELL:-/bin/sh}'

I've never had interactive response issues when doing that, though I've certainly seen it when I've forgotten to start a niced shell for the job.
 
1 members found this post helpful.
Old 01-15-2019, 02:37 PM   #24
Pixxt
Member
 
Registered: May 2008
Distribution: Slackware, Debian,
Posts: 170

Rep: Reputation: 63
BFQ is the buggiest io sched in the kernel many reports of lost files, system freezes and kernel panics. BFQ is called snake oil as there is no hard data i.e(third party benchmarks,tests) that suggests that BFQ is any faster or lower latency then CFQ or Deadline. In fact just in the past few months or so BFQ was kicked out of Con Kolivas ck patch-set for Linux for being too buggy and having no gains on performance and latency.

Just stick to CFQ or Deadline on spinning disks until kernel 5.0 which removes deadline, CFQ and noop altogether.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Incredible Render Performance In KdenLive With NVENC - But 1 Big Problem immortanjohn Linux - General 1 07-16-2018 11:17 AM
LXer: Stable Linux kernel hit by ext4 data corruption bug - Update LXer Syndicated Linux News 0 10-25-2012 10:20 AM
LXer: Stable Linux kernel hit by ext4 data corruption bug LXer Syndicated Linux News 0 10-24-2012 11:20 PM
Is it safe to format USB flash to ext4 or ext4? joham34 Linux - Newbie 2 01-08-2011 12:58 PM
LXer: Incredible Times, Incredible Technology LXer Syndicated Linux News 0 10-29-2010 02:20 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware

All times are GMT -5. The time now is 04:34 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration