CPU iowait due to HDD overload is lagging the video recording saved on fast SSD

ViableAlternative · 03-16-2021, 10:17 AM

Well, as I said earlier, I'm not trying to pick a fight. I'm just trying to share my own first hand observations on what it's like to come here fresh. There is an undertow here that makes one feel reluctant to risk asking the "wrong" question. Is that really the aura this place hopes to project?

Everyone has been just super with me personally, but in reading the back catalog here, I can see that not everyone has been so fortunate.

TB0ne · 03-16-2021, 10:34 AM

Quote:

Originally Posted by ViableAlternative

Well, as I said earlier, I'm not trying to pick a fight. I'm just trying to share my own first hand observations on what it's like to come here fresh. There is an undertow here that makes one feel reluctant to risk asking the "wrong" question. Is that really the aura this place hopes to project? Everyone has been just super with me personally, but in reading the back catalog here, I can see that not everyone has been so fortunate.

Neither am I. But again, you need to think about what's being asked, and how. It isn't that it's a 'wrong' question, but when there aren't details/information and the poster plain WILL NOT provide them, what can be done? Can you answer a question about anything, when you don't know any details about what's being asked?

And to go a bit further, why should you/I/anyone else have to look things up for other people? Entering a search-term in any search engine isn't complex or arduous...and asking someone to show SOME effort of their own isn't a bad thing. This isn't a help desk, and none of us get paid, so when people come and post things like "I need a script to do this, thanks", or "How to I configure <PROGRAM>", asking them to show what they've done, or to read the instructions isn't rude.

This site, as with anything, will give back what you put in. No details/effort? That's the kind of answer you'll get back. We have gone off-topic here...best to move this to the "Suggestions & Feedback" forum should you wish to continue. And I'd also suggest you read many of the threads there, posted by many long-time members about these very topics.

ViableAlternative · 03-16-2021, 10:39 AM

Part of it too might be a disconnect between people possessing varying levels of expertise. You seldom find at the universities that the program director for an entire department to be the one teaching the entry level 101 courses. They forget what it feels like to be "dumb as dirt" and might in fact mistake ignorance as something more contemptible. So they make best use of their time on the doctoral candidates, allowing their subordinates to wade through the tide of us unwashed masses in the 101 pool. lol!

postcd · 03-16-2021, 05:15 PM

Can you guys stop chit chatting there (and use PM possibly) and bringing this offtopic. Please keep silent if you have nothing useful to say to the topic. Am i right @jeremy @onebuck ?

Following is a bit ontopic, there was mentioned nice and ionice. But how i can increase the process priority without sudo?
Maximum prio is: /bin/nice -n -20 /usr/bin/ionice -c1 -n0 process
but it needs sudo. "sudo -u me" does not work - Operation not permitted
I think that i would need ionice in this case.

rnturn · 03-16-2021, 08:02 PM

Quote:

Originally Posted by postcd

Can you guys stop chit chatting there (and use PM possibly) and bringing this offtopic. Please keep silent if you have nothing useful to say to the topic. Am i right @jeremy @onebuck ?

Following is a bit ontopic, there was mentioned nice and ionice. But how i can increase the process priority without sudo?
Maximum prio is: /bin/nice -n -20 /usr/bin/ionice -c1 -n0 process
but it needs sudo. "sudo -u me" does not work - Operation not permitted
I think that i would need ionice in this case.

So you're issuing that sudo command and receiving that error?

Code:

$ sudo -u me
Operation not permitted
$

Have you tried:

Code:

$ sudo nice -n -20 ionice -c1 -n0 -p <pid>

(Or some subset of that command? I.e. either 'nice' or 'ionice' individually?)

Are you receiving an error message from that? A prompt for root's password?

BTW, Try using the "script" command to capture a command/response sequence that you can paste into "code" tags (simple example):

Code:

$ script session.log
Script started, file is session.log
$ date
Tue Mar 16 19:53:05 CDT 2021
$ uptime
 19:53:09  up 9 days 17:52,  5 users,  load average: 0.38, 0.41, 0.51
$ exit
Script done, file is session.log
$

then "cat session.log" or pull it into an editor (say, Kate), select all of it, copy it onto the clipboard, and paste it into the LQ post composition window. It'll accurately show people what you've done and how the system responded without having to re-key system responses---less change for typos, etc.. (You could edit out the script command itself and the messages it displays to save some space.)

HTH...

postcd · 03-17-2021, 04:11 AM

Quote:

Originally Posted by rnturn

Code:

$ sudo nice -n -20 ionice -c1 -n0 -p <pid>

This works, sudo is necessary, else it says "Operation not permitted".
But in my case the point is to repeatedly run app (in this case Firefox) and give it the highest io priority so other apps will not interrupt for example its multimedia playback if other app utilizing maximum available disk resources... So running your nice/ionice command manually always i start Firefox (from within XFCE's whisker menu) is not convenient. Btw. Firefox does not allow using sudo when launching it:

/bin/nice -n -20 sudo /usr/bin/ionice -c1 -n0 /usr/lib/firefox/firefox %u -p -no-remote

Quote:

Running Firefox as root in a regular user's session is not supported. ($XAUTHORITY is /home/me/.Xauthority which is owned by me.)

So i was trying to edit whisker menu launch entry to open command like following in terminal, but obviously bad syntax and FF is not going to background or B, nothing happen upon launch:

Quote:

/usr/lib/firefox/firefox %u -p -no-remote & sleep 10;for pid in $(ps aux|grep firefox|awk '{print $2}');do ionice -c1 -n0 -p $pid;done;exit
or
sleep 10 && for pid in $(ps aux|grep firefox|awk '{print $2}');do ionice -c1 -n0 -p $pid;done;/usr/lib/firefox/firefox %u -p -no-remote &

btw. on Linux there is no command to send to a complex python script system (that is launched via bash script) so it allow using only lets say 50% of the disk resources? This is about virtualized containers rather only? I have just tried to prefix the .py launching command by:
/bin/nice -n 19 /usr/bin/ionice -c2 -n7
Maybe it will help reduce this py system impact on other apps.

----------

UPDATE:
Thank you rnturn, Following post of yours was helpful.
I have used your script to create two each for changing priority of any process that i mention as this script parameter

runprochighprio

Quote:

#!/bin/bash
# Run process with higher priority. ./thisscriptname /usr/bin/process

$1 &
PID=$!
renice -19 $PID >/dev/null
ionice -c2 -n1 -p $PID

runproclowprio

Quote:

#!/bin/bash
# Run process with lower priority. ./thisscriptname /usr/bin/process

$1 &
PID=$!
renice +19 $PID >/dev/null
ionice -c2 -n7 -p $PID

both add exec. permission:
chmod +x scriptname

THen in XFCE app menu (whisker) entry (mouse right click) and replaced for example for the Firefox:
/usr/lib/firefox/firefox %u -p -no-remote
by:
/home/me/scripts/runprochighprio /usr/lib/firefox/firefox -p -no-remote

i am guessing it is working, FF is too many processes so i do not know which one is which, but for different app, i see:
"ionice -p pidhere" shows
best-effort: prio 1
while other apps:
none: prio 0
System monitor and Task manager though shows fake priority 0 (same as other processes).

rnturn · 03-19-2021, 02:26 AM

Quote:

Originally Posted by postcd

I have just tried to prefix the .py launching command by:
/bin/nice -n 19 /usr/bin/ionice -c2 -n7
Maybe it will help reduce this py system impact on other apps.

Not sure where Python came into this but should be able modify the scheduling priority and I/O limit of just about anything.

I tried this last night:

Code:

#!/bin/bash
/usr/bin/chromium &
PID=$!
renice +19 $PID >/dev/null
ionice -c2 -n7 -p $PID

and, sure enough, Chromium was running at that low scheduling priority (as you'd expect). It ought to work with Firefox -- or just about anything -- as well. I've had good luck simply prefixing the Firefox command in KDE launcher with "nice -some-value" and can verify via "htop" that it is running at a lower scheduling priority. The above wrapper script (or one like it) would be needed to tweak the process with ionice. I'm not aware of any mechanism that will allow you to run a one-liner to affect both process parameters.

BTW, I'm not sure just how well ionice deals with I/O intensive processes. I'd suggest doing a test where you hit a filesystem rather hard, say,

Code:

$ find /usr/bin -type f -exec md5sum {} \;

and observe the disk IO rate parameter for that process in "htop", and after a time, tweak the process using ionice. You may find that "nice" is all you need but, as they say, YMMV.

HTH...