Noob IO behaviour iritations
 

Bit of a rant - sorry .

The CPU scheduler under Linux seems to do a great job - I can run many CPU intensive things at once .

They all just gracefully slow down the more you add.

My problem is that this is not the case with IO (reading writing from disk).

I ran Opera (with ~50 tabs), Firefox (with ~20) & F-Spot (photo management - 10000+ pics) yesterday evening all at once.

Its a 600 MB RAM system with 1 disk for OS & 1 disk for swap & space for pictures etc (anything valuable which must live).

My issue is that Opera & F-spot will both consume close to 100% CPU & it creates no problem at all - a bit slower but thats it .

Not the case for IO - even when f-spot is reniced to something like 15 .

I can then end up not being able to browse just because (a reniced to 15) f-spot is reading in pictures (which is the behavior of an unnamed OS although to a far greater extend - which was one reason I switched for daily stuff) .

Yes its a huge amount of pics & it all swaps like 600MB but shouldnt the IO priority adjust to CPU priority ... ?

Does IO scheduling not go as far in its functionality as CPU scheduling under Linux ?

Miny rant over :) & curious 4 feedback.

So pick a different I/O scheduler and see if you get a better fit.
grep dmesg for scheduler, then go google for the differences between them.

I/O scheduling and CPU scheduling are really not very much related to one another, in terms of their influence on system performance. The CPU is so-o-o much faster than everything else, it just needs to give fair-share scheduling to all the CPU-bound processes while responding quickly enough to I/O interrupts. The fact that you're using nice is, well, a nice thing to do ... helps it keep its, well, priorities straight. :) (Actually, I didn't write that as a pun... I just happened a-pun it.)

Anyhow ... :)

You probably want to expand the system above 600MB if you can, because this will give you lots of buffer-space. That's good.

It sounds to me like you might be saturating your I/O system. It might be time for another disk-interface card. (The onboard I/O interfaces are not designed for raw speed.) A "smart" card that knows how to do its own buffering and such is also a good thing. These cards are not particularly expensive.

Spread the information out across multiple DASD .. err, disk volumes ;) ... on multiple channels that are fully equipped to operate simultaneously. FireWire, for instance, has more parallel-I/O capability than "good ol' EIDE."

Once the hardware can do it, Linux can schedule it.

AAhhhhh IC .. so its stone age hardware design from last century problems again ;)

THX for your replies.


Yeah I completly forgot that oviously the bus will be a bottleneck at some point .


Yes makes sense .. but :

Two IDE channels - one for each disk .

If it says UDMA100 - AFAIK it means that the max data possible that can be moved would be 100 MB a second .

One disk is 100 the other 66 or so ... so in theory the data should be able to move from the disks to memory & processor as fast as the disks can spit it out .

Or is this again something where there is something in a PC which doesnt agree to fast data movement ?

Okay .. reread ... PCs are simply not designed with max speed in mind it seems.


THX again .

Maybe time to get a new system :)

:)

Well, that's not really quite the case...

The stock I/O controllers that are built in to most motherboards are not designed for high throughput. They're more-or-less designed to efficiently run your grandmother's machine while she browses the web and sends you unwanted pictures. ;)

If you purchase a fast I/O-controller card and snap it into one of the motherboard slots, you immediately get much faster throughput.

If you need very fast performance, including the ability to have multiple I/O transfers going on "really 'at once,'" you start moving into the world of, say, FireWire. (FireWire is basically the latest incarnation of SCSI...) Now you are moving data on a multi-lane high-speed highway.

The odd thing is, this greatly-improved hardware really does not cost that much, nor is it particularly hard to find. You just won't find it at OfficeMax (usually), and the red-shirted kid at CompUSA might give you a blank stare ... which is no surprise. ;)

THX again :)

Firewire is AFAIK what Apple uses (introduced as well ?) for their external disks & video moving stuff .

& which is also more common place for PCs to now have .

I thought "fibre-channel" is the new SCSI ?

I never thought as Firewire as being fast because I thought it was similiary slowish like USB .

The USB port reducing the data that can be moved ... well I have to then look up things .

& Update my general hardware knowledge .

THX again :)

Fiber-channel is a still-faster backplane mechanism, based on optical fibers.