LinuxQuestions.org - is it me, or is tar slow?

- Slackware (https://www.linuxquestions.org/questions/slackware-14/)

- - is it me, or is tar slow? (https://www.linuxquestions.org/questions/slackware-14/is-it-me-or-is-tar-slow-319068/)

is it me, or is tar slow?

On a P4 3GHz 512 RAM with newly installed slack 10.1 kernel 2.6.10 , I get major hangage with some batch tar-ing (e.g. 10-20 files), a la:

ls *.tar | xargs -i tar -xf \{\}
- or -
find -maxdepth 1 -name "*.tar" -exec tar -xf {} \;
- or -
what have you...

I get major mouse stuttering and can hear the load on my fans, until the process stops or I stop it. Then there's a little 'after-shock' a few seconds later, for a few seconds.

This doesn't seem right to me. Any thoughts?

Could you run one of those tar commands prefixed with the time command, ie. 'time find...' and give us a size on what you're tar/gzipping? (du -chs would be ideal).

Thanks for the quick reply, mdarby.

$ du -chs
67M .
67M total

I tried running the extractions a few more times with 'time', but I must say, the results are baffeling.

1st attempt:
===========

$ /bin/ls *.tar | xargs -i time tar -xf \{\}
0.00user 0.01system 0:00.01elapsed 83%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+180minor)pagefaults 0swaps
0.00user 0.03system 0:00.03elapsed 94%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+252minor)pagefaults 0swaps
0.00user 0.00system 0:00.00elapsed 75%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+180minor)pagefaults 0swaps
0.00user 0.01system 0:00.01elapsed 93%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+252minor)pagefaults 0swaps
0.00user 0.00system 0:00.00elapsed 57%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+252minor)pagefaults 0swaps
0.01user 0.07system 0:00.08elapsed 98%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+252minor)pagefaults 0swaps
0.00user 0.00system 0:00.00elapsed 71%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+181minor)pagefaults 0swaps
0.00user 0.01system 0:00.01elapsed 85%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+181minor)pagefaults 0swaps
0.00user 0.00system 0:00.00elapsed 88%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+252minor)pagefaults 0swaps
0.00user 0.00system 0:00.00elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+181minor)pagefaults 0swaps
0.00user 0.00system 0:00.01elapsed 81%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+181minor)pagefaults 0swaps
0.00user 0.01system 0:00.01elapsed 88%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+252minor)pagefaults 0swaps
0.00user 0.01system 0:00.02elapsed 90%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+254minor)pagefaults 0swaps
0.00user 0.01system 0:00.02elapsed 90%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+183minor)pagefaults 0swaps
0.00user 0.02system 0:00.03elapsed 96%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+181minor)pagefaults 0swaps
0.02user 0.26system 0:02.97elapsed 9%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+180minor)pagefaults 0swaps

Not bad, except maybe that last file - so I moved it from the directory it was in, and then everything extracted fine. But then I put it back in, and it extracted speedily, like the rest, as well. Extracting the individual files one at a time from the command line also seemed speedy.

So then I tried again on a directory full of a similar number of archives, and got some really bad output, hanging. I went back to the same directory where things had pretty much worked, and also got bad hanging! To the point where I had to send the kill signal and wait for the 'after-shocks' to die down and give control of my mouse back. See output below:

2nd attempt
=============

$ /bin/ls *.tar | xargs -i time tar -xf \{\}
0.00user 0.00system 0:00.46elapsed 1%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+180minor)pagefaults 0swaps
0.00user 0.02system 0:01.07elapsed 2%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+252minor)pagefaults 0swaps
0.00user 0.09system 0:02.27elapsed 4%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+251minor)pagefaults 0swaps
0.01user 0.08system 0:03.12elapsed 3%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+180minor)pagefaults 0swaps
0.00user 0.04system 0:02.07elapsed 2%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+251minor)pagefaults 0swaps
0.00user 0.03system 0:00.80elapsed 3%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+180minor)pagefaults 0swaps
0.01user 0.27system 0:16.06elapsed 1%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+251minor)pagefaults 0swaps
0.00user 0.02system 0:03.04elapsed 0%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+253minor)pagefaults 0swaps
0.00user 0.03system 0:05.81elapsed 0%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+181minor)pagefaults 0swaps
0.00user 0.03system 0:05.45elapsed 0%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+180minor)pagefaults 0swaps
0.00user 0.01system 0:00.78elapsed 2%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+180minor)pagefaults 0swaps
0.00user 0.01system 0:02.26elapsed 0%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+180minor)pagefaults 0swaps
0.00user 0.02system 0:02.53elapsed 1%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+180minor)pagefaults 0swaps
0.00user 0.38system 0:02.24elapsed 17%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+252minor)pagefaults 0swaps
0.00user 0.00system 0:00.17elapsed 4%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+252minor)pagefaults 0swaps
0.00user 0.04system 0:02.07elapsed 2%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+251minor)pagefaults 0swaps

Finally, I tried a few more random files - files that had both taken long and short times to extract before. They extracted promptly. Then really quickly I ran the batch extraction line - about 7 files extracted quickly, and then they got bogged down and I got output like attempt #2 above.

I have no idea how to trouble-shoot this. Your help is greatly appreciated.

Did you compiled the kernel yourself or used the prepackaged one?
Check if you have dma disabled (if you have it will likely slow all operations on the disk) with: hdparm -d /dev/hdX

@yekibud:

;)

Try

Code:

time find -maxdepth 1 -name "*.tar" -exec tar -xf {} \;

Just prefix whatever command you're running with 'time'

gbonvehi,

Well, I got the 2.6 kernel from Slackware testing, but I did a menuconfig, like I usually do. The hdparm man says mda *should* usually be enabled for optimum performance. I seem to have MDA enabled, but when I try setting it, I get this:

$ hdparm -d1 /dev/hda

/dev/hda:
setting using_dma to 1 (on)
HDIO_SET_DMA failed: Operation not permitted
using_dma = 0 (off)

So, not sure whether you are suggesting it should be on or off. In any case, it's off now, and I don't know yet if it will change anything if it's on. More reading...

mdarby,

Here's the time output for the command line you gave me:

real 0m15.733s
user 0m0.050s
sys 0m0.448s

Lots of mouse stuttering still.

I would ensure that your motherboard IDE controller drivers are compiled into your kernel (or verify that they're being loaded via 'modprobe')

Could you post the output from

Code:

htparm -t /dev/hda

If you have modern hardware (newer IDE) you can try this tweak:

Code:

hdparm -c1 -X70 -u1 -d1 /dev/hda

*Use at your own risk!* I've never had a problem with the above tweak, but YMMV!
With the tweak, I get about 60MB/s from my drive, as opposed to the original 17MB/s

yeki. You most certainly do not have dma enabled -- you need that, or any disk-intensive utility will absolutely suck. What's more, 'operation not permitted' means that you don't have support for your IDE chipset enabled. You either need to load the module for it, or compile it in.

--Shade

Thanks, guys - I think you set me in the right direction, but I'm having a hell of a time getting DMA enabled. LSPCI tells me my IDE controller is VT82C586A, but when I recompile my kernel for VIA82cxxx, I still get the hdparm -d1 /dev/hda error:

/dev/hda:
setting using_dma to 1 (on)
HDIO_SET_DMA failed: Operation not permitted
using_dma = 0 (off)

Found another thread related to DMA and chipset problems, which involved disabling generic IDE support and setting four different DMA options in the kernel .config, but that solution didn't work for me. Man, what a headache... what am I missing?

I would double check your chipset type with the mobo manufacturer; I've seen cases where lspci doesn't report (or isn't told) the correct type.

Are you compiling these drivers into your kernel, or selecting them as modules? I'm not sure if modules would work in this case... I normally compile everything in.

PC Chips, the mobo manufacturer, has some IDE drivers... for windows, only. But the chipset is indeed VT8237 - there's a big chip with a shiny sticker that says so smack-dab on the board.

I've been googling and googling for some kind of patch or related problems - and I did come accross people talking about a patch at some obscure french site (www.pyrenees.org/udma/udma.html), that doesn't exist any more. Could it be that this is unsupported hardware? Is there a blacklist I can check somewhere?

And, yes, I compiled the VIA82cxxx driver into the kernel...

Loosing...

hope...

Thanks for your effort, gbonvehi! I read those posts, too. ;)

And, yeah, tried disabling the generic IDE support - still nothing.

No problem, I always learn helping people :)

Can you post your motherboard name/model?