Slackware - ARM This forum is for the discussion of Slackware ARM. |
Notices |
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
Are you new to LinuxQuestions.org? Visit the following links:
Site Howto |
Site FAQ |
Sitemap |
Register Now
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
|
 |
|
03-26-2014, 03:27 AM
|
#1
|
Slackware Contributor
Registered: Apr 2008
Distribution: Slackware
Posts: 1,710
|
Changing package compression from gzip to LZMA (txz)
Hello
Slackware switched to using LZMA to compress packages in order to solve space issues on the CDs. Since this isn't an issue for ARM, and coupled with my understanding (from xz(1)) that there would be issues decompressing content on machines with limited RAM (128MB, 256MB), it made no sense to change.
However, I'm not sure if there really are issues decompressing LZMA packed packages on machines with small amounts of RAM, so I am thinking of changing the compression. I would assume that the decompression would definitley be slower, however.
Can any of you with real hardware (there's no point testing QEMU - it's not representative of real hardware speed) that 128 or 256MB RAM test these assumptions and provide some timing info please?
Thanks
|
|
|
03-26-2014, 07:27 AM
|
#2
|
Member
Registered: Dec 2013
Location: Italy
Distribution: Slackware
Posts: 636
Rep:
|
I downloadad the kernel source package from slackware x86 (it's one of the beefy packages).
Firs I timed how lon it takes to do a "tar tf":
Code:
real 0m50.771s
user 0m37.350s
sys 0m7.170s
Then I tryed to install it:
Code:
bash-4.2# time installpkg kernel-source-3.10.17_smp-noarch-3.txz
Verifying package kernel-source-3.10.17_smp-noarch-3.txz.
Installing package kernel-source-3.10.17_smp-noarch-3.txz:
PACKAGE DESCRIPTION:
# kernel-source (Linux kernel source)
#
# Source code for Linus Torvalds' Linux kernel.
#
# This is the complete and unmodified source code for the Linux kernel.
#
Executing install script for kernel-source-3.10.17_smp-noarch-3.txz.
Package kernel-source-3.10.17_smp-noarch-3.txz installed.
real 4m7.048s
user 1m58.150s
sys 0m59.510s
bash-4.2#
After having installed it I still have 91Mb free memory (and I've no swap):
Code:
bash-4.2# free
total used free shared buffers cached
Mem: 122996 120484 2512 0 36840 51692
-/+ buffers/cache: 31952 91044
Swap: 0 0 0
bash-4.2#
I did notice though that the cpu usage mas much more intensive then memory usage ... top was telling me that xz was using over 90% cpu and only about 10% ram (but I did not sit the whole 4 minutes watching it).
Last edited by louigi600; 03-27-2014 at 02:01 AM.
|
|
|
03-27-2014, 03:26 PM
|
#3
|
Member
Registered: Aug 2003
Location: Melbourne, Australia
Distribution: Slackware, Slackwarearm
Posts: 896
Rep: 
|
I have installed gcc-4.8.2-arm-1.tgz on a raspberrypi with 256mb ram and gpu_mem set at 32mb. Install was off an external usb drive which is where I have my /Slackware directory when I actually install slackware arm on an sd card in a pi. Pi access was via ssh. I installed your tgz packaged first then used makepkg to create a txz package to install.
Code:
root@shed-2:/mnt/sda1# ROOT=/tmp/build time installpkg gcc-4.8.2-arm-1.tgz
Verifying package gcc-4.8.2-arm-1.tgz.
Installing package gcc-4.8.2-arm-1.tgz:
<snipped>
12.36user 4.42system 0:21.89elapsed 76%CPU (0avgtext+0avgdata 6256maxresident)k
3264inputs+85888outputs (15major+45596minor)pagefaults 0swaps
Code:
root@shed-2:/mnt/sda1# ROOT=/tmp/build time installpkg gcc-4.8.2-arm-1.txz
Verifying package gcc-4.8.2-arm-1.txz.
Installing package gcc-4.8.2-arm-1.txz:
<snipped>
25.83user 6.20system 0:39.00elapsed 82%CPU (0avgtext+0avgdata 35872maxresident)k
152inputs+85936outputs (2major+51284minor)pagefaults 0swaps
I've done this several times installing both to a direrctory on the pi and the usb drive and time/cpu load wil vary a bit each time. Below are results from installs to the external drive.
I can do some more testing maybe this evening or tomorrow if you want me to do something specific.
Code:
11.95user 5.10system 0:20.08elapsed 84%CPU (0avgtext+0avgdata 6256maxresident)k
39136inputs+85896outputs (33major+40829minor)pagefaults 0swaps
25.99user 5.99system 0:36.53elapsed 87%CPU (0avgtext+0avgdata 35872maxresident)k
13288inputs+85944outputs (1major+51296minor)pagefaults 0swaps
|
|
|
03-28-2014, 11:35 AM
|
#4
|
Slackware Contributor
Registered: Apr 2008
Distribution: Slackware
Posts: 1,710
Original Poster
|
Hi guys
Thanks for that feedback. It's not far off what I was expecting really, and it's good to see that swap wasn't impacted.
I'll slowly start changing the package compression to LZMA then as the packages are updated.
|
|
|
03-28-2014, 04:33 PM
|
#5
|
Senior Member
Registered: Dec 2008
Posts: 1,212
|
Quote:
Originally Posted by louigi600
...
After having installed it I still have 91Mb free memory (and I've no swap):
...
|
So what? The package is not installed into memory.
|
|
|
03-28-2014, 04:37 PM
|
#6
|
Senior Member
Registered: Dec 2008
Posts: 1,212
|
Quote:
Originally Posted by drmozes
However, I'm not sure if there really are issues decompressing LZMA packed packages on machines with small amounts of RAM, so I am thinking of changing the compression.
|
If you have more than 65MB free memory on your ARM hardware, then use xz if you wish. The official distribution is intended to be generic, however, so it must stay with gzip.
|
|
|
03-29-2014, 03:52 AM
|
#7
|
Slackware Contributor
Registered: Apr 2008
Distribution: Slackware
Posts: 1,710
Original Poster
|
Quote:
Originally Posted by guanx
If you have more than 65MB free memory on your ARM hardware, then use xz if you wish. The official distribution is intended to be generic, however, so it must stay with gzip.
|
The official distribution? What do you mean?
|
|
|
03-29-2014, 06:39 AM
|
#8
|
Senior Member
Registered: Dec 2008
Posts: 1,212
|
Quote:
Originally Posted by drmozes
The official distribution? What do you mean?
|
The slackwarearm compiled and distributed by Stuart Winter.
|
|
|
03-29-2014, 07:17 AM
|
#9
|
Slackware Contributor
Registered: Apr 2008
Distribution: Slackware
Posts: 1,710
Original Poster
|
Quote:
Originally Posted by guanx
The slackwarearm compiled and distributed by Stuart Winter.
|
That's me.
|
|
|
03-29-2014, 07:20 AM
|
#10
|
Senior Member
Registered: Dec 2008
Posts: 1,212
|
Quote:
Originally Posted by drmozes
That's me.
|
Ok. The doctor advised me to take complete rest. Now I see he is right.
|
|
|
03-29-2014, 03:09 PM
|
#11
|
Member
Registered: Dec 2013
Location: Italy
Distribution: Slackware
Posts: 636
Rep:
|
Quote:
Originally Posted by guanx
So what? The package is not installed into memory.
|
No it simply means that checking, uncompressing and installing of the package did not use much of the the 128Mb avalible on the system to do the job.
|
|
|
03-29-2014, 04:40 PM
|
#12
|
Senior Member
Registered: Dec 2013
Distribution: Slackware
Posts: 1,982
|
I know there is a limit on RAM required to decompress a package compress with xz. This depends on the options used to compress it, see xz man page. The default -6 level requires 9 MiB when decompressing, while -9 requires 65 MiB.
As for CPU usage, it is definitely higher than with gzip.
|
|
|
03-29-2014, 06:15 PM
|
#13
|
Senior Member
Registered: Dec 2008
Posts: 1,212
|
Quote:
Originally Posted by louigi600
No it simply means that checking, uncompressing and installing of the package did not use much of the the 128Mb avalible on the system to do the job.
|
I guess that's simply because your memory gets reclaimed after use. Correct me if it does not.
|
|
|
03-31-2014, 05:35 AM
|
#14
|
Member
Registered: Dec 2013
Location: Italy
Distribution: Slackware
Posts: 636
Rep:
|
Quote:
Originally Posted by guanx
I guess that's simply because your memory gets reclaimed after use. Correct me if it does not.
|
Slackware's package creation utility uses default compression level (-6), or at least that's what I could tell by examining the script, so I think that I never used that much memory anyway ... as someone said earlier for -6 compression level only 9 Mb are required for decompression and that's in line with what top was telling me about xz as the installation was going on.
I don't think that the kernel would free up 9 Mb when there are 90 Mb free out of 128 and there is not much running on my nas that would require memory anyway.
And BTW: Slackware for the Intel platform has been using xz package compression for quite some tine now so I guess that Stuart wants to make the official ARM port more standard exactly by using the standard Slackware package compression algorithm.
|
|
|
03-31-2014, 07:30 AM
|
#15
|
Senior Member
Registered: Dec 2008
Posts: 1,212
|
Quote:
Originally Posted by louigi600
Slackware's package creation utility uses default compression level (-6), or at least that's what I could tell by examining the script, so I think that I never used that much memory anyway ...
|
Thanks for the info! That looks good. Let's hope no one sets the XZ_DEFAULTS environment variable to change compression to -9.
Quote:
Originally Posted by louigi600
I don't think that the kernel would free up 9 Mb when there are 90 Mb free out of 128
|
That your kernel doesn't free up memory of dead processes really surprised me but since I am still under medical supervision and cannot use computer for long time, I won't make any comment on this.
Quote:
Originally Posted by louigi600
And BTW: Slackware for the Intel platform has been using xz package compression for quite some tine now so I guess that Stuart wants to make the official ARM port more standard exactly by using the standard Slackware package compression algorithm.
|
My opinion is that Stuart can make the decision on his own. Whatever his decision is, it won't affect me very much because I rarely install packages directly on small systems.
|
|
|
All times are GMT -5. The time now is 08:02 AM.
|
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
|
Latest Threads
LQ News
|
|