LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware
User Name
Password
Slackware This Forum is for the discussion of Slackware Linux.

Notices


Reply
  Search this Thread
Old 05-09-2009, 07:53 AM   #76
niels.horn
Senior Member
 
Registered: Mar 2007
Location: Rio de Janeiro - Brazil
Distribution: Slackware64-current
Posts: 1,004

Rep: Reputation: 91

Quote:
Originally Posted by grissiom View Post
I think .txz or .tgz does not affect your system performance unless you are running a system that aimed at install packages over and over again
This I understand.
What I meant to ask is if anyone noticed the upgrade-process being slower, not the system performance after upgrading

I have a local mirror of slackware-12.2 & slackware-current, so upgrading the machines in the network is usually quite fast (no downloading of packages over slow lines while upgrading, just once a night rsync-ing the mirror).
I was wondering if changing to xz (de-)compression slowed things down in a way we can actually notice.
 
Old 05-09-2009, 09:50 AM   #77
grissiom
Member
 
Registered: Apr 2008
Location: China, Beijing
Distribution: Slackware
Posts: 423

Rep: Reputation: 45
Quote:
Originally Posted by niels.horn View Post
This I understand.
What I meant to ask is if anyone noticed the upgrade-process being slower, not the system performance after upgrading

I have a local mirror of slackware-12.2 & slackware-current, so upgrading the machines in the network is usually quite fast (no downloading of packages over slow lines while upgrading, just once a night rsync-ing the mirror).
I was wondering if changing to xz (de-)compression slowed things down in a way we can actually notice.
I got your idea But I use slackpkg to do upgrading system.(I do keep a rsync in local but not upgrade from it) AFAI can tell is most of the time is cost on network transfer and the upgrading process have no difference psychological.
 
Old 05-09-2009, 11:15 AM   #78
Shingoshi
Member
 
Registered: Oct 2006
Location: Cochise County, Arizona
Distribution: Gentoo-AMD64 / Slackware64-Current
Posts: 474
Blog Entries: 28

Rep: Reputation: 34
I told you so!! And boy it feels good!!

Lasse, I have a deep sense of vindication in having advocated lzma from years ago.
And now that xz is reality in Slackware, there can be no question that I was right to do so.
Thank you for this sense of "I told you so", to everyone who argued against me for these past few years.

Starting back a few years ago, I started using pkgtools-tukaani (and tukbuild) for building my packages with lzma compression. Then with the help of src2pkg, I also started using lzma to compress my man pages. Now I will be switching once more. This time to xz. And in the same vein of logic, I have also been compressing the binary files in each package with upx-ucl. Again, it saves a lot of space and has virtually no effect on performance. upx is the same compressor that NASA used to transfer data between Earth and the Mars Explorer. So if they had sense enough to use it, I figured I should too. Maybe sometime in the future, Pat may actually start compressing man pages also. That's a bigger possibility than him compressing binaries with upx-ucl. So I'll just sit back and enjoy the fact that I'm receiving benefits that few if any others see as relevant.

Again, Thank you Lasse!
Shingoshi
 
Old 05-09-2009, 11:37 AM   #79
H_TeXMeX_H
LQ Guru
 
Registered: Oct 2005
Location: $RANDOM
Distribution: slackware64
Posts: 12,928
Blog Entries: 2

Rep: Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301
I think it's important to consider the speed of decompression, RAM usage, as well as compression ratio. So, don't think there is an ultimate right answer to this. I usually prefer gz over other compression algorithms because it's very fast at compression / decompression, uses little RAM, and has decent compression ratio. From what I've seen lzma is slower to decompress and requires more RAM. It's always a trade off ... always.
 
Old 05-09-2009, 02:07 PM   #80
Shingoshi
Member
 
Registered: Oct 2006
Location: Cochise County, Arizona
Distribution: Gentoo-AMD64 / Slackware64-Current
Posts: 474
Blog Entries: 28

Rep: Reputation: 34
xz Parallelized Operations to come...

Quote:
Originally Posted by H_TeXMeX_H View Post
I think it's important to consider the speed of decompression, RAM usage, as well as compression ratio. So, don't think there is an ultimate right answer to this. I usually prefer gz over other compression algorithms because it's very fast at compression / decompression, uses little RAM, and has decent compression ratio. From what I've seen lzma is slower to decompress and requires more RAM. It's always a trade off ... always.
The next step is to get xz parallelized like pigz (http://www.zlib.net/pigz/). This one is new to me.
Edit: I just created the pigz package in a few seconds using src2pkg.
There's also pbzip2 (http://compression.ca/pbzip2/) as well. This one I've used myself.

Shingoshi

Last edited by Shingoshi; 05-09-2009 at 02:41 PM.
 
Old 05-09-2009, 03:29 PM   #81
disturbed1
Senior Member
 
Registered: Mar 2005
Location: USA
Distribution: Slackware
Posts: 1,133
Blog Entries: 6

Rep: Reputation: 224Reputation: 224Reputation: 224
I wonder if the entire system was rebuilt. Noticed some cleaned up SlackBuilds, and a couple of gcc 4.3 fixes. I submitted some info on flac, considering it doesn't build on current, and was getting ready to submit a handful of others. The ones I had, have already been added.
 
Old 05-09-2009, 04:52 PM   #82
forum1793
Member
 
Registered: May 2008
Posts: 312

Rep: Reputation: 34
[Edit] Deleted my previous statement as it was wrong.
Just tried decompressing a mesa.tgz file. The file is around 85MB. It took about 5 seconds using both gz and xz the first time and 4 seconds for gz and 6 seconds for xz the second time to produce the .tar. I did not get fractions of a second so they're probably not too far apart.

If many people are using the rsync this could tie up the servers for a bit. It might be useful to offer a script to batch convert (some of) the existing tgz files on the end users machine. I don't understand enough about the scripting language to try this.

[Edit] Deleted erroneous statement about mesa, drm, etc., versions. I was comparing a slackware version to a git version and thought the slackware version was newer but upon review it's not.

Last edited by forum1793; 05-09-2009 at 07:46 PM.
 
Old 05-09-2009, 05:55 PM   #83
Alien Bob
Slackware Contributor
 
Registered: Sep 2005
Location: Eindhoven, The Netherlands
Distribution: Slackware
Posts: 8,559

Rep: Reputation: 8106Reputation: 8106Reputation: 8106Reputation: 8106Reputation: 8106Reputation: 8106Reputation: 8106Reputation: 8106Reputation: 8106Reputation: 8106Reputation: 8106
Quote:
Originally Posted by forum1793 View Post
Someone asked if it was taking longer in the new format. I didn't time this but to me it seems to be taking quite a bit longer. I would guess (totally a guess) that it is 1.5x to twice as long to decompress and install. I understand the capacity (disk and download bandwidth) issues and am not complaining.
Decompressing and installing a .txz package does not take 1.5x as long as a .tgz package. The decompression will be a bit slower but it will add much, much less than 50% extra time. Note that package decompression is only part of the package installation time.
Also I've verified that the -current installer still works on hardware that is equipped with just 64 MB of RAM.

Quote:
I also note there are upgrades to mesa, drm, and the radeon drivers.
There are no such updates to slackware-current, I do not know where you got that idea?

Eric
 
Old 05-09-2009, 07:19 PM   #84
niels.horn
Senior Member
 
Registered: Mar 2007
Location: Rio de Janeiro - Brazil
Distribution: Slackware64-current
Posts: 1,004

Rep: Reputation: 91
maybe the text in the tar-1.22-i486-2 package should be changed now:
Quote:
Slackware's package system uses tarfiles compressed with GNU gzip.
 
Old 05-09-2009, 07:36 PM   #85
Shingoshi
Member
 
Registered: Oct 2006
Location: Cochise County, Arizona
Distribution: Gentoo-AMD64 / Slackware64-Current
Posts: 474
Blog Entries: 28

Rep: Reputation: 34
Is there another alternative to increase rsync's performance?

I no longer use rsync. So I'm not familiar with the issues that are being raised about it. I used to use rsync when I did Gentoo distfiles updates some time ago. But, I was also using deltas (emerge-delta-webrsync) of the sources so that I was only downloading the differences between the older (existing) sources, and the newer sources. All that I remember was how fast and how much time was saved for each download. So my question is how could this be used for Slackware, now that some (think they) are experiencing lower performance as a result of the new .txz package format.

Shingoshi

Last edited by Shingoshi; 05-09-2009 at 08:03 PM.
 
Old 05-09-2009, 07:49 PM   #86
forum1793
Member
 
Registered: May 2008
Posts: 312

Rep: Reputation: 34
Quote:
Originally Posted by Alien Bob View Post
There are no such updates to slackware-current, I do not know where you got that idea?

Eric
You are right, wishful thinking on my part. These are probably too unstable still to consider even for -current.
 
Old 05-09-2009, 07:52 PM   #87
niels.horn
Senior Member
 
Registered: Mar 2007
Location: Rio de Janeiro - Brazil
Distribution: Slackware64-current
Posts: 1,004

Rep: Reputation: 91
rsync is not used by the actual upgrade process.

I use SlackPkg to maintain the boxes up-to-date (both stable & -current machines).
To avoid all systems downloading the same packages, I maintain a mirror of Slackware-12.2 & Slackware-current on a local ftp server. It is synchronized every night with rsync, when internet traffic is low.
 
Old 05-10-2009, 12:09 AM   #88
Woodsman
Senior Member
 
Registered: Oct 2005
Distribution: Slackware 14.1
Posts: 3,482

Rep: Reputation: 546Reputation: 546Reputation: 546Reputation: 546Reputation: 546Reputation: 546
Is there a way to make Ark from KDE 3.5.10 recognize the new txz compression format?

I installed the new a/xz package from current. I added the x-txz file association, modified the ark.desktop file and Konqueror now lists the *.txz files as XZ Archive. Double-clicking opens Ark, but I don't know of a way to add xz files to the drop-down list.

If there is no way to do this then that would be unfortunate.

Last edited by Woodsman; 05-10-2009 at 12:20 AM.
 
Old 05-10-2009, 01:53 AM   #89
rworkman
Slackware Contributor
 
Registered: Oct 2004
Location: Tuscaloosa, Alabama (USA)
Distribution: Slackware
Posts: 2,559

Rep: Reputation: 1351Reputation: 1351Reputation: 1351Reputation: 1351Reputation: 1351Reputation: 1351Reputation: 1351Reputation: 1351Reputation: 1351Reputation: 1351
Quote:
Originally Posted by Woodsman View Post
Is there a way to make Ark from KDE 3.5.10 recognize the new txz compression format?

I installed the new a/xz package from current. I added the x-txz file association, modified the ark.desktop file and Konqueror now lists the *.txz files as XZ Archive. Double-clicking opens Ark, but I don't know of a way to add xz files to the drop-down list.

If there is no way to do this then that would be unfortunate.
This is untested completely (not even compile tested), and I probably missed some needed bits, but it should get you on the right track: see the attachment for a diff to kdeutils-3.5.10.
Attached Files
File Type: txt kdeutils-3.5.10-support_xz_and_lmza_stuff.diff.txt (4.4 KB, 45 views)

Last edited by rworkman; 05-10-2009 at 10:31 AM.
 
Old 05-10-2009, 04:37 AM   #90
H_TeXMeX_H
LQ Guru
 
Registered: Oct 2005
Location: $RANDOM
Distribution: slackware64
Posts: 12,928
Blog Entries: 2

Rep: Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301
Quote:
Originally Posted by Alien Bob View Post
Decompressing and installing a .txz package does not take 1.5x as long as a .tgz package. The decompression will be a bit slower but it will add much, much less than 50% extra time. Note that package decompression is only part of the package installation time.
Also I've verified that the -current installer still works on hardware that is equipped with just 64 MB of RAM.


There are no such updates to slackware-current, I do not know where you got that idea?

Eric
I see, well that's good to hear.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Converting a compiled program in tgz format into an rpm package. xode Linux - Software 1 01-04-2008 12:54 PM
LXer: Linux Standard Base plans cross-format package API LXer Syndicated Linux News 0 01-17-2007 10:54 PM
apt package repositories/format question Michael_S Debian 2 12-05-2006 02:40 PM
Native package format wombat53 Slackware 8 06-13-2005 07:10 PM
Possible NEW package format Kocil Slackware 21 09-11-2004 06:30 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware

All times are GMT -5. The time now is 11:41 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration