LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware
User Name
Password
Slackware This Forum is for the discussion of Slackware Linux.

Notices


Reply
  Search this Thread
Old 02-19-2011, 04:49 PM   #46
ruario
Senior Member
 
Registered: Jan 2011
Location: Oslo, Norway
Distribution: Slackware
Posts: 2,557

Rep: Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761

Quote:
Originally Posted by NonNonBa View Post
I didn't test, but I would say that your gain are not significative (countable in seconds) unless you do a huge install at once (what you never do in fact, as the Slackware installer use the pkgtools)
You aren't considering the fact that some people do slim install initially and then later decide to install a large selection of packages. For example I might have a fairly small desktop style setup with just Xfce and then later decide to switch to KDE and mass install all the KDE components.
 
Click here to see the post LQ members have rated as the most helpful post in this thread.
Old 02-20-2011, 05:22 AM   #47
NonNonBa
Member
 
Registered: Aug 2010
Distribution: Slackware
Posts: 192

Rep: Reputation: Disabled
Quote:
Originally Posted by gnashley
NonNonBa, it's not the 'rm' calls themselves that take so long -it's the chceking to see if the file is claimed by any other package before removing it that takes so much 'grep file /var/log/packages/*' for every file a package contains makes a lot of work...
No, if I remember well, it concatenates all the logs content but the one of the removed package in a buffer with cat, then use comm to make the diff. So, the diff must fit in four-five processes. Only the first should be really time-consuming, as the logs have not been accessed yet, I think.

Quote:
Originally Posted by ruario
Firstly I don't really see the point of replacing the tar format with cpio and secondly I don't personally like the packages that the spack tools make. In my eyes the internal structure is overly complex, which means that making small edits to them after creation suddenly becomes a major hassle unless you entirely commit yourself to using the spack tools.
On the cpio choice: cpio has the ability of checking file integrity through its CRC format, is able to store node devices and is required on a Slackware to create ramdisk (I've also read some debats -- on the lkml, I think -- on the reason why kernel devs chose cpio instead or tar to format ramdisk).

On the spack package format: it seems complex because there are two cpio archives, but at the end you just need cpio and xz/gzip. The first archive contains the "doinst.sh", the slack-desc, a special but very straightforward shell script which inits the dir tree (it allows to the archiver to just manage the files), and another compressed cpio archive containing the files. The goal is to be able to access to the install stuff without having to decompress anything (pkgtools decompress the package to get the "slack-desc" if the dedicated .txt is not found along). It's not so hard to edit : put your files in a directory, edit them, make a compressed cpio archive, put the archive in to the place where the install stuff is, add directories to the "pre-install.sh" script if needed, make a cpio archive, end of the story. Of course, it will be easier with the spack-tools, but it's the same with the package tools: you'll never do as many checking as makepkg does; if you want an orthodox package, you have to use the packaging tools, whatever they are. The purpose of a package toolkit is just to convey files from a packager to an installer, it don't think using a format easily editable by a third-party tools is a major feature in this story.

Quote:
Originally Posted by ruario
To be clear however, that is not to say what NonNonBa has done isn't clever. It is just a statement that spack isn't for me 'personally'. If someone else finds it useful then great!
Of course, it's your right! I don't claim Spack is the End of the packaging tool History.

Quote:
Originally Posted by ruario
You aren't considering the fact that some people do slim install initially and then later decide to install a large selection of packages. For example I might have a fairly small desktop style setup with just Xfce and then later decide to switch to KDE and mass install all the KDE components.
Ok, let say you gain 10 minutes (what I think you don't): what are ten minutes regarding the time you will spend in configuring you fresh Slack or in checking/fixing all the points mentioned in the CHANGES_AND_HINTS.TXT? That's my point. Of course, as developer, it's fun to optimize things, but I don't think it is or should be so important for the end-user.
 
Old 02-20-2011, 05:40 AM   #48
ruario
Senior Member
 
Registered: Jan 2011
Location: Oslo, Norway
Distribution: Slackware
Posts: 2,557

Rep: Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761
Quote:
Originally Posted by NonNonBa View Post
It's not so hard to edit : put your files in a directory, edit them, make a compressed cpio archive, put the archive in to the place where the install stuff is, add directories to the "pre-install.sh" script if needed, make a cpio archive, end of the story.
As opposed to edit the files and re-targz. :P

Quote:
Originally Posted by NonNonBa View Post
... don't think using a format easily editable by a third-party tools is a major feature in this story.
For me it is a major feature. That is my absolute favourite thing about Slackware packages and the reason I really dislike RPM.

Your package format reminds me of deb and sure you can edit debs with standard tools but it becomes more involved. I wouldn't personally be willing to sacrifice this easy of use. If you want to quickly extract the meta data an easier solution would just be to always put meta data files ("install/") at the start of the tar archive. Arch packages do this with their meta data files. You don't need multiple levels of archives within each other.

Quote:
Originally Posted by NonNonBa View Post
Ok, let say you gain 10 minutes (what I think you don't): what are ten minutes regarding the time you will spend in configuring you fresh Slack or in checking/fixing all the points mentioned in the CHANGES_AND_HINTS.TXT? That's my point. Of course, as developer, it's fun to optimize things, but I don't think it is or should be so important for the end-user.
I think you are seriously underestimating how impatient I am! :P

Last edited by ruario; 02-20-2011 at 06:02 AM. Reason: added point about why I like the Slackware package format
 
Old 02-20-2011, 06:14 AM   #49
gnashley
Amigo developer
 
Registered: Dec 2003
Location: Germany
Distribution: Slackware
Posts: 4,928

Rep: Reputation: 612Reputation: 612Reputation: 612Reputation: 612Reputation: 612Reputation: 612
ruario, I agree with your dislike of rpm's for exactly the same reason. Worse still, even if you have some version of rpm around, it will not be able to open all versions of rpm archives. Of course, the use of tar-1.13 for slack suffers some from the same problem -it would take a long from user X to figure out how to properly unpack (or list) a tar-1.13 archive without thinking it was corrupted.

On that note, I remember a quirk of the 'pet' packages for Puppy Linux, which are really just 'tar.gz' archives. But, they then cat the md5sum of the archive onto the end of the package and this makes tar report an error every time you unpack one -unless you truncate the package to remove the md5sum -huh, the ideas that some people get..... (some peoples kids, I mean)
 
Old 02-20-2011, 08:33 AM   #50
ruario
Senior Member
 
Registered: Jan 2011
Location: Oslo, Norway
Distribution: Slackware
Posts: 2,557

Rep: Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761
Ok, I am still playing around with these options and I have yet another method for package creation.

Code:
find ./ | sed "s,^\./\(.\),\1," > ../example-1.0-i486-1.filelist
tar --no-recursion -T ../example-1.0-i486-1.filelist -cvf- | xz -c > ../example-1.0-i486-1.txz
./
install/
install/slack-desc
usr/
usr/man/
usr/man/man1/
usr/man/man1/example.1.gz
usr/bin/
usr/bin/example
usr/doc/
usr/doc/example-1.0/
usr/doc/example-1.0/COPYING
usr/doc/example-1.0/ChangeLog
usr/doc/example-1.0/README
Why do it this way? Because on further investigation it seems that '--transform' is not particularly backwards compatible. It only appeared in GNU tar 1.20. Even Slackware 12.2 didn't have tar-1.20+, so you can't always count on it always being available. This is presumably why nobody thought to mention it before, given the age of this thread. Damn and I thought I been so clever noticing something really obvious. I guess not!

However, both '--no-recursion' and '-T' (a.k.a '--files-from') are really old options. They have been in GNU tar since even before the release of version 1.13. So this latest method is the most backwards compatible of the examples I have provided so far, and there should be no chance of ending up with problems related to the argument list being too long. I even tested this method with GNU tar version 1.14 from Slackware 10.0 and it seems to create the packages in the correct format.

Hence it could potentially be used by tools like src2pkg for package creation, rather than bundling GNU tar version 1.13. Just a thought!

Last edited by ruario; 02-20-2011 at 08:55 AM. Reason: Altered the example slightly to avoid changing the timestamp on ./
 
Old 02-20-2011, 09:48 AM   #51
ruario
Senior Member
 
Registered: Jan 2011
Location: Oslo, Norway
Distribution: Slackware
Posts: 2,557

Rep: Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761
Hmm ... I just realised that this discovery means that my own specialised conversion script op2slk (for converting Opera tar packages into Slackware style packages) would not work on anything older than Slackware 13. Opps! I have now updated it to use this new method instead.
 
Old 02-20-2011, 09:58 AM   #52
NonNonBa
Member
 
Registered: Aug 2010
Distribution: Slackware
Posts: 192

Rep: Reputation: Disabled
Quote:
Originally Posted by ruario
Your package format reminds me of deb and sure you can edit debs with standard tools but it becomes more involved.
Why would you do that? If you want to edit a deb package, it is better to install the tools dedicated to this task. You can't blame nails because they are not easy to hammer using pliers, can you?

On this subject, there is even a funny comment from Pat in installpkg :

Code:
if [ "$(cat $TMP/$shortname | grep '^\./' | wc -l | tr -d ' ')" = "1" ]; then
    # Good.  We have a package that meets the Slackware spec.
    cat $TMP/$shortname >> $ADM_DIR/packages/$shortname
  else
    # Some dumb bunny built a package with something other than makepkg.  Bad!
    # Oh well.  Bound to happen.  Par for the course.  Fix it and move on...
    echo "WARNING:  Package has not been created with 'makepkg'"
    echo './' >> $ADM_DIR/packages/$shortname
    cat $TMP/$shortname >> $ADM_DIR/packages/$shortname
fi
You see, obey the Great Slacker, and use the good tool for the good task(, "dumb bunny").

Quote:
Originally Posted by ruario
I think you are seriously underestimating how impatient I am!
But you can't even be impatient, here! All is in your head.

I mean, you can be impatient when your browser takes 2s to load a page, because you endure it, but this is not the case when your package manager consumes some more minutes during an installation. It's like loosing a drop from a glass of water, you can't say you feel it less quenchs your thirst.
 
Old 02-20-2011, 10:26 AM   #53
gnashley
Amigo developer
 
Registered: Dec 2003
Location: Germany
Distribution: Slackware
Posts: 4,928

Rep: Reputation: 612Reputation: 612Reputation: 612Reputation: 612Reputation: 612Reputation: 612
One other thing to think about is, that the issue with tar-1.13 has to do with the way it lists the archives -that is what gives the desired output for the database files. The internal format is different and looks like this:
./
./usr/path
./opt/old/path
Listing the archive comes out like this:
./
usr/path
opt/old/path
because tar-1.13 eats the first two chars of the entries except for the base './' dir, so the internal format is the same as with later versions when archives are created using 'tar -cf filename .'

This whole thread now brings painful memories of when I was trying to figure it all out...
 
Old 02-20-2011, 11:27 AM   #54
ruario
Senior Member
 
Registered: Jan 2011
Location: Oslo, Norway
Distribution: Slackware
Posts: 2,557

Rep: Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761
Quote:
Originally Posted by NonNonBa View Post
Why would you do that? If you want to edit a deb package, it is better to install the tools dedicated to this task.
Sure it can be but the point is I shouldn't have to! I flip between a variety of machines at work and I might prepare the file on one and then use them on another. That is why like simple formats. They gives me more options and hence more flexibility.

Quote:
Originally Posted by NonNonBa View Post
You see, obey the Great Slacker, and use the good tool for the good task(, "dumb bunny").
I think you are reading too much into that and possibly missinterpretting the comment. The point here is that someone made a non compatible package and hence they are "dumb bunny" for not fully understanding what you are doing before attempting it.

By all means start with the provided tools but if you want something more flexible I think you should be able to. Furthermore, I don't believe that the Slackware package format was intentionally designed to be complicated. I believe it was designed to be as simple as possible (like everything else in Slackware). The fact that it is slightly more complex now is purely down to the fact that Slackware is an old distro and its long history has meant that things have changed. PatV could not have predicted the changes to the format of (GNU) tar packages from the onset, so when things did change it unintentionally became more complex. However your format is more complex then it needs be from the onset, at least in my eyes, others may disagree.
 
Old 02-20-2011, 11:33 AM   #55
ruario
Senior Member
 
Registered: Jan 2011
Location: Oslo, Norway
Distribution: Slackware
Posts: 2,557

Rep: Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761
Quote:
Originally Posted by gnashley View Post
The internal format is different and looks like this:
./
./usr/path
./opt/old/path
No the internal format is also minus the './' on all but the first entry. Download a file that was obviously created by tar 1.13 like say the package I linked earlier:

http://ftp.it.be/packages/slackware/....14-i486-4.tgz

If you 'tar tf' it in a modern GNU tar (or even BSD tar) its format is displayed as:

./
usr/path
opt/old/path

So all you need to do is make your own packages follow this convertion and you are good to go.
 
Old 02-20-2011, 11:37 AM   #56
ruario
Senior Member
 
Registered: Jan 2011
Location: Oslo, Norway
Distribution: Slackware
Posts: 2,557

Rep: Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761
Quote:
Originally Posted by NonNonBa View Post
I mean, you can be impatient when your browser takes 2s to load a page, because you endure it, but this is not the case when your package manager consumes some more minutes during an installation.
Both annoy me equally.
 
Old 02-20-2011, 01:26 PM   #57
ruario
Senior Member
 
Registered: Jan 2011
Location: Oslo, Norway
Distribution: Slackware
Posts: 2,557

Rep: Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761
@gnashley: Here are two packages:

1. example-1.0-noarch-1ro.tgz
2. example-1.0-noarch-1ro.tgz

The first was made the official way (with an unhacked makepkg), like so:
Code:
makepkg -c n ../example-1.0-noarch-1ro.tgz
The second was made with a copy of tar version 1.25 (that I compiled myself), as follows:
Code:
find ./ | sed "s,^\./\(.\),\1," > ../example-1.0-noarch-1ro.filelist
tar --no-recursion -T ../example-1.0-noarch-1ro.filelist -cf- | gzip -9c > ../example-1.0-noarch-1ro.tgz
Can you detect any difference between the files testing with the tars included with Slackware 13.1 (tar-1.13, tar-1.23 and bsdtar-2.8.0)? Or for that matter installing and uninstalling with Pkgtools? If you can see the difference, how are you detecting it?

I can't see any difference. Well the file sizes are slightly different but that is it.

Last edited by ruario; 02-21-2011 at 08:39 AM. Reason: added the mention of BSD tar
 
Old 02-20-2011, 05:58 PM   #58
Darth Vader
Senior Member
 
Registered: May 2008
Location: Romania
Distribution: DARKSTAR Linux 2008.1
Posts: 2,727

Rep: Reputation: 1247Reputation: 1247Reputation: 1247Reputation: 1247Reputation: 1247Reputation: 1247Reputation: 1247Reputation: 1247Reputation: 1247
BTW, You known that in the libarchive package is shipped also bsdtar and bsdcpio?
 
Old 02-21-2011, 12:24 AM   #59
ruario
Senior Member
 
Registered: Jan 2011
Location: Oslo, Norway
Distribution: Slackware
Posts: 2,557

Rep: Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761Reputation: 1761
Well I do, which is why I mentioned BSD tar earlier. Actually I am a big fan of the libarchive tar and cpio utilities, on the whole preferring them to their GNU counterparts. Not least because libarchive allows BSD tar and BSD cpio to open a wide range of formats (even rpm files or the ar archives used by deb). It is also nice that BSD tar (and cpio) can automatically detect the file format, including compression, even when passed a file in via a pipe (something GNU tar/cpio cannot do), so you never have to specify compression switches on extraction. BSD tar has other nice features like being able to merge one archive received through a pipe with another it is creating. The only thing I don't like quite as much is BSD tar's '-s', since it is not quite as powerful as the GNU tar '--transform' equivalent but I won't be surprised if that changes in the future. libarchive is very actively maintained.

Anyway none of these utilities, GNU tar, GNU cpio (using -Hustar), BSD tar and BSD cpio, seem to be able to distinguish between the formatting of my two example files.

Last edited by ruario; 02-21-2011 at 12:26 AM.
 
Old 02-21-2011, 01:03 PM   #60
NonNonBa
Member
 
Registered: Aug 2010
Distribution: Slackware
Posts: 192

Rep: Reputation: Disabled
Quote:
Originally Posted by ruario
I think you are reading too much into that and possibly missinterpretting the comment. The point here is that someone made a non compatible package and hence they are "dumb bunny" for not fully understanding what you are doing before attempting it.
Do you know an official Slackware package specification? Except "use makepkg", I don't. This is the only official, stable and straightforward way to get an orthodox Slackware package.

On the other hand, it's also true your Slackware is your Slackware. So, you have the right to be on your own, breaking everything to run the wild path... and having fun! My opinions on the subject sit far below that point; so, good luck!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
BackUp & Restore with TAR (.tar / .tar.gz / .tar.bz2 / tar.Z) asgarcymed Linux - General 5 12-31-2006 02:53 AM
Slack 10.2 pkgtools in 10.0 linuxhippy Slackware 3 01-17-2006 09:54 AM
pkgtools Nightmare HELP!!!!!!!!!! waterox Slackware 4 09-12-2005 12:46 PM
pkgtools problem??? brbrow Slackware 2 09-10-2003 01:58 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware

All times are GMT -5. The time now is 10:37 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration