I have been a slackware user for quite sometime now, and just have a question. Why does pkgtools rely on tar-1.13 and not a newer version? Perhaps this was disclosed some time ago, but I missed it. I did a quick search on the forums came up empty.

It uses tar-1.13 because of a couple of 'quirks' with that version which do exactly what PatV wants ot to do when creating and installing packages. Actually the part about what happens when creating the packages is really related to the way that tar-1.13 lists the contents of the package. If you look at the database file for any official package in /var/log/packages, you'll see that the contents are listed like this:
./
usr/bin
var/log

That is, the listing (using the tar-1.13 '-t' option) begins with the toplevel of the tar archive shown as './' or the current directory. Then, subsequent entries are shown with a relative path.
Later versions of tar don't show the toplevel directory. This in itself is no problem. The problem with beginning to use a later version of tar is that tar-1.13 actually creates these entries with a leading './' -it cuts of the first two characters when using the -t option. So, if you create an package with a later version of tar and then try to install it with tar-1.13 the first two characters of the path get cut off. The above example would show like this:
r/bin
r/log

Okay, now the other part of the quirks - tar-1.13 used a non-POSIX-compliant default when installing linked directories. The best way to understand is with an example where your system has a link which points to a directory -like /usr/share/doc which points to /usr/doc. If you create a package which contains /usr/share/doc as a real directory, when you install it with tar-1.13 the existing link will not be overwritten by the installed package. Later versions of tar *will* overwrite the existing link and replace it with a real directory.

I am not sure, because I have never asked PatV about it, but I think part of his criteria for using tar-1.13 may have to do with the way tar-1.13 handles directory permissions.
A couple of years ago I did a lot of work to figure out how all this was supposed to work as I wanted to use a later version of tar with my src2pkg program. I did get the syntax worked out to use a later version of tar and correctly handle the links and perms(metadata), but the first problem with the difference in the way archives are listed appears to have no reliable resolution. Since the pkgtools have recently been modified to include new formats, I do ask myself if PatV might want to make the plunge and start using a newer tar -there will probably be problems some day with compiling tar-1.13 with newer glibc/gcc combinations. But, making the switch would be a little uncomfortable, I think, as it would involve some hackish workarounds in order to have files properly listed in the database. And the file listing is not simply a cosmetic matter -removepkg uses that list to remove files installed by the package, so the paths must be correct or no files will be removed when removepkg is run(except the database file *** that means you lose the listing -that pkgtools would show the packages as being removed when in fact all the package files are still present).

12 members found this post helpful.

Great detailed explanation, gnashley!

I remembered reading some discussions about this tar-1.13 dependency some time (years?) ago somewhere on the internet but couldn't find it. But while I was still googling, your post arrived here

I wonder if there's a way to patch newer versions, just so you can have a newer version of tar with all the fixes that might have been implemented, but still be able to use it in this way.

It is possible to make newer versions of tar to work with older packages, but getting packages created with a newer tar to work with older versions of tar would be more difficult. Again, the real problem is not with the way entries are actually created in the archive, rather with the way they get listed by installpkg. And doing an upgrade to a newer version would require that the new pkgtools be installed twice -once by the old version of installpkg and then again by the new version -just to get the files listed correctly. A couple of changes to installpkg could make it possible to handle either old or new packages, but this still would be very hackish. I guess an interim version of pkgtools could be created which uses both old and new tar versions in order to be able to handle old or new packages -but my guess is that Pat will stick with tar-1.13 till the bitter end -I mean until it will no longer compile with gcc/glibc versions being used. And since the tar package doesn't get rebuilt too often, the problem would probably be right on top of us before being discovered. When the day comes, it will force a rebuild of every package in the distro -which has never been done all at once, AFAIK. Package format is a critical subject if you want to have any sort of backward or forward compatibility...

I pulled a lot of hair working on this problem for src2pkg -but I finally ended up sticking with tar-1.13 and adding some extra patches to add a couple of key features introduced later, including auto-detection of compression type and preservation of directory metadata. At the time I was doing most of this work, tar was really in a state of flux -version 1.14 and 1.15 were interim versions for a couple of the key 'quirks' and Slackware was shipping with those versions as the normal tar. So any comprehensive solution which would be compatible back to slackware 8.1 or 9.0 would have to detect the version of tar installed and adjust the syntax accordingly -messy at best!
My bet is we'll wind up at a 'watershed' distro release which will break backwards-compatibility, or we'll see tar-1.13 patched to continue being able to compile it with newer gcc/glibc.

Ok, thanks, I understand it better now.

Maybe this is why the new tbz, tlz, etc. formats were added ? Maybe Pat wants to move away from tgz for this reason ?

tgz versus the other compression schemes is a separate issue. There's no expectation of backwards compatibility, and it's easy to tell whether a package will work on your system or not. I'm careful to use the term "compression scheme" rather than "format" because the things that need to be thought about are different.

It's a time of change, so a switch to the new tar might actually happen.

Maybe sometime when I'm bored I'll add a --quirks option so that newer tar can create old tar style packages. Right now I'm having more than enough fun writing a safer sentry.

I've been doing some tests with /bin/tar and /bin/tar-1.13 creating gzipped tar archives and I think I understand the problem you are describing. It looks like tar-1.13, as you said, creates the list of files inside the archive storing paths differently from more recent versions of tar. However, I have verified two things.

* First, that new tars do create that first entry "./" as tar-1.13 does.
* Second, that filtering the output of tar when listing the tar contents with " | sed -e '1! s:^\./::' " makes the listing equal for both tar-1.13 and new tars and for both files created with the old tar and the new tar.

So I'm wondering if something as simple as that, combined with your solutions to the second set of problems, couldn't be used to escape from the dependency on old versions of tar.

Given that the pkgtools rely on the tar package itself, is it technically possible for Slackware to send updates on this package if security issues are reported?

If you were going to make a malicious package, you'd just put the nasty payload in doinst.sh so any arbitrary code execution issues really don't matter in this context.

I was not refering to any generic malicious code. My reference was limited to the tar package itself. How could Pat deal with any reports of Security issues related to the tar package (i.e. tar-1.6.1-i486-1.tgz) itself. Can it be upgraded quickly? I don't think so.

Another very minor issue is that this tar package conflicts with the dump Slackbuild.

I'm having a hard time imagining any other types of issues that could possibly surface. This is a specific version of tar included specifically for the use of pkgtool, so it's not going to be used by users during day to day operation (they'll use the newer tar version). pkgtool is generally run as root when installing anyway, so any privilege escalation vulnerabilities aren't a worry (you're already at the highest level) and even if it wasn't 'tar' is not suid and relies on the users own privileges anyway, so no chance of escallation.

So, what's left? 'tar' could be made to overwrite files that the user has permissions to write to? Well, yes... but that's sort of what it does anyway, and is why its always a good idea to do a '-tvf' before you '-xvf'.

Anyway, being forced to upgrade because of security issues is more of a proprietary software issue. With open source you also have the options of just patching your current version yourself if necessary, and if necessary, I guess Pat would do just that.

The really tricky part is getting newer versions to do what you want to match what the combination of -lUp does when installing a package, plus not overwriting existing links with actual dirs from the archive.
I've busted my head the whole morning on this again, just to referesh my meory and notes about. The upshot is that newer tar versions do not give the desired behaviour using the replacement syntax as documented -replacing '-l' with '--no-overwrite-dir' reverses the behaviour, using U and p together also gives different results bewteen old and new versions. Oh my head...

I made a quick Google search for Security issues for tar-1.13 and I only found the following:

Thus if we stick to tested packages, like those from Slackbuilds, or those we make ourselves, there should not be any risks. Still, the risk exists and users should be warned.

Chris

Slackware uses tar-1.13 -there are a whole bunch of intermediate versions between 1.13 and 1.14 spread over several years. Unless that vulnerbaility is present in tar-1.13 it has no bearing and I think it is not. Be encouraged to test it out though -if the problem really is there in tar-1.13 it might provide an extra incentive to upgrade.