I find rpm's of these tar-1.13 micro versions: .3, .6, .8, .11, .14, .17 and .25
If I recall correctly, some of the first changes made after tar-1.13(.0) were already borking compatibility with the slackware package format/behaviour, so I'm sure this contributed to there being no updates.

We have to assume that tar-1.13 has been kept because it does exactly what PatV wants it to do -or at least he thinks it does. I find it very strange that later versions seem to do the opposite of what the replacement syntax documentation says...

I confirm that this issue exists when archiving with tar-1.13 and extracting the tarball again with tar-1.13. In my opinion it takes some of imagination to call this bug a security issue and does not affect the pkgtools as packages are exploded at root level and there is no point securing anything because you are at mercy of the package at that point in any case. Moreover, a cautious user always inspects a tarball before extracting it.

The security issue in question respects uid permissions, so for example user tux1 cannot overwrite anything in /root even is he extract a malicious tarball.

As GazL said, it is improbable that this tar version is used for anything else other then for pkgtools.

I just downloaded the tar.SlackBuild from current and found this.

Even with the new compression schemes being added to pkgtools, they still rely on tar to package all the file together. I agree with gnashley that tar-1.13 will be used up until it will no longer compile. I am not sure how long slackware has been using this outdated version of tar (even if it is just for packages), but a solution will have to made. Even if it is not any time soon.

I am surprised that when gnu decided to change the way tar unpacked files (related to the above symlink issue) that a compatibility switch was not implemented.

I wasn't aware that the newer GNU tar versions did this. I'll have to keep an eye out for this in future as I do tend to symlink to directories quite a bit. Does a traditional i.e. non GNU version of tar such as the bsd one also clobber symlinks in this way?

If people are really concerned about security risks associated with this older version of tar, then move it to /sbin and change the perms so that only root can use it. That way a malicious user cannot trick others into using it as easily.

As far as pkgtools using it, if root is installing something bad you're screwed anyways (as previously stated).

"With newer versions of tar, installing any new package will remove the /opt symlink and plop down a new directory there." Actually, this is not true with thelatest versions. The problem comes when you try to combine the specific options of 'don't clobber links', (-U)nlink first and (-p)reserve-permissions. It gets really confusing because the syntax with the new versions appears to do the opposite of what you think it will.


"I am surprised that when gnu decided to change the way tar unpacked files (related to the above symlink issue) that a compatibility switch was not implemented."
The change was made in order to conform to POSIX standards. As stated above, this feature does work with some later versions, by default, even though it is not supposed to. But, using the -U option makes it not work. Apparently with tar-1.13 links are not unlinked first.

Does a traditional i.e. non GNU version of tar such as the bsd one also clobber symlinks in this way? I'm not sure about that. You'd need to check the version you are using by creating a test archive which contains a real directory in place of a linked directory which already exists. Play with the -U, -p and --no-overwrite-dir options.
What I have found that seems to do the right thing is:
tar-1.19 -xpv --one-file-system -f tarball-name
This seems to be stable from tar-1.16 onward. But I am unsure what criterion PatV was using for the -U option. If you add it, then the newer tar verisons will indeed replace a link with a dir. But, I think leaving it off may cause old files to not be overwritten with new ones. (Note that --one-file-system replaces the old '-l' option)

Heck, here are the notes I was writing the other day and mentioned:

* First, that new tars do create that first entry "./" as tar-1.13 does.
* Second, that filtering the output of tar when listing the tar contents with " | sed -e '1! s:^\./::' " makes the listing equal for both tar-1.13 and new tars and for both files created with the old tar and the new tar.

create a couple of test directories, each containing a file and a sub-directory
with another file inside that:
ls -R test1 test2
test1:
test1.txt testdir1

test1/testdir1:
test1.txt

test2:
test2.txt testdir2

test2/testdir2:
test2.txt

Then create a tar archive of the first one using a 'new' tar version. Use the same syntax
as used by makepkg -the tar files are created from *within* the package tree:
( cd test1 && tar cvf ../test1.tar . )

Then, do the same with the old version of tar:
( cd test2 && tar-1.13 cvf ../test2.tar . )

Then compare the listings 'crossing' the tar versions:

# list new-style archive with new tar
tar -tf test1.tar
./
./testdir1/
./testdir1/test1.txt
./test1.txt
# list the new-style archive with tar-1.13
tar-1.13 -tf test1.tar
./
./testdir1/
./testdir1/test1.txt
./test1.txt

# list the old-style archive with new tar
tar -tf test2.tar
./
testdir2/
testdir2/test2.txt
test2.txt

# list the old-style archive with old tar
tar-1.13 -tf test2.tar
./
testdir2/
testdir2/test2.txt
test2.txt

It's useful to compare what happens if you create the tar archives using this syntax:
( cd test1 && tar cvf ../test3.tar * )

( cd test2 && tar-1.13 cvf ../test4.tar * )

# list new-style archive with new tar
tar -tf test3.tar
test1.txt
testdir1/
testdir1/test1.txt

# list the new-style archive with tar-1.13
tar-1.13 -tf test3.tar
test1.txt
testdir1/
testdir1/test1.txt

# list the old-style archive with new tar
tar -tf test4.tar
test2.txt
testdir2/
testdir2/test2.txt

# list the old-style archive with old tar
tar-1.13 -tf test4.tar
test2.txt
testdir2/
testdir2/test2.txt

Now look at what installpkg does:

$TAR tzf $package 1> $TMP/tmplist$$ 2> /dev/null
...
cat $TMP/tmplist$$ | grep -v "/$" | while read file ; do
if [ -L "$ROOT/$file" ]; then
rm -f "$ROOT/$file"
fi
done

to check for existing links
And then to really install the package and create the file-listing at the same time:

( cd $ROOT/ ; $TAR -xzlUpvf - ) < $package >> $TMP/$shortname 2> /dev/null
if [ "`cat $TMP/$shortname | grep '^./' | wc -l | tr -d ' '`" = "1" ]; then
# Good. We have a package that meets the Slackware spec.
cat $TMP/$shortname >> $ADM_DIR/packages/$shortname
else
# Some dumb bunny built a package with something other than makepkg. Bad!
# Oh well. Bound to happen. Par for the course. Fix it and move on...
echo './' >> $ADM_DIR/packages/$shortname
cat $TMP/$shortname | grep -v '^./$' | cut -b3- >> $ADM_DIR/packages/$shortname
fi

Note that it's expecting a tar listing that begins with this entry: './'
It also implies the other archive entries are listed as relative paths

# list the old-style archive with old tar (the one made using '.', not '*')
tar-1.13 -tf test2.tar
./
testdir2/
testdir2/test2.txt
test2.txt

This format is critical because of what removepkg does:

if fgrep "./" $ADM_DIR/packages/$PKGNAME 1> /dev/null 2>&1; then
TRIGGER="^\.\/"
else
TRIGGER="FILE LIST:"
fi
if [ ! "$WARN" = true ]; then
echo "Removing files:"
fi
sed -n "/$TRIGGER/,/^$/p" < $ADM_DIR/packages/$PKGNAME | \
fgrep -v "FILE LIST:" | sort -u > $TMP/delete_list$$

The other aspect of starting to use a new version is this syntax:
( cd $ROOT/ ; $TAR -xzlUpvf - ) < $package

The '-l' option does something completely different with tar-1.14 than with tar-1.13.
The '-l' option disappears with tar-1.15
And, the above syntax implies the default behaviour which is 'critical' to not have
links to dirs overwritten by real dirs contained in the package. This must be explicitly
given now with --no-overwrite-dir
The -U option and -p need to be looked at as well, as their behaviour with different versions
of tar, uing the same syntax, gives variable results
This looks like it should be right:
( cd $ROOT/ ; $TAR -xzUpv --no-overwrite-dir --one-filesystem -f - ) < $package
but in fact, the 'U' (unlink first) option behaviour changes across tar-1.13/14/15/16.
With tar-1.13 the U option would only unlink *files* ??? With later versions it seems
to do the opposite of what you'd expect. Using tar-1.19 this works as desired:
( cd $ROOT/ ; $TAR -xzpv --one-filesystem -f - ) < $package
using the -U option with later versions causes the link to be overwritten by a real
dir contained in the archive

To test this out, create a new dir with a dummy directory inside:
mkdir -p test/dir
Then create links from dir to the names of the dirs inside test1.tar and test2.tar:
ln -sf test/dir test/testdir1
ln -sf test/dir test/testdir2
Then cd into 'test' and test unpacking test1.tar and test2.tar:
(cd test ; tar-1.13 -xlUpv -f ../test1.tar) #or tar-1.13 -xUpv --one-file-system -f ../test1.tar
That should leave the link existing. Now check if the new version works:
remove the archive contents
rm -f test/test1.txt
rm -rf dir/test1.txt
tar-1.19 -xUpv --one-file-system --no-overwrite-dir -f ../test1.tar
# I'm using tar-1.19, but I think this behaviour is consistent since 1.16
You'll see that overwrites the link test/testdir1 with the real directory testdir1
which test1.tar contains. Using:
tar-1.19 -xpv --one-file-system -f ../test1.tar
does give the right behaviour.

Can't you just use a transform?
(Note: '--show-transformed-names' is only needed for display purposes)

P.S. Whilst we are at it, it would be nice to set '--owner 0' and '--group 0' as well so that it is possible to run makepkg as a non root user. ;)

I have also noticed that spkg will fix a package with './usr' style formatting on install such that the log makes it look like it was created 'correctly'.

@gnashley: I had a quick scan through your notes. So how about this as an installpkg.patch for installpkg?
Note: I did not use '--one-file-system' because this is extraction and 'man tar' (tar-1.23) states that this option is to "stay in local file system when creating archive" (emphasis mine).

And this as a makepkg.patch for makepkg?
Note: I did not set '--owner 0' or '--group 0' as perhaps a user would have some reason for creating a package with different owner and group.

If it's only the symlink problem, I think "cpio -i" will work as well.

Why would you exploit a ../.. traversal in tar (which is documented in the tar documentation, btw), when you could just put anything you like in a shell script?

tar-1.13 is fine for the package utilities. In fact one of the "solutions" to that tar issue that was proposed was making tar refuse to extract a file through a symlink, which would have broken it for our purposes. It's better not to have to chance it every time tar gets changed upstream.

If it's not broke fix it till it is. :)

Since it is documented, than the users have been warned.

guanx, cpio is not a good option as it arbitrarily creates new directories chmod 755 if they don't already exist.

Creating passable packages using modern tar is possible alright -it's the extraction of them that is the problem -even if the file paths can be fixed by using transform. The real problem is that links shouldn't be overwritten by a real directory included in the archive(non-posix behaviour 'fixed' in later versions of tar), and that existing directory metadata shouldn't be overwritten. I think the latter may be possible with a combination of later tar options. It would mean *not* using 'U' (unlink first).

Glad to see PatV chime in here! Have I really gotten it right about the three issues and the reason for sticking with tar-1.13? Are there any other desirable behaviours of tar-1.13 that I've overlooked? Mind you, I'm more than happy to keep using since we know what it does.