LinuxQuestions.org
Did you know LQ has a Linux Hardware Compatibility List?
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware
User Name
Password
Slackware This Forum is for the discussion of Slackware Linux.

Notices

Reply
 
Search this Thread
Old 04-27-2009, 03:41 PM   #16
gnashley
Amigo developer
 
Registered: Dec 2003
Location: Germany
Distribution: Slackware
Posts: 4,768

Rep: Reputation: 477Reputation: 477Reputation: 477Reputation: 477Reputation: 477

I find rpm's of these tar-1.13 micro versions: .3, .6, .8, .11, .14, .17 and .25
If I recall correctly, some of the first changes made after tar-1.13(.0) were already borking compatibility with the slackware package format/behaviour, so I'm sure this contributed to there being no updates.

We have to assume that tar-1.13 has been kept because it does exactly what PatV wants it to do -or at least he thinks it does. I find it very strange that later versions seem to do the opposite of what the replacement syntax documentation says...
 
Click here to see the post LQ members have rated as the most helpful post in this thread.
Old 04-28-2009, 03:05 PM   #17
ChrisAbela
Member
 
Registered: Mar 2008
Location: Malta
Distribution: Slackware 14.1
Posts: 467

Rep: Reputation: 89
I confirm that this issue exists when archiving with tar-1.13 and extracting the tarball again with tar-1.13. In my opinion it takes some of imagination to call this bug a security issue and does not affect the pkgtools as packages are exploded at root level and there is no point securing anything because you are at mercy of the package at that point in any case. Moreover, a cautious user always inspects a tarball before extracting it.

The security issue in question respects uid permissions, so for example user tux1 cannot overwrite anything in /root even is he extract a malicious tarball.

As GazL said, it is improbable that this tar version is used for anything else other then for pkgtools.
 
Old 04-28-2009, 05:25 PM   #18
tank728
Member
 
Registered: Sep 2003
Posts: 142

Original Poster
Rep: Reputation: 16
I just downloaded the tar.SlackBuild from current and found this.

Quote:
# This old version is the only one that won't clobber synlinks, e.g.:
# someone moves /opt to /usr/opt and makes a symlink. With newer
# versions of tar, installing any new package will remove the /opt
# symlink and plop down a new directory there.
# Well, there's a lot of other bugs (the remote stuff particularly I'm
# told is flaky) in tar-1.13, so it'll only be here now for use by the
# Slackware package utils. And, we'll even let people remove it and
# the pkgutils will still try to work (but eventually they'll pay the
# price
Even with the new compression schemes being added to pkgtools, they still rely on tar to package all the file together. I agree with gnashley that tar-1.13 will be used up until it will no longer compile. I am not sure how long slackware has been using this outdated version of tar (even if it is just for packages), but a solution will have to made. Even if it is not any time soon.

I am surprised that when gnu decided to change the way tar unpacked files (related to the above symlink issue) that a compatibility switch was not implemented.
 
Old 04-28-2009, 05:41 PM   #19
GazL
Senior Member
 
Registered: May 2008
Posts: 3,440

Rep: Reputation: 959Reputation: 959Reputation: 959Reputation: 959Reputation: 959Reputation: 959Reputation: 959Reputation: 959
I wasn't aware that the newer GNU tar versions did this. I'll have to keep an eye out for this in future as I do tend to symlink to directories quite a bit. Does a traditional i.e. non GNU version of tar such as the bsd one also clobber symlinks in this way?
 
Old 05-02-2009, 01:42 PM   #20
shadowsnipes
Senior Member
 
Registered: Sep 2005
Distribution: Slackware
Posts: 1,441

Rep: Reputation: 70
Quote:
Originally Posted by ChrisAbela View Post
I made a quick Google search for Security issues for tar-1.13 and I only found the following:



Thus if we stick to tested packages, like those from Slackbuilds, or those we make ourselves, there should not be any risks. Still, the risk exists and users should be warned.

Chris
If people are really concerned about security risks associated with this older version of tar, then move it to /sbin and change the perms so that only root can use it. That way a malicious user cannot trick others into using it as easily.

As far as pkgtools using it, if root is installing something bad you're screwed anyways (as previously stated).
 
Old 05-02-2009, 02:06 PM   #21
gnashley
Amigo developer
 
Registered: Dec 2003
Location: Germany
Distribution: Slackware
Posts: 4,768

Rep: Reputation: 477Reputation: 477Reputation: 477Reputation: 477Reputation: 477
"With newer versions of tar, installing any new package will remove the /opt symlink and plop down a new directory there." Actually, this is not true with thelatest versions. The problem comes when you try to combine the specific options of 'don't clobber links', (-U)nlink first and (-p)reserve-permissions. It gets really confusing because the syntax with the new versions appears to do the opposite of what you think it will.


"I am surprised that when gnu decided to change the way tar unpacked files (related to the above symlink issue) that a compatibility switch was not implemented."
The change was made in order to conform to POSIX standards. As stated above, this feature does work with some later versions, by default, even though it is not supposed to. But, using the -U option makes it not work. Apparently with tar-1.13 links are not unlinked first.

Does a traditional i.e. non GNU version of tar such as the bsd one also clobber symlinks in this way? I'm not sure about that. You'd need to check the version you are using by creating a test archive which contains a real directory in place of a linked directory which already exists. Play with the -U, -p and --no-overwrite-dir options.
What I have found that seems to do the right thing is:
tar-1.19 -xpv --one-file-system -f tarball-name
This seems to be stable from tar-1.16 onward. But I am unsure what criterion PatV was using for the -U option. If you add it, then the newer tar verisons will indeed replace a link with a dir. But, I think leaving it off may cause old files to not be overwritten with new ones. (Note that --one-file-system replaces the old '-l' option)
 
1 members found this post helpful.
Old 05-02-2009, 02:11 PM   #22
gnashley
Amigo developer
 
Registered: Dec 2003
Location: Germany
Distribution: Slackware
Posts: 4,768

Rep: Reputation: 477Reputation: 477Reputation: 477Reputation: 477Reputation: 477
Heck, here are the notes I was writing the other day and mentioned:

* First, that new tars do create that first entry "./" as tar-1.13 does.
* Second, that filtering the output of tar when listing the tar contents with " | sed -e '1! s:^\./::' " makes the listing equal for both tar-1.13 and new tars and for both files created with the old tar and the new tar.

create a couple of test directories, each containing a file and a sub-directory
with another file inside that:
ls -R test1 test2
test1:
test1.txt testdir1

test1/testdir1:
test1.txt

test2:
test2.txt testdir2

test2/testdir2:
test2.txt

Then create a tar archive of the first one using a 'new' tar version. Use the same syntax
as used by makepkg -the tar files are created from *within* the package tree:
( cd test1 && tar cvf ../test1.tar . )

Then, do the same with the old version of tar:
( cd test2 && tar-1.13 cvf ../test2.tar . )

Then compare the listings 'crossing' the tar versions:

# list new-style archive with new tar
tar -tf test1.tar
./
./testdir1/
./testdir1/test1.txt
./test1.txt
# list the new-style archive with tar-1.13
tar-1.13 -tf test1.tar
./
./testdir1/
./testdir1/test1.txt
./test1.txt

# list the old-style archive with new tar
tar -tf test2.tar
./
testdir2/
testdir2/test2.txt
test2.txt

# list the old-style archive with old tar
tar-1.13 -tf test2.tar
./
testdir2/
testdir2/test2.txt
test2.txt

It's useful to compare what happens if you create the tar archives using this syntax:
( cd test1 && tar cvf ../test3.tar * )

( cd test2 && tar-1.13 cvf ../test4.tar * )

# list new-style archive with new tar
tar -tf test3.tar
test1.txt
testdir1/
testdir1/test1.txt

# list the new-style archive with tar-1.13
tar-1.13 -tf test3.tar
test1.txt
testdir1/
testdir1/test1.txt

# list the old-style archive with new tar
tar -tf test4.tar
test2.txt
testdir2/
testdir2/test2.txt

# list the old-style archive with old tar
tar-1.13 -tf test4.tar
test2.txt
testdir2/
testdir2/test2.txt

Now look at what installpkg does:

$TAR tzf $package 1> $TMP/tmplist$$ 2> /dev/null
...
cat $TMP/tmplist$$ | grep -v "/$" | while read file ; do
if [ -L "$ROOT/$file" ]; then
rm -f "$ROOT/$file"
fi
done

to check for existing links
And then to really install the package and create the file-listing at the same time:

( cd $ROOT/ ; $TAR -xzlUpvf - ) < $package >> $TMP/$shortname 2> /dev/null
if [ "`cat $TMP/$shortname | grep '^./' | wc -l | tr -d ' '`" = "1" ]; then
# Good. We have a package that meets the Slackware spec.
cat $TMP/$shortname >> $ADM_DIR/packages/$shortname
else
# Some dumb bunny built a package with something other than makepkg. Bad!
# Oh well. Bound to happen. Par for the course. Fix it and move on...
echo './' >> $ADM_DIR/packages/$shortname
cat $TMP/$shortname | grep -v '^./$' | cut -b3- >> $ADM_DIR/packages/$shortname
fi

Note that it's expecting a tar listing that begins with this entry: './'
It also implies the other archive entries are listed as relative paths

# list the old-style archive with old tar (the one made using '.', not '*')
tar-1.13 -tf test2.tar
./
testdir2/
testdir2/test2.txt
test2.txt

This format is critical because of what removepkg does:

if fgrep "./" $ADM_DIR/packages/$PKGNAME 1> /dev/null 2>&1; then
TRIGGER="^\.\/"
else
TRIGGER="FILE LIST:"
fi
if [ ! "$WARN" = true ]; then
echo "Removing files:"
fi
sed -n "/$TRIGGER/,/^$/p" < $ADM_DIR/packages/$PKGNAME | \
fgrep -v "FILE LIST:" | sort -u > $TMP/delete_list$$

The other aspect of starting to use a new version is this syntax:
( cd $ROOT/ ; $TAR -xzlUpvf - ) < $package

The '-l' option does something completely different with tar-1.14 than with tar-1.13.
The '-l' option disappears with tar-1.15
And, the above syntax implies the default behaviour which is 'critical' to not have
links to dirs overwritten by real dirs contained in the package. This must be explicitly
given now with --no-overwrite-dir
The -U option and -p need to be looked at as well, as their behaviour with different versions
of tar, uing the same syntax, gives variable results
This looks like it should be right:
( cd $ROOT/ ; $TAR -xzUpv --no-overwrite-dir --one-filesystem -f - ) < $package
but in fact, the 'U' (unlink first) option behaviour changes across tar-1.13/14/15/16.
With tar-1.13 the U option would only unlink *files* ??? With later versions it seems
to do the opposite of what you'd expect. Using tar-1.19 this works as desired:
( cd $ROOT/ ; $TAR -xzpv --one-filesystem -f - ) < $package
using the -U option with later versions causes the link to be overwritten by a real
dir contained in the archive

To test this out, create a new dir with a dummy directory inside:
mkdir -p test/dir
Then create links from dir to the names of the dirs inside test1.tar and test2.tar:
ln -sf test/dir test/testdir1
ln -sf test/dir test/testdir2
Then cd into 'test' and test unpacking test1.tar and test2.tar:
(cd test ; tar-1.13 -xlUpv -f ../test1.tar) #or tar-1.13 -xUpv --one-file-system -f ../test1.tar
That should leave the link existing. Now check if the new version works:
remove the archive contents
rm -f test/test1.txt
rm -rf dir/test1.txt
tar-1.19 -xUpv --one-file-system --no-overwrite-dir -f ../test1.tar
# I'm using tar-1.19, but I think this behaviour is consistent since 1.16
You'll see that overwrites the link test/testdir1 with the real directory testdir1
which test1.tar contains. Using:
tar-1.19 -xpv --one-file-system -f ../test1.tar
does give the right behaviour.
 
1 members found this post helpful.
Old 02-10-2011, 03:53 AM   #23
ruario
Senior Member
 
Registered: Jan 2011
Location: Oslo, Norway
Distribution: Slackware
Posts: 1,866

Rep: Reputation: 885Reputation: 885Reputation: 885Reputation: 885Reputation: 885Reputation: 885Reputation: 885
Quote:
Originally Posted by gnashley View Post
I did get the syntax worked out to use a later version of tar and correctly handle the links and perms(metadata), but the first problem with the difference in the way archives are listed appears to have no reliable resolution.
Can't you just use a transform?
Code:
tar --transform "s,^\./\(.\),\1," --show-transformed-names -cvf- . | xz -c > ../example-1.0-i486-1.txz
./
usr/
usr/man/
usr/man/man1/
usr/man/man1/example.1.gz
usr/bin/
usr/bin/example
usr/doc/
usr/doc/example-1.0/
usr/doc/example-1.0/COPYING
usr/doc/example-1.0/ChangeLog
usr/doc/example-1.0/README
install/
install/slack-desc
(Note: '--show-transformed-names' is only needed for display purposes)

P.S. Whilst we are at it, it would be nice to set '--owner 0' and '--group 0' as well so that it is possible to run makepkg as a non root user.

Last edited by ruario; 02-16-2011 at 03:35 AM. Reason: added and an example binary ;)
 
Old 02-10-2011, 04:12 AM   #24
ruario
Senior Member
 
Registered: Jan 2011
Location: Oslo, Norway
Distribution: Slackware
Posts: 1,866

Rep: Reputation: 885Reputation: 885Reputation: 885Reputation: 885Reputation: 885Reputation: 885Reputation: 885
I have also noticed that spkg will fix a package with './usr' style formatting on install such that the log makes it look like it was created 'correctly'.
 
Old 02-10-2011, 06:13 AM   #25
ruario
Senior Member
 
Registered: Jan 2011
Location: Oslo, Norway
Distribution: Slackware
Posts: 1,866

Rep: Reputation: 885Reputation: 885Reputation: 885Reputation: 885Reputation: 885Reputation: 885Reputation: 885
@gnashley: I had a quick scan through your notes. So how about this as an installpkg.patch for installpkg?
Code:
--- installpkg
+++ installpkg
@@ -93,22 +93,7 @@
 
 # So that we know what to expect...
 umask 022
-TAR=tar-1.13
-$TAR --help 1> /dev/null 2> /dev/null
-if [ ! $? = 0 ]; then
-  TAR=tar
-fi
-if [ ! "$(LC_MESSAGES=C $TAR --version)" = "tar (GNU tar) 1.13
-
-Copyright (C) 1988, 92,93,94,95,96,97,98, 1999 Free Software Foundation, Inc.
-This is free software; see the source for copying conditions.  There is NO
-warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
-
-Written by John Gilmore and Jay Fenlason." ]; then
-  echo "WARNING: pkgtools are unstable with tar > 1.13."
-  echo "         You should provide a \"tar-1.13\" in your \$PATH."
-  sleep 5
-fi
+TAR=tar
 
 usage() {
  cat << EOF
@@ -519,7 +504,7 @@
     cat $DESCRIPTION | grep "^$shortname:" >> $ADM_DIR/packages/$shortname 2> /dev/null
   fi
   echo "FILE LIST:" >> $ADM_DIR/packages/$shortname
-  ( cd $ROOT/ ; $packagecompression -dc | $TAR -xlUpvf - ) < $package >> $TMP/$shortname 2> /dev/null
+  ( cd $ROOT/ ; $packagecompression -dc | $TAR -xpvf- ) < $package >> $TMP/$shortname 2> /dev/null
   if [ "$(cat $TMP/$shortname | grep '^\./' | wc -l | tr -d ' ')" = "1" ]; then
     # Good.  We have a package that meets the Slackware spec.
     cat $TMP/$shortname >> $ADM_DIR/packages/$shortname
Note: I did not use '--one-file-system' because this is extraction and 'man tar' (tar-1.23) states that this option is to "stay in local file system when creating archive" (emphasis mine).

And this as a makepkg.patch for makepkg?
Code:
--- makepkg
+++ makepkg
@@ -29,23 +29,8 @@
 
 CWD=$(pwd)
 
-TAR=tar-1.13
+TAR=tar
 umask 022
-$TAR --help 1> /dev/null 2> /dev/null
-if [ ! $? = 0 ]; then
-  TAR=tar
-fi
-if [ ! "$(LC_MESSAGES=C $TAR --version)" = "tar (GNU tar) 1.13
-
-Copyright (C) 1988, 92,93,94,95,96,97,98, 1999 Free Software Foundation, Inc.
-This is free software; see the source for copying conditions.  There is NO
-warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
-
-Written by John Gilmore and Jay Fenlason." ]; then
-  echo "WARNING: pkgtools are unstable with tar > 1.13."
-  echo "         You should provide a \"tar-1.13\" in your \$PATH."
-  sleep 5
-fi
 
 make_install_script() {
   COUNT=1
@@ -284,28 +269,28 @@
 rm -f ${TARGET_NAME}/${TAR_NAME}.${EXTENSION}
 case $EXTENSION in
 'tgz' | 'tar.gz' )
-  $TAR cvf - . | gzip -9c > ${TARGET_NAME}/${TAR_NAME}.${EXTENSION}
+  $TAR --show-transformed-names --transform 's|^\./\(.\)|\1|' -cvf- . | gzip -9c > ${TARGET_NAME}/${TAR_NAME}.${EXTENSION}
   ERRCODE=$?
   if [ ! $? = 0 ]; then
     echo "ERROR:  gzip returned error code $? -- makepkg failed."
   fi
   ;;
 'tbz' | 'tar.bz2' )
-  $TAR cvf - . | bzip2 -9c > ${TARGET_NAME}/${TAR_NAME}.${EXTENSION}
+  $TAR --show-transformed-names --transform 's|^\./\(.\)|\1|' -cvf- . | bzip2 -9c > ${TARGET_NAME}/${TAR_NAME}.${EXTENSION}
   ERRCODE=$?
   if [ ! $ERRCODE = 0 ]; then
     echo "ERROR:  bzip2 returned error code $ERRCODE -- makepkg failed."
   fi
   ;;
 'tlz' | 'tar.lzma' )
-  $TAR cvf - . | lzma -c > ${TARGET_NAME}/${TAR_NAME}.${EXTENSION}
+  $TAR --show-transformed-names --transform 's|^\./\(.\)|\1|' -cvf- . | lzma -c > ${TARGET_NAME}/${TAR_NAME}.${EXTENSION}
   ERRCODE=$?
   if [ ! $ERRCODE = 0 ]; then
     echo "ERROR:  lzma returned error code $ERRCODE -- makepkg failed."
   fi
   ;;
 'txz' | 'tar.xz' )
-  $TAR cvf - . | xz -c > ${TARGET_NAME}/${TAR_NAME}.${EXTENSION}
+  $TAR --show-transformed-names --transform 's|^\./\(.\)|\1|' -cvf- . | xz -c > ${TARGET_NAME}/${TAR_NAME}.${EXTENSION}
   ERRCODE=$?
   if [ ! $ERRCODE = 0 ]; then
     echo "ERROR:  xz returned error code $ERRCODE -- makepkg failed."
Note: I did not set '--owner 0' or '--group 0' as perhaps a user would have some reason for creating a package with different owner and group.

Last edited by ruario; 02-10-2011 at 08:12 AM. Reason: corrected the transform
 
Old 02-10-2011, 10:10 AM   #26
guanx
Senior Member
 
Registered: Dec 2008
Posts: 1,014

Rep: Reputation: 146Reputation: 146
Quote:
Originally Posted by tank728 View Post
I just downloaded the tar.SlackBuild from current and found this.



Even with the new compression schemes being added to pkgtools, they still rely on tar to package all the file together. I agree with gnashley that tar-1.13 will be used up until it will no longer compile. I am not sure how long slackware has been using this outdated version of tar (even if it is just for packages), but a solution will have to made. Even if it is not any time soon.

I am surprised that when gnu decided to change the way tar unpacked files (related to the above symlink issue) that a compatibility switch was not implemented.
If it's only the symlink problem, I think "cpio -i" will work as well.
 
Old 02-10-2011, 11:19 AM   #27
volkerdi
Slackware Maintainer
 
Registered: Dec 2002
Location: Minnesota
Distribution: Slackware! :-)
Posts: 875

Rep: Reputation: 1816Reputation: 1816Reputation: 1816Reputation: 1816Reputation: 1816Reputation: 1816Reputation: 1816Reputation: 1816Reputation: 1816Reputation: 1816Reputation: 1816
Quote:
Originally Posted by ChrisAbela View Post
I made a quick Google search for Security issues for tar-1.13 and I only found the following:

Thus if we stick to tested packages, like those from Slackbuilds, or those we make ourselves, there should not be any risks. Still, the risk exists and users should be warned.

Chris
Why would you exploit a ../.. traversal in tar (which is documented in the tar documentation, btw), when you could just put anything you like in a shell script?

tar-1.13 is fine for the package utilities. In fact one of the "solutions" to that tar issue that was proposed was making tar refuse to extract a file through a symlink, which would have broken it for our purposes. It's better not to have to chance it every time tar gets changed upstream.
 
2 members found this post helpful.
Old 02-10-2011, 11:31 AM   #28
XGizzmo
Member
 
Registered: Mar 2007
Distribution: Slackware
Posts: 264

Rep: Reputation: 68
If it's not broke fix it till it is.
 
Old 02-10-2011, 03:23 PM   #29
ChrisAbela
Member
 
Registered: Mar 2008
Location: Malta
Distribution: Slackware 14.1
Posts: 467

Rep: Reputation: 89
Quote:
Why would you exploit a ../.. traversal in tar (which is documented in the tar documentation, btw), when you could just put anything you like in a shell script?
Since it is documented, than the users have been warned.
 
Old 02-10-2011, 03:31 PM   #30
gnashley
Amigo developer
 
Registered: Dec 2003
Location: Germany
Distribution: Slackware
Posts: 4,768

Rep: Reputation: 477Reputation: 477Reputation: 477Reputation: 477Reputation: 477
guanx, cpio is not a good option as it arbitrarily creates new directories chmod 755 if they don't already exist.

Creating passable packages using modern tar is possible alright -it's the extraction of them that is the problem -even if the file paths can be fixed by using transform. The real problem is that links shouldn't be overwritten by a real directory included in the archive(non-posix behaviour 'fixed' in later versions of tar), and that existing directory metadata shouldn't be overwritten. I think the latter may be possible with a combination of later tar options. It would mean *not* using 'U' (unlink first).

Glad to see PatV chime in here! Have I really gotten it right about the three issues and the reason for sticking with tar-1.13? Are there any other desirable behaviours of tar-1.13 that I've overlooked? Mind you, I'm more than happy to keep using since we know what it does.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
BackUp & Restore with TAR (.tar / .tar.gz / .tar.bz2 / tar.Z) asgarcymed Linux - General 5 12-31-2006 02:53 AM
Slack 10.2 pkgtools in 10.0 linuxhippy Slackware 3 01-17-2006 09:54 AM
pkgtools Nightmare HELP!!!!!!!!!! waterox Slackware 4 09-12-2005 12:46 PM
pkgtools problem??? brbrow Slackware 2 09-10-2003 01:58 PM


All times are GMT -5. The time now is 04:22 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration