Current's pkgtools directories move

burdi01 · 06-20-2018, 07:35 AM

As a heavy pkgtools and slackpkg user in "normal" and $ROOT situations I tested Current's new pkgtools package and found it be working as expected.
However, I noticed one caveat: a symlink to a directory is not the same as a directory.

Some programs expect their /var/log/XXX logging directory to exist. Cups is/was one of these -- that is why the cups package creates a /var/log/cups directory.
To keep the sizes down my backup script removes as much "noise" as possible:

Code:

rm -f $1/var/log/*       # messages and the like  
....
rm -f $1/var/log/cups/*
....
rm -f $1/var/log/removed_{packages,scripts}/*

As long as /var/log/{packages,removed-{packages,scripts}} are directories this works as expected, but as soon as they are symlinks to directories they are removed by the "messages and the like" rm and the "target directories" do not get cleared ...

I personally regard the directories move discussion to be over the top, but I do understand the "purists" point of view.

GazL · 06-20-2018, 10:03 AM

using "rm -f *" and banking on it failing to delete directories feels a little sloppy anyway. There are better ways to do that.

I've just reviewed the impact of this change on my 'slacklist' script (what I use in place of slackpkg) and the only change that I found necessary was adding '-H' to a couple of find commands in order to propperly handle the symlinks.

I'm looking forward to this change.

update:
migration to new pkgtools in testing/ went without issue.

zerouno · 06-20-2018, 10:17 AM

I think that the best solutions to exclude something from a backup is not to delete it ma to exclude it.
The logs are useful so I do not see reason to delete /var/log/messages

For many time I asked me why the packages database was in a log directory instead in /var/lib as it should be.
A link is needed to keep compatibility from some official (slackpkg) and some non-official tools, and for thousands of personal scripts to manage the distro.

Regard other software that uses /var/log/packages as a directory, there was slackpkg and slackpkg+. Both are ready to accept link for that directory. I don't know for slapt-get.

Didier Spaier · 06-20-2018, 12:52 PM

Quote:

Originally Posted by zerouno

Regard other software that uses /var/log/packages as a directory, there was slackpkg and slackpkg+. Both are ready to accept link for that directory. I don't know for slapt-get.

I just checked. /var/log/packages is only mentioned in package.h like this:

Code:

#define SLAPT_PKG_LOG_DIR "/var/log/packages"

In turn, SLAPT_PKG_LOG_DIR is used in package.c:

Code:

didier[~/Slint/repo/x86_64/slint-14.2.1/source/slapt-get/slapt-get-0.10.2t/src]$ grep -r SLAPT_PKG_LOG_DIR
package.h:#define SLAPT_PKG_LOG_DIR "/var/log/packages"
package.c:    strlen(SLAPT_PKG_LOG_DIR)+
package.c:    strncat(pkg_log_dirname, SLAPT_PKG_LOG_DIR,strlen(SLAPT_PKG_LOG_DIR));
package.c:    strncat(pkg_log_dirname, SLAPT_PKG_LOG_DIR,strlen(SLAPT_PKG_LOG_DIR));

So, that will be only a line to change for Slackware-15.0 and derivatives.

About the transition, I don't know the C language, maybe someone can tell if the functions strlen and strncat accept a symlink as argument?

If yes, there should be no issue.

Anyway I will email Jason Woodward and the derivatives maintainers that I know who ship and use slapt-get (most do).

kjhambrick · 06-20-2018, 01:47 PM

Another one of my dumb Qs

I would like to test the new pkgtools Package on my Slackware64 14.2+current box.

I don't use any package management tools other than installpkg, upgradepkg and removepkg ...

Is it as simple as: cd /my/local/mirror/ && upgradepkg testing/packages/pkgtools-15.0-noarch-20.txz ?

Thanks All !

-- kjh

What is installed on my Slackware64 14.2+current box:

Code:

# ver pkgtools

-rw-r--r-- 1 root root 4907 Jun 15 02:57 /var/log/packages/pkgtools-15.0-noarch-19

From the ChangeLog.txt ...

Code:

testing/packages/pkgtools-15.0-noarch-20.txz:  Rebuilt.
  This update is a bit scarier than usual, so we're going to test it here
  first and then move it into the main tree in a couple of days if there
  are no serious bug reports. It's well-tested here, and works with the
  slackpkg that's in -current now, but I don't know about slackpkg+ so that's
  another reason to let it cool down here first. The purpose of this update
  is to migrate the package database and directories from /var/log to
  /var/lib/pkgtools. /var/log was never a good place for this data, as it is
  considered by many to be a directory that could be wiped to free up some
  space. Originally the package database was in /var/adm, but the FSSTND
  (later FHS) group decided that directory should be a symlink to /var/log,
  and I went along with that since it was years ago and I was a n00b and didn't
  know any better. /var/lib/pkgtools will be a better and safer location.
  The removed_packages and removed_scripts directories are really just logs
  that aren't actually used for anything - those will remain under /var/log,
  but moved into /var/log/pkgtools. Everything under /var/log will be
  considered potentially non-permanent by the pkgtools - if any directories or
  symlinks disappear from there, the pkgtools will automatically recreate them
  as needed. In fact, the migration process will create symlinks from all the
  old directory locations to the new ones, so anything that expects the old
  locations (including slackpkg, for now) should continue to work. Once this
  moves into the main tree, the plan is to fix other packages to use the new
  installer script directory (/var/lib/pkgtools/setup) and change the installer
  and slackpkg to use the new native locations for everything. When slackpkg
  is changed over to use the new native locations, I'll also make sure to float
  that in testing/ for a few days before moving it to the main tree to avoid
  more unintentional disruption to slackpkg+ users.
  Be aware that the package database migration is a one-way operation, but even
  so if you later downgrade to an older version of the pkgtools it will still
  work through the compatibility symlinks.

the3dfxdude · 06-20-2018, 02:04 PM

Yes, symlinks store something that says, "look over here instead". symlinks can be resolved to the actual path they point to. See "man 3 realpath" or "man 2 readlink" for C. I think some File I/O APIs does this automatically. BTW, strcat are just string operations. I think you are looking for routines that do 'stat' or 'open'.

If some third party program does not work because it opened the symlink directly, and you are paranoid, you can switch the symlink to a hard link. Then just politely ask the author to just change the real path now.

Didier Spaier · 06-20-2018, 02:12 PM

Quote:

Originally Posted by the3dfxdude

Yes, symlinks store something that says, "look over here instead". symlinks can be resolved to the actual path they point to. See "man 3 realpath" or "man 2 readlink" for C. I think some File I/O APIs does this automatically. BTW, strcat are just string operations. I think you are looking for routines that do 'stat' or 'open'.

If some third party program does not work because it opened the symlink directly, and you are paranoid, you can switch the symlink to a hard link. Then just politely ask the author to just change the real path now.

Thanks for your answer. As an aside, hard links are not allowed for directories.

Darth Vader · 06-20-2018, 03:39 PM

Quote:

Originally Posted by Didier Spaier

I just checked. /var/log/packages is only mentioned in package.h like this:

Code:

#define SLAPT_PKG_LOG_DIR "/var/log/packages"

In turn, SLAPT_PKG_LOG_DIR is used in package.c:

Code:

didier[~/Slint/repo/x86_64/slint-14.2.1/source/slapt-get/slapt-get-0.10.2t/src]$ grep -r SLAPT_PKG_LOG_DIR
package.h:#define SLAPT_PKG_LOG_DIR "/var/log/packages"
package.c:    strlen(SLAPT_PKG_LOG_DIR)+
package.c:    strncat(pkg_log_dirname, SLAPT_PKG_LOG_DIR,strlen(SLAPT_PKG_LOG_DIR));
package.c:    strncat(pkg_log_dirname, SLAPT_PKG_LOG_DIR,strlen(SLAPT_PKG_LOG_DIR));

So, that will be only a line to change for Slackware-15.0 and derivatives.

About the transition, I don't know the C language, maybe someone can tell if the functions strlen and strncat accept a symlink as argument?

If yes, there should be no issue.

Anyway I will email Jason Woodward and the derivatives maintainers that I know who ship and use slapt-get (most do).

These two functions, strlen and strncat are utilities for strings manipulation, then they will not care if the given string defines a path pointing to an UNIX symlink or contains entirely the original Middle English version of The Tragedy of Hamlet, Prince of Denmark.

BTW, I believe that the slapt-get author has only to point the SLAPT_PKG_LOG_DIR to the new location, or even better, to rename it also like: SLAPT_PKG_LIB_DIR or similar.

the3dfxdude · 06-20-2018, 05:08 PM

Quote:

Originally Posted by Didier Spaier

Thanks for your answer. As an aside, hard links are not allowed for directories.

Yes you are right. I should have been more careful in what I was saying in this case. I was indicating that because it shows why in some cases typical file operations don't seem to work when it is a symlink, but a hard link should act like the original file almost in every way. In this case, probably should just check and see if it can handle it, and just change the path to the new directory for the next version of slapt.

Didier Spaier · 06-20-2018, 05:19 PM

Quote:

Originally Posted by the3dfxdude

In this case, probably should just check and see if it can handle it, and just change the path to the new directory for the next version of slapt.

Yes. I will make tests in a VM and see what I come up with, installing Slint-14.2.1 then upgrading pkgtools from Slackware-current. Now I know what to do tomorrow

zerouno · 06-21-2018, 02:01 AM

Note that also sbopkg uses the /var/log/packages
but I think that only scripts using 'ls' may have some problem since - for 'ls' - the command
ls /var/log/packages
is not the same of
ls /var/log/packages/

for all the other tools the kernel should resolve the link trasparently.
slackpkg has recently replaced all 'ls /var/log/packages' in 'ls /var/log/packages/'

burdi01 · 06-21-2018, 03:58 AM

Quote:

using "rm -f *" and banking on it failing to delete directories feels a little sloppy anyway. There are better ways to do that.

The rm manpage reads:

Code:

RM(1)                            User Commands                           RM(1)
NAME
       rm - remove files or directories
SYNOPSIS
       rm [OPTION]... [FILE]...
DESCRIPTION
       This  manual  page  documents  the  GNU version of rm.  rm removes each
       specified file.  By default, it does not remove directories.
....

So what is sloppy?

burdi01 · 06-21-2018, 04:04 AM

Quote:

... for 'ls' - the command
ls /var/log/packages
is not the same of
ls /var/log/packages/

What is the difference?

GazL · 06-21-2018, 04:11 AM

Code:

$ ls -l
total 0
drwxr-xr-x 2 test users 40 Jun 21 10:05 dir
-rw-r--r-- 1 test users  0 Jun 21 10:05 file
$ rm *
rm: cannot remove 'dir': Is a directory
$ echo $?
1
$

You're invoking a command in a way such that you're expecting it to write to stderr and exit with a FAILURE status code. I try to avoid that wherever possible.

zerouno · 06-21-2018, 04:41 AM

Quote:

Originally Posted by burdi01

What is the difference?

try that:

Code:

mkdir first
$ touch first/1
$ touch first/2
$ ln -s first second
$ ls -l
total 4
drwxr-xr-x 1 mrossini users 4 Jun 21 11:21 first/
lrwxrwxrwx 1 mrossini users 5 Jun 21 11:21 second -> first/

now see that difference:

Code:

$ ls -l first/
total 0
-rw-r--r-- 1 mrossini users 0 Jun 21 11:21 1
-rw-r--r-- 1 mrossini users 0 Jun 21 11:21 2
$ ls -l first
total 0
-rw-r--r-- 1 mrossini users 0 Jun 21 11:21 1
-rw-r--r-- 1 mrossini users 0 Jun 21 11:21 2
$ ls -l second/
total 0
-rw-r--r-- 1 mrossini users 0 Jun 21 11:21 1
-rw-r--r-- 1 mrossini users 0 Jun 21 11:21 2
$ ls -l second
lrwxrwxrwx 1 mrossini users 5 Jun 21 11:21 second -> first/

when the argument ('first') is a directory you cannot see the difference (but there is); in both cases 'first' and 'first/' you will see the content of the directory.
when the argument is a link ('second') to a directory, ls will show the properties of the link ('second -> first/')
if you add a '/' to the link ('second/'), the kernel convert 'second' in 'first', then ls will show the result of the conversion.

Try that:

Code:

$ ln -s three four
$ ls -l three
/bin/ls: cannot access 'three': No such file or directory
$ ls -l four
lrwxrwxrwx 1 mrossini users 5 Jun 21 11:30 four -> three
$ ls -l four/
/bin/ls: cannot access 'four/': No such file or directory

If I want the list of packages and I ask for 'ls /var/log/packages', in the legacy pkgtools style 'ls' returns the list of packages. In the new style it returns the string 'packages' that is the name of link.
'ls -l /var/log/packages' will return the path of destination link

Code:

$ ls -l /var/log/packages
lrwxrwxrwx 1 root root 24 Jun 20 16:57 /var/log/packages -> ../lib/pkgtools/packages/

'ls -l /var/log/packages/' resolve the link and show the content of the directory (the list of installed packages).