Are you sure about that? Unless I'm misunderstanding you that's not my experience. Any missing leading directories will be created with permission based on the umask currently in effect (assuming you specify '-d' to have it create them at all rather than just failing).

Also, if there are any system directories that you absolutely never want changed (such as /usr and /usr/bin etc.) you can exclude them from the archive at creating time.
Here's an example
As you can see, the shared system directories and the "." directory are not included in the cpio archive at all, but the executable and private directory under /var are in there ready to be extracted. In theory one could set it up such that the only package that contains the system directories would be aaa_base and you have a really safe way of doing packages.

Unless, I'm overlooking something, which is quite possible as it's a big topic with far too many facets to think about in one brain.. - please tell me if I am.

I think you are referring to this old bug. However, if you look at the Changelog for GNU cpio you will see this was fixed long ago (version 2.9).

The meta data seems fine and symlinks are not overwritten in modern tar as long as you leave out '-U', at least in my own quick tests. Though I'll grant you this is a complex issue so I may have missed something.

I have no problem with sticking with tar-1.13 for Slackware. I have come around to the idea that others pointed out and Patrick confirmed he agrees with, i.e. "It works so why change?". Slackware generally just works because of this no nonsense approach of not changing just for change's sake.

However, I am still interested in this issue because I want to be able to use pkgtools on other distros for these reasons:

1) Repacking and/or quick testing of Slackware packages when not running Slackware
2) I can use pkgtools as a supplementary packager on non-Slackware distros

That second point probably deserves some clarification. For stuff I compile from source I want an easy way to keep track of packages, however making an rpm or deb (or even a Arch PKGBUILD) adds a lot more overhead than using Slackware's makepkg. Also not every source package provides a good (or sometimes any) 'make uninstall'. Yes I know about checkinstall but it has several draw backs. Off the top of my head, a limited selection of package formats (read, no Arch support) and it installs stuff to / first and then makes the package. I want to look at the contents first and perhaps adjust things before installation. I won't comment on your src2pkg as I only have limited familiarity with it. I hear great things but I don't think I need that level of automation personally. Though maybe I am wrong! I would also still have the issue that there will always be some package format you don't support and I use a wide range of distros in my job.

Furthermore using two package managers does not cause issues (like file ownership clashes) as long as you leave the native package manager to look after the main '/usr' structure and leave your secondary manager (pkgtools in my case) to look after your locally created packages in '/usr/local'.

Now the 'correct' way to port pkgtools for use on another distro would be to make as few changes as possible and hence include a copy of tar-1.13. However this means extra build dependencies and a development environment installed on the distro you are porting to. One of the the nice things about pkgtools is that they are just shell scripts using standard utilities. Hence if I can simulate the correct behaviour I would like to do some minor patches to pkgtools so that I can use the native version of tar included with the distros I work with.

I am fairly comfortable with the transform for package creation but I have a vague worry that I may not have considered or noticed everything with regards to extraction.

I should also add I think this discussion is also helpful in case there every comes a day when Patrick does want/need to move from tar-1.13. If we already know the best way to do things with modern tar, he can just lift the examples from this thread.

Another option for anyone else considering something along these lines would be to patch only makepkg to use a transform for package creation (since this seems pretty safe) and using spkg for install, upgrade and remove. You don't really need explodepkg since you can just unpack with 'tar xf' and pkgtool itself is just an extra front end. Of course it still requires that you compile spkg, so it may not seem like much of an improvement over compiling tar-1.13 (which has less dependencies anyway). However on the plus side spkg is very quick and should be well tested on the install/upgrade side, since SalixOS uses it. And no it isn't using tar-1.13 on the back end, the author wrote his own tar unpacking routine.

The "Spack" project has some interesting tools;

spkadd — install or upgrade the new experimental CPIO package format

spkrm — remove the package (of both TAR and CPIO formats)

spkcpio — generate a package in the new experimental CPIO format


Why reinvent the wheel when someone else has already reinvented it?

http://requiescant.tuxfamily.org/pag...105F.html#cpio

I have some experience with spkg -the biggest benefit is when using it to remove (or upgrade) packages.
As far as using a newer version of tar, you are on the right track. I did lots of tests, but it was back when tar latest was about 0.16 or 0.17. As you say, using the '-U' option appears to cause later tar to overwrite links with real dirs. I can't remember exactly what happened about directory meta-data -the oenwerships of a directory in the package shouldn't overwrite the ownership nor perms of an existing dir. The other option used by installpkg when unpacking packages is the '-l' -in tar-1.13 that means '--one-file-system', but in ltare versions it means something else completely.
One other thing to look at is when makepkg writes the tar file -it uses:
'tar name .'. For later versions you should probably use:
'tar name *'

Look closely at the expected listing in the /var/log/packages/* files. The contents should read:
./
relative/path/to
relative/path/to/contents
In later versions of tar, 'tar name .' will get you archives which list content like this (notice no './' topdir and leading ./ for others):
./relative/path/to
./relative/path/to/contents

The upshot is yes, you'd need to do a little tweaking in both the package creation in makepkg and when listing/unpacking them in installpkg.

However, the archives(packages) won't be cross-compatible -at least not with more effort with 'transform' or something. Old-style packages won't be easily handled by you new installer, and new-style packages won't be compatible with old pkgtools.

Let us know what you find out, or get working...

I've made packages with only (modern) tar and a simple transform in the past and they install and uninstall just fine with the original pkgtools.

Here is a quickly patched pkgtools for use on non-Slackware based distros for the purposes of quick editing and repacking Slackware packages, or for use managing self built Slack-style packages in /usr/local.

I made the following changes:

1. The scripts are installed into /usr/local/sbin.
2. The man files for scripts are installed into /usr/local/share/man.
3. I removed the checks for tar-1.13 and just use the native tar, without waiting 5 seconds.
4. I altered 'installpkg' to not use '-l' '-U' on extraction.
5. I renamed 'makepkg' and all references to it, to become 's-makepkg'.
6. I altered 's-makepkg' (makepkg) to use a transform on creation and display transformed names.
7. I changed absolute paths such as '/sbin/installpkg' in 'upgradepkg'.
8. I swapped the order of the '-type f' and '-maxdepth 1' find options in removepkg.

I did "2" because this seems like the appropriate location on non-Slackware distros for self-made packages. I did "5" because of a potential name clash on Arch and distros that use Arch pkgtools, which provide their own makepkg utility. I did "8" because on some distros (e.g. Arch), find prints a warning if you have them the other way around.

Here is archive containing the original pkgtools and a very simple script to repackage them along the lines listed above:

pkgtools-13.1-src-3.tar.gz

On a non-Slackware based distro (there is no point on Slackware or derivatives since you could just use the original utilities) you could install them as follows:

Everything would then be installed as:

I could then install and use pkgtools to track my source installs. For example if I wanted to install the Karmen window manager on a distro that does not include it (like Ubuntu). I could simply issue the following:

Note: Yes this is quick and dirty but my intention here is only to show that it is fairly easy to use pkgtools on non-Slackware based distros to track and install source packages installed into /usr/local, comparative to making a .deb (or .rpm for that matter).

And if I later decide to remove karmen:
or perhaps the pkgtools themselves:
Of course I still don't know for 100% that my changes to installpkg (see "4" above) are sufficient but I guess after playing with this for a few weeks I probably will.

P.S. I realise (but should perhaps point out for the benefit of anyone who doesn't) that it wouldn't be wise to try and make brand new packages on say Ubuntu and then try to use them on Slackware as I am likely to end up depending on some lib that either doesn't exist on Slackware or is of a different version. However I don't think it will be a problem to repackage or edit precompiled (binary) packages since I have already tested the transform trick when making Opera packages as I mentioned previously.

If I wanted to stray that far from the original utils I'd go with spkg and get the speed benefit.

You hit a bit rough... Spack uses its own package layout, provides a quite different UI, and doesn't require anything in Slackware to work; the only thing it shares with the Slackware's pkgtools is the generated package log file.

Are you sure? deal with the package DB doesn't consume so much time than that, the "strangling node" is the decompression of the package, where you can't go faster than gzip/xz go. I didn't test, but I would say that your gain are not significative (countable in seconds) unless you do a huge install at once (what you never do in fact, as the Slackware installer use the pkgtools) or you make an upgrade, as upgradepkg installs each new package twice...

As I mentioned, the real speec gain with spkg is when removing or upgrading packages.

Or here is an alternative hack to get the file names in tar-1.13 style:

However it is not really as good because for a big package you might hit a problem with the argument list being too long. On the other hand '--no-recursion' is an option that has been around for over 10 years, whilst '--transform' has only been around since GNU tar 1.20 from 2008-04-14. Also it is worth noting that even Slackware 12.2 didn't have tar-1.20+, it only had 1.16 (and 1.13 obviously).

Even when removing? I didn't think all the rm calls consumed so much time... well, why not after all, but removing is however something you rarely massively do.

Don't misinterpret what I say, though. I don't mean spkg is worthless, just that speed is IMHO not a good criteria to choose, as it's not something you can really enjoy in daily operations. The stability (C makes things not depend on command outputs) and the quality of the UI are more meaningful, I think.

Working on my pet C++ project called pkgbuild, I written an C++ API on top of libarchive, able to create and extract tarballs using the Slackware specifications. Yeah, pkgbuild is capable to create several tarballs, from a single directory tree, because it internal handle the tarballs.

So, I know very well the secrets of Slackware tarball-ing. So, I can say, the great secret of "Why TAR-1.13?" is only the directory symlink management and the store method of filenames.

If people really need, I can write an "pkgextract", able to automatically handle all compression methods suported by libarchive, and an "pkgcreate", able to create TAR/TGZ/TBZ/TLZ/TXZ tarballs using only the switches from command line.

NonNonBa, it's not the 'rm' calls themselves that take so long -it's the chceking to see if the file is claimed by any other package before removing it that takes so much 'grep file /var/log/packages/*' for every file a package contains makes a lot of work...

Thanks very much to ruario for spending so much time on all this and providing some alternatives. And, Darth Vader I'd love to see something built on libarchive which handles this stuff gracefully -but please make it in C, not C++ -we don't want an installer that needs libstdc++...

He he ... I think you give me too much credit. You seem to have done the bulk of testing and also worked out that '-U' was no longer needed. So thaks to you!

As a side note, to amuse myself and also to further prove "There is more than one way to skin a cat", I managed to think of yet another way to get the files stored "tar-1.13 style". Funnily enough it was wadsworth's mention of NonNonBa's spack and its use of cpio.

Before I give the example though just some quick thoughts for wadsworth and NonNonBa. I installed and looked at the spack tools very breifly. Firstly I don't really see the point of replacing the tar format with cpio and secondly I don't personally like the packages that the spack tools make. In my eyes the internal structure is overly complex, which means that making small edits to them after creation suddenly becomes a major hassle unless you entirely commit yourself to using the spack tools. I really like Slackware packages because I can just un tar them, tweak them, and then tar them up again. I don't need anything unusual preinstalled. In fact I don't even need to run Slackware as my host distro when I make the edits since (as I demonstrated earlier) you can just use any modern version of tar as long as you pick the right options (or even cpio as I will demonstrate below). To be clear however, that is not to say what NonNonBa has done isn't clever. It is just a statement that spack isn't for me 'personally'. If someone else finds it useful then great!

Anyway, back to what I was saying with regards to cpio. It occured to me that since cpio can optionally output in UStar format instead of one of the native cpio formats, you could actually use have it make the tar package.

In a way the following example kind of combines both the earlier tricks I used with (modern) GNU tar. I use the regex that I had previously used in the transform example and get the file/directory list via the find command, like the second example.

So, how about this?

As I said earlier, this is more for fun but you never know it might be useful, should PatV ever switch to using a transform and then the GNU tar maintainer does something odd like change the way transforms work (in some non compatible way). Whilst that seems exceedingly unlikely, it is always nice to have a backup method.