Own up! You use Vi anyway

Alien_Hominid, I'll explain about this:
"If fixed, the partially-broken feature will allow software to be installed using a 'virtual chroot'- that is, instead of installing files to your main filesystem they are diverted into a subdirectory so that nothing ever gets clobbered or overwritten. And this is done without having to use any special chroot installation or extra programs."

The file system translation feature in installwatch was written by a contributor and was first introduced in 2002 -in checkinstall-1.6.0beta1. There were two slight errors that were fixed by other contribs and these were included in checkinstall-1.6.0beta2 and 1.6.0beta3. Version 1.6.0beta4 included a patch which circumvented possible errors which no one had had or reported. From the time that the new fstrans feature was added people were reporting problems with it on other distros. Slackware included the checkinstall-1.6.0 version with Slackware version-11.0. It would still work mostly without problems. Meanwhile checkinstall had been upgraded to 1.6.1 (November 2006) which included a patch which added the tracking of two more system calls and a fix for the 'famous' getcwd bug which had been present since 1.6.0beta1. Then when Slackware-12.0 came out the same problems started to show as what had been occurring with other distro for some time -namely incompatibility with newer version of the coreutils.

More specifcally, the problem is with glibc-2.5 and kernel versions starting with 2.16. When coreutils are built against glibc-2.5 and then used with kernel-2.16 or later, some of the coreutils use new glibx functions called the 'at-style' functions or 'openat' functions. These use file descriptors with a relative path instead of an absolute path. And this is what breaks the file system translation with installwatch -hence the maintainers advice to use checkinstall with the fstrans=no flag. You can search for that term and find references and the same advice for nearly every distro starting from whenever thy started using glibc-2.5.

Now, in order to understand what the fstrans is supposed to do, let me explain the basic functionality of installwatch. installwatch works by 'putting up a fence' between the core glibc functions and the external programs which call them. An example: when the program cp is called to copy a file into your filesystem it first looks to see whether the destinatin file already exists. If it does, then it calls thew glibc function 'unlink' to remove the existing file. Then it calls the glibc function 'open' to create the new file and again calls 'open' to open the file which is to be copied. Then the contents are passed from the original file into a buffer and from there into the new file. What instllwatch does is intercept these calls to the glibc functions allowing it do something before the call is actually executed. For instannce, if you are using the backup function of installwatch, when it catches a call to open to copy over an already existing file, it first creates a backup copy of the file before passing the original call to the glibc function. This is the very mechanism which src2pkg uses to safely do a real installation of software on your system -it orders installwatch to keep the backups before installing new files. Then the new files are actually installed to your system. The src2pkg copies them into the package tree where the package content is being assembled. As soon as this is done, src2pkg then uses the backups created by installwatch to restore your system to its' original state. It does this as soon as possible, before proceeding to do anything more with the package content.

Now, if you've understood that, here's what the file system translation does. It catches these system calls and changes the paths before pasing the call along to glibc. For instance, a file which is supposed to be inatlled to /usr/bin, is instead installed to /tmp/my-package/usr/bin -or wherever you choose. I say that this is like a 'virtual chroot' because it works much the same way as if you were to chroot into a side directory and run 'make install' there. Of course, using the DESTDIR variable in Makefiles is also supposed to do this. Trouble is that DESTDIR is quite undependable. Many source Makefiles don't support it all and , worse, some support it partially -that is part of the program gets installed into your DESTDIR and part gets installed to the real filesystem root! This why src2pkg does not use DESTDIR. The installwatch backup feature coupled with a quick reshuffling of the original files is much more dependable. But still, there a few programs which can cause real headaches when installed like this -especially the programs that are being used by src2pkg itself -bash, glibc and the coreutils. Thius is why src2pkg includes statically-compiled versions of the few programs from coreutils which are used during restoration of your system -just after 'make install' has been run.
It should be clear that having all calls that create or modify files on your system would be more safely done in a separate directory which is is a ghost of your real '/' directory.

So, that's the feature which is partially broken in installwatch -it also is/was only partially implemented. I've gathered several patches which add the tracking of more glibc functions -like for file changes which occur when sed is called to edit a file in-place and others.
What I'm working on right now is to get installwatch to work with the new-style function calls which include some of the most basic, important and commonly used ones -open, link, unlink, chmod, etc. Once done, this will alow for much safer usage of src2pkg on running systems -even though the only report I've ever had of problems with src2pkg was from a user who had re-compiled coreutils with it. Even there, if he had used the --bindir=/bin option like the original Slackware package, he would have had no problems.

Thanks for very very informative explanation. So in the near future 'virtual chroot' will replace installwatch 'backup'. Am I right? What's wrong with using chroot and make install? Can I use src2pkg for rebuilding glibc or you still don't recommend this yet?

Cool! I'm going to try it out!

Blessed be!
Pappy

Using a chroot environment is not very handy as you need to have quite a few tools installed in the chroot environment in order to run the commands. Turns into quite a lot of extra space consumed and it's pretty complex for some users. src2pkg would have to check for the completeness and integrity of the chroot environment or set one first, or even worse, set it up on-the-fly and undo it for each package. It would still have to sort out the newly installed files from the files that were part of the chroot setup.

No, I do not recommend trying to rebuild glibc using src2pkg -unless you comment out the fake_install routine and do it using DESTDIR. I have not had the guts to try that one myself even -even though I have put the mechanism in place that *might* handle it. As mentioned, you can recompile coreutils and *sort of* get away alright. I have also recompiled bash on-the-fly, but that too didn't come off very clean, requiring a logout and then logging in again. Still, all these can be easily handled using DESTDIR since it can be counted on to work (usually) with these basic packages.

The virtual chroot, or translated filesystem will probably not completely replace the backup mechanism. Instead it will be an option. I had also recently decided to re-instate an optional way of using DESTDIR when it appears to be supported by the Makefile(s). It will require a good deal of testing both the backup and translation features to figure out which method works best and under what circimstances. This is somewhat dependent on what specific versions of tar and coreutils do, as well. this is especially related to what happens when files overwrite links or vice-versa and what happens when things are not all on the same filesystem.

Anyway, as always, you can count on me trying it out a *lot* on my setup before turning it loose in a release. I hate writing e-mails that begin with Ouch!

As a general rule, anytime you want to re-compile a package that is part of the basic system or is a package that comes with Slackware, you should make sure you understand how src2pkg works and consult the SlackBuild script to see what special needs the package may have. You can always 'port' a SlackBuild into the src2pkg format quite easily -usually just by pasting the middle parts of the script into the middle part of the src2pkg script plus adding or commenting out a line or two.
You might ask what advantage there is in that. There are several, but mainly because of the ease of updating the script for later versions and that src2pkg scripts are more 'portable' -in the sense that they don't use hard-coded paths. If you later decide you want to unpack & build sources some place other than /tmp, you just change your conf file or pass the option to src2pkg as an environmental variable and it's done.

Thanks. I asked this question because I was optimizing Slackware a lot (making more Gentoo like), but was doing it manually for some packages I use. So I was interested if I could use src2pkg in some script to update and optimize my system again, but this time automatically.

Yes you can, but as I said you need to be very careful with the first few basic packages which everything else relies. On some time ago I built a system this way by starting with a statically compiled toolchain and then rebuilt the whole system on top of itself. In my case I was changing the optimization. And if you are upgrading the kernel or glibc you really need to reboot at least once. But in theory it's pretty easy.
Have a look at this script for an example of how you can build groups of packages in a certain order.
http://distro.ibiblio.org/pub/linux/...pBuild.src2pkg

Since src2pkg can download and run any script with the src2pkg suffix, you could in fact start a complete rebuild over the net with a single command to download a script which downloads other scripts, each of which downloads everything it needs.

You might consult with LQ member 'tuxdev' who told me that he had also been using src2pkg 'a la gentoo' since the feature to download sources was added. In fact he first proddec me into fully implementing the feature by sending me a patch. I had in fact just implemented the feature partially a few days before that, but his patch and enthusiasm got me moving to improve the feature.

Ah, okay, that explains it. I'm sure you're correct that the latest released version of Emacs will eventually make it into Slackware. In the meantime, I'm glad I was able to use src2pkg to compile a Slackware package for Emacs 22.1.

I'm willing to give writing a SlackBuild a try, but it would take me a while, having not done one before.

Regards,

-Drew