My apporach on how to keep Slackware-current upgraded
 

Earlier this month a fellow LQ newbie asked us How to maintain a sane slackware64-current with Eric Hameleers multilib and ktown? Well, I gave my two cents there.

Now I'm here to expand on how I keep up with Slacware-current evolution in a safe manner, i.e. by avoiding system breakage between upgrades. Keep in mind that even if all this works perfectly for me and I haven't had not a single glitch in this process in the past couple years, it is solely my approach to the task. I have no intention at all that this may become an ultimate approach to it. It's just my way of doing things that could eventually help others with ideas.

The scripts are here:

https://gist.github.com/denydias/a10...08b8c1bec63ac8

And this is how they work:

• 1_mirror.sh: this is my mirroring script. It runs everyday on my NAS to keep up with Slackware-current and AlienBOB's Ktown. It also performs a very important task to anyone avoiding bad things to happen: package preservation.
• 2_minskyup.sh: this one upgrades Slackware-current according to the best practices exposed in Upgrading Slackware to a New Release. There are different functions for system and kernel upgrades. The later includes automation to preserve the last working kernel version so you can have a working system if the new kernel don't boot. It also includes the recovery and memtest images from Slackware's usbboot.img, meaning that if something goes really, really wrong, you'll have the tools to fix it right there. There are also functions to upgrade many of the tools of my daily workflow, that for one reason or another I decided to use the upstream binaries instead of a ready made package or SlackBuild out there: virtualbox, vagrant, docker, aws-cli, cli53, (git)hub, jq, ngrok, phpmyadmin, spectre-meltdown-checker, testssl, asciinema, calibre, pup, Synology Drive and zoom.
• 3_minskyup_completion.sh: this one goes in /etc/bash_completion.d/ to offer tab completion for minskyup.
• 4_lilo.conf: an example of my boot entries so the kernel upgrades works as expected.

But how it works? Whenever a see a new Slackware64 -current ChangeLog entry, or anytime I want, I run:

This will upgrade any packages but kernel. When there is a ktown upgrade, I go to init 3 and run it without --force as it's not advisable to upgrade KDE stuff with X running.

If the changelog entry also contains a new kernel version, then I run:

This command will ask me to confirm all the changes it detects, so I have the information I need to decide where or not is safe to move on.

As for the other minskyup options, they are pretty self explanatory. For instance, to upgrade virtualbox:

It'll detect the currently installed version and the latest one from Oracle. If the installed one is outdated, it moves on and upgrades.

But there are some other tools that do not provide a way to discover which the latest version is. In that case you have to check first, but it's quite simple too:

And so on...

So, this is how I do it. WFM. Damn, really WFM! :D

PS: why minskyup? My box is named after Marvin Minsky. :cool:

Thank you for your highly interesting post!

I am taking a far less ambitious approach to keeping my Slackware systems up-to-date, but now that I finally got around to finishing my script for maintaining the Linux kernel under Slackware, I thought I’d share the procedure that I follow. Now, as you said:
I actually restarted writing my script from scratch some four or five times, since I found it got too complex while I worked on it, and I then began to read various sources (man pages, the slackpkg source code, etc.), which taught me about quite a few useful features that could make my script so much simpler and more straightforward.

I maintain a local copy of the Slackware repository, which I keep up-to-date with the rsync utility.
I blacklisted the kernel packages (as well as ‘SBo’, ‘alien’, and ‘compat32’) in my ‘/etc/slackpkg/blacklist’ file, and to install the Slackware updates, I run the typical command sequence:
Then, to maintain the Linux kernel versions on my Slackware system, I simply run my ‘upgrade-kernel.sh’ script without any parameters:
The script presents a dialog-based interface that, depending on the circumstances, will let me install the latest kernel version and/or remove selected kernel versions (except for the currently running one).

In case anyone is interested, I attach the script (with an extension of “.txt” instead of “.sh”) to this post.

I'm glad you liked it. Thank you.

That's what I call useful comments! Well done. ;)

Dear denydias,

thanks for sharing your workflow. I have one question concerning ktown. Is it really necessary to:

before:

Doesn't the latter also upgrade ktown as well?

Greetings
Marcus

@luvr thanks for providing your script. Is it necessary in order to make it work to use a local mirror or does it also work with a remote? If the latter is the case, could you please share your opinion on why it's better to use a local mirror?

Greetings
Marcus

Glad you figured it out.

In case anyone ever runs into the same issue: After you rsync your local repository, a new kernel (or, by extension, any new or updated package) won’t get picked up until you run
Since I usually run the common sequence of
right after the rsync, and only then run my kernel update script, I won’t normally run into this issue.

That was exactly the point, thanks. Have you seen my additional question:

For me it's usually:

A couple more steps (including "geninitrd" and running the NVidia driver installer with "-K -k") if there's a kernel upgrade.

@dugan what is the compat32pkg about that you are using? Are you converting the regular packages in multilib with that? Would it also be possible to use aliens multilib repository instead?

http://compat32pkg.sourceforge.net/

I've found that Alien's multilib repo is often out of date.

As it stands, the script requires a local mirror.

The immediate reason is that it looks for an active “file:” line in your ‘/etc/slackpkg/mirrors’ file, and it will quit if it doesn’t find one.

The more fundamental reason is that the ‘installpkg’ command (which the script runs to install the new kernel packages) only installs packages that are available locally. Therefore, to support a remote mirror, the script will have to download the packages to a local directory before installing them. While I did consider implementing this (using something like ‘wget’ or ‘curl’), I decided in the end that it wasn’t worth the hassle. After all, the script was really just meant for my own use—“just a hobby, won't be big and professional like gnu”… ;)

To be honest, I really didn’t give this question all that much thought. When I looked into Slackware package management for the first time (long before I even envisaged ever installing Slackware permanently and for real), the Slackware documentation kind of suggested using a local mirror, and that’s really the primary reason why I’m still using a local mirror.

@luvr thanks for the reply. I can imagine that a local mirror is also a bit more 'safe' especially for upgrades from stable to -current. I am using it now as you have implemented and till now I am very happy with it. Thanks for the work.

@luvr I have noticed one little thing concerning your script: I had the local mirror configured without the trailing slash first, like: file://home/ftp/pub/Linux/Slackware/slackware64-current and your script exited when it tried to detect the local mirror. After modifying the slackpkg mirror configuration so that it looked like this: file://home/ftp/pub/Linux/Slackware/slackware64-current/ everything worked as expected.

That’s correct—initially, I made the trailing slash optional, but then I saw the following note in the ‘/etc/slackpkg/mirrors’ file (emphasis mine):
So I reasoned that, since the trailing slash was required anyway, I could just as well enforce it.

Having said that, making the trailing slash optional requires just one simple change to the script (replacing a single ‘+’-sign with an asterisk), as implemented by the following patch:
To apply the patch, save it to a file (say, ‘/tmp/plus2asterisk.patch’), then ‘cd’ to the directory where you keep the ‘upgrade-kernel.sh’ script file, and run:

@luvr thanks for pointing that out and for the patch. I think I will let it like this now as it works perfectly. I have to admit that I did not read the note in the slackpkg mirrors config. I have just tried the script without having the trailing slash set before and then I have realized that in your script there was one, so I have added it an it worked.

OOPS! My script does not pick up today’s upgraded kernel.
 

Today’s “Current (pre-release) ChangeLog for x86_64” announces availability of an upgraded kernel in the repositories:
My ‘upgrade-kernel.sh’ script, however, doesn’t pick up this new kernel, because kernel version “4.19.66” is already installed on my computer. Consequently, the script believes that there is no new kernel version available for installation. The script considers only the version number, and ignores the build number, which was incremented from “1” to “2”, while the version was left unchanged.

That makes me wonder what would be the recommended way to install this new kernel version. If you used ‘installpkg’, then, surely, it would get installed, but I guess you would then end up with a “broken /var/log/packages - with two versions of the same package”, wouldn’t you? Even so, I’m not aware of any real problems that this would cause; the only irregularity, as far as I can tell, would be a seemingly innocent warning from ‘slackpkg’ about the issue.

Anyway, to get my ‘upgrade-kernel.sh’ script, as it currently stands, to install this upgraded kernel, you can do the following:

• Reboot the system with an older kernel that is still installed. (If you don’t have any, then you’re out of luck… Sorry!)
• Run the script and instruct it to remove the latest kernel—i.e., in this case, version 4.19.66.
• Rerun the script to install the new kernel version.

I’m not entirely sure if it would be worth it to try and adapt my script so it would handle this type of condition. Is this a common occurrence, or is it pretty much exceptional? In any case, this is the first time that I’m seeing it.

@luvr thanks for pointing that out. It would be great if you could find a way that your script could handle those edge cases. Could you please elaborate how you handle the parallel installation? As far as I have seen on my system /var/log/packages shows several versions of the same package (which I have also expected to be like that). As I have blacklisted the kernels in slackpkg there is also no warning about it.

It happened before, although it isn't all that usual for the kernel to get a recompile, especially in -current, where upgrades happen that often that it mostly will be a version upgrade anyway.
A quick search in the ChangeLog's found a case in 14.1, where the kernel (3.10.17) even was rebuilt twice, within a few days:
There may be more, but I only did a limited quick search.

BTW: the current (last) updated kernel for 14.1 is a -2 build too:
linux-3.10.107/kernel-generic-3.10.107-i486-2.txz
because of a security vulnerability.

It seems to me that it is not an option to leave two (or even more) builds of the same kernel version installed side-by-side, since they use the exact same names for the files that they install.

For instance:
So, if I were to install Build “-2” without removing Build “-1” first, then the files from Build “-1” would be overwritten. In effect, there would, then, be two packages claiming ownership of the files.

In other words, I guess that if there’s a rebuild available of an already installed kernel version, then it shouldn’t be installed over the previous build, but the previous one should be removed first.

Any thoughts?

Of course build -2 must be an UPGRADE to build-1, because, as you said, they will need the same (but possible recompiled) modules, include files etc.

If you use slackpkg then you get a dialog screen and the output from removepkg. The tools check the newer rebuilt package Skipping over duplicate file names. The net effect is that the older package is removed from the package list but the newer rebuilt files remain in place.

Or you can directly use removepkg, not get the dialog screen and just get the Skipping messages. The same end result is the older package is removed from the package list and the newer rebuilt package & files remain.

True, but are you suggesting that I use ‘upgradepkg’ to install Build-2? Wouldn’t that remove all earlier kernel versions? Or could I use the “oldpackagename%newpackagename” construct to tell ‘upgradepkg’ that Build-2 (i.e., the new package) is an upgrade to Build-1 (i.e., the old package), and would it, then, leave the older versions alone?

(I’m really just thinking aloud here. If no-one has the answer to these questions, I will try out the “oldpackagename%newpackagename” construct and see what gives.)

So, if I understand you correctly, your suggestion would be to ‘installpkg’ the new build first, and subsequently ‘removepkg’ the old one? In any case, that sounds to me like it should work.

Correct, that works.

I've been using scripts that follow that exact workflow for system upgrades (including the kernel). The scripts leverage slackpkg and manage the upgrade process from beginning to end including managing EFI & LILO files.

That's actually what "upgradepkg" does, it removes the existing package (you can even specify explicitly WHICH one instead of letting it search for it).
(from /sbin/upgradepkg).

It will never remove multiple old packages, only the one that is superceeded BY the newly installed one.

@luvr one thing that would be nice is to automatically create symlinks to the previous kernel (the one which is actively running) like: /boot/vmlinuz-generic-prev

Hey! I'm sorry for the late reply. For an unknown reason I'm not receiving LQ's reply notification messages.

Yes, it does. But I want to make sure to upgrade ktown before other stuff.

My script slipped on that too. A fix is in my personal backlog. I'm planning to use the approach to replace only the current kernel (build N) with the newer one (build N+1) and leave the previous one alone. I don't know how long I'll take to come up with that fix as I'm having busy days here.

That was my initial idea, too, but I've come to the conclusion that that won't work, since the new build will overwrite the previous one anyway.

From how I understood it, the only way to handle those edge cases is to replace the old version using the upgradepkg old2%new2 feature, right?

The sequence bellow may work just fine for the case:

This does not look like it will mess with the previous one, being the previous kernel a version older then $KCUR, which in this particular use case happens to have only the build number increased. I don't see it touching modules and generic for the previous kernel.

On the other hand, the only way to preserve headers and source for the previous kernel is to copying their files before the installpkg/upgradepkg of them, but I don't really see the need for this. In the event of the new kernel don't boot, I'll use the previous one just to inspect the situation, fix it and boot into the new one asap.

I would like to know how often it really happens that a new kernel does not boot. I must admit that i am using the huge kernel and never had any issues with it, neither on stable nor on -current. I can imagine that an upgrade could eventually cause issues when one is using LVM / Full-Disk-Encryption or things like that.

For me the the main issues when using the -generic kernel is that I always have to remember to run lilo afterwards, which I sometimes forget and then (and only then) I have to boot the old kernel and fix the initrd.

So I am really interested in your real-life experience and would be happy if you could share some of the situations you had on kernel upgrades.

LVM / Full-Disk-Encryption is my use case. I can't afford have no way around with a faulty new kernel. It never happened before, but who knows when...

That's the reason why I decided to automate it and to keep the previous working kernel. Before that I had a TFTP server at home so I could boot to the rescue image and fix things, which also involves to mess with BIOS network and boot settings as I always keep my notebook pretty closed to booting with anything I can't control. There was one time I was away from home. You could imagine the rest. My current approach gives me all the tools I need to recover from a faulty upgrade, anywhere.

autoslackpkg-0.4 RC1
 

I've been using my own Slackware upgrade scripts since last year. At that time I had 3 systems on current and the kernel upgrades were becoming robotic for me. Hopefully the scripts are ready for others to test drive. I named the scripts "autoslackpkg".

https://www.go4it2day.com/news/autoslackpkg-0.4.html

The scripts leverage slackpkg and manage kernel changes from start to finish on both EFI or LILO systems. The key to successfully using the scripts is to make sure 4 config files are appropriately customized. There are rudimentary help screens that try to explain the config customizations.
I use the scripts on 11 systems currently, 14.2 & current, x86 & x86_64, desktops, servers, gateways & LXC containers. I've tried to hit all my edge cases but I'm sure there are others I haven't run into yet. You're welcome to use them or tear them apart for ideas for your own scripts.

Enjoy!

That is absolutely correct, but when I mentioned the "previous" kernel, I actually meant the earlier build of the $KCUR version. That will, obviously, get overwritten. Any earlier kernel versions that are still installed will, clearly, not be touched. Should the new build, for any reason, fail to boot up, then you can still fall back on such an older version - PROVIDED that you kept at least one of those.

Sounds like a great suggestion, and would make such a fallback a whole lot easier, should it ever be required. I'll look into it. Sure seems pretty much doable anyway.

I had some spare time today and could face the kernel build bump problem in my script. Follow are the results (with mocked values, of course).

First with a regular kernel upgrade, e.g. 4.19.66 to 4.19.67:

Now with just a kernel build bump, e.g. 4.19.66-1 to 4.19.66-2:

Now let the kernels come so I can found the bugs in my logic... :cool:

Hmmm... I have just tried the following (while “kernel-source-4.19.66-noarch-1” was the highest kernel-source version installed):
It doesn’t do what I intended, though, in that it removes all earlier versions of the kernel-source package, then installs the specified new package. Come to think of it, this doesn’t really come as a surprise; after all, it just performs the upgrade that I request—which, by definition, will remove all earlier versions of the package to be upgraded.

Thus, a sequence of ‘removepkg’ and ‘installpkg’ will be the way to go in order to upgrade an installed kernel version to a newer build.

Only if you want to keep the previous working kernel and its source so you can recompile it for any reason. But kernel-headers won't work that way, as it always replace the previous files.

My approach is a little different: although I keep the previous kernel, I do so only to count on a booting system in case the worst happens, i.e. the just installed new kernel won't boot at all. As I have no intention to recompile the previous kernel, nor compile anything in the system against it anymore, I don't need kernel-source nor kernel-headers of the previous kernel.

As such, when a new kernel arrives (like 4.19.67 to replace 4.19.66), my script just upgradepkg kernel-source and kernel-headers, effectively replacing them with the newer versions. On the other hand, when the kernel upgrade is just a bump (like 4.19.66-1 to 4.19.66-2), there is no need to touch kernel-source and kernel-headers, so my script just skip this step.

In any case, version or bump upgrades, my script actively removepkg the selected kernel then installpkg the new one. I prefer this way to avoid conflicts that may left orphan files or dead links around in the system.

Good catch… looking at it like that, I don’t actually need the older sources or headers either.
For now, it doesn’t hurt to keep them around either, but I’ll think it through a little further before I make up my mind about how to handle them in the future.

Indeed, you'll wipe out the earlier versions… which are, in effect, just dead weight when you leave them installed.

That idea hadn't even crossed my mind just yet, but you're absolutely right!

I think I’ll experiment a little further to see what works best for me… but your comments certainly make sense. In any case, I remove the links that come with a kernel installation, and I create and maintain my own set of links—so far, without issues.

I do have an updated version of my script ready now, which maintains an additional set of links to the ‘previous’ kernel files (i.e., to the kernel version that is running when the script is being executed), but I’ll have to double-check that it doesn’t do any crazy things when it has to remove kernel packages.

UPDATE to my 'upgrade-kernel.sh' script.
 

Attached to this post is an update to my ‘upgrade-kernel.sh’ script.

NEW FEATURE:

• The script now maintains a set of symlinks to the “previous” kernel version—i.e., the one that is active at the time a new kernel is being installed. These symlinks will have the same names as their “new-kernel” counterparts, but with a suffix of ‘-prev’ appended.

Caveat: When you remove one or more kernel versions while no new kernel version is available, these ‘-prev’ symlinks will be recreated to point to the kernel version that is running at that time. As a consequence, if you’re running the latest installed kernel version at that time (which you normally will), then the ‘-prev’ and the “new-kernel” symlinks will both be made to point to that latest version. I don’t consider this a significant problem, because:

1. the primary goal of the script is to install kernel upgrades—at which time it will let you remove any versions that you no longer want to keep;
2. the currently active kernel is known to be good anyway, since otherwise you wouldn’t be running it in the first place.

TO DO:

• Add support for installing a new build of an already installed kernel version.

I will be testing this support on my Slackware-current install, which I temporarily reverted back to kernel version 4.19.66, Build 1. I’m using a test version of the script that I point to what it thinks is my local repository mirror, but is in reality a simple folder into which I can copy any kernel version and build that I want to test with.

Kernel 4.19.68 upgrade handled as usual. Now let's wait for a bump (e.g. 4.19.68-1 to 4.19.68-2) to test this case. It may take years though.

Next update to my 'upgrade-kernel.sh' script.
 

Attached to this post is the next update to my ‘upgrade-kernel.sh’ script.

NEW FEATURE:

• The script now supports installation of a new build of an already installed kernel version.
  To install a new build of the latest installed kernel version, it will first install the new build, and next remove the previously installed one. In this way, the removal operation is really only required in order to clean up installation registration of the overridden (and overwritten) build.

KNOWN ISSUE:

• It remains a bad idea to remove the latest installed kernel version, since the version-independent symbolic links to the kernel files will then be left behind, but they will become “dangling” pointers—i.e., they will point to ‘nowhere’.
  Note that the script won’t even let you remove the latest kernel version anyway, unless you are running an earlier kernel version and there is no new kernel available for installation.

EXTRAS:

• Script ‘fix-kernel-symlinks.sh’ will recreate the version-independent symbolic links to the latest installed kernel files.
  It will also allow you to select the kernel version to which you want to point the fallback (i.e., “previous”) symbolic links.
• Script ‘rebuild-kernel-initrd.sh’ allows you to select the installed kernel versions for which you want to rebuild the Initial RAM Disks (“initrd”).

Thank luvr, I am going to try it out soon and give some feedback. The dangling symlinks is not so much an issue in my point of view.

Could you elaborate a bit more why exactly now three scripts are needed instead of one like before? Wouldn't it be possible to merge all the functionality into the main script?

The main script, ‘upgrade-kernel.sh’, is the only one that’s needed. The other two, ‘fix-kernel-symlinks.sh’ and ‘rebuild-kernel-initrd.sh’ are just quick hacks that should not be required, but that can be used to repair anomalies with the symlinks or the initrds, should that ever be necessary. The ‘fix-kernel-symlinks.sh’ script for instance, will fix the “dangling pointer” issue for you, should you ever run into it.

I used the ‘fix-kernel-symlinks.sh’ while I was developing the latest version of my ‘upgrade-kernel.sh’ script, so I could verify if the latter behaved properly under various conditions. I haven’t encountered a situation in which the ‘rebuild-kernel-initrd.sh’ was useful, but you never know.

Thus, to recapitulate: Under normal conditions, you should still only really need the main ‘upgrade-kernel.sh’ script.

Thanks for pointing that out. I have to add some options to the mkintrd generation. Can I just modify the following line to look e.g. like this:

$(/usr/share/mkinitrd/mkinitrd_command_generator.sh -a -l sg-latin1 -h /dev/cryptvg/swap '-o '"${INITRD}" --run "${KERNEL}")

Greetings

Lioh

Yes, that will work fine.

@luvr. I am actually not sure what caused this situation but currently the following broken symlink has been created:

initrd-*-prev -> initrd-*-4.19.80*

I have used the script for the first time on a newly installed system and this link has been generated on the first kernel update. The 'old' initrd was named just initrd.gz, maybe that's the root cause.

What a coincidence... I have just run into the same issue myself. It happens because one of the patterns to find existing initrds does not find any files and thus returns just the pattern unchanged.

I now have an updated version of the script available, which I will upload later. (I don't have my Slackware system handy at the moment, but the solution is as simple as inserting an appropriate "shopt" command near the top of the script.)

Thanks for reporting this issue. It reminds me to upload the new version.

Thanks for the reply. I will wait for the updated version. As far as I understood this can only happen on the first run/migration to your script.

Yes, that's correct.

UPDATE to my 'upgrade-kernel.sh' script: BUG FIX.
 

Attached to this post is the next update to my ‘upgrade-kernel.sh’ script. It fixes a bug where, the first time that it is run on a freshly installed Slackware system, it will create a broken symlink, such as:
(where the version string, ‘4.19.81’, will be replaced by the actual kernel version, of course.)

To fix this bug, the script now sets the ‘nullglob’ shell option when it begins execution:
(Previously, this option got set in one of the functions defined in the script, but the function was called too late to prevent the broken symlink.) As the bash man page explains, the ‘nullglob’ shell option allows patterns that do not match any files to expand to a null string, rather than themselves.

As before, only the main script, ‘upgrade-kernel.sh’, is required.

Helper scripts ‘fix-kernel-symlinks.sh’ and ‘rebuild-kernel-initrd.sh’ let you recreate the symbolic links to the installed kernel files and rebuild the initrd, respectively, just in case.