Alternative update approach to slackpkg

GazL · 10-01-2021, 11:03 AM

I've been revising/polishing a couple of scripts I use to keep my slackware system updated and I think they're about ready for a wider audience now.

The README pretty much explains what and how they work, so I'll not repeat it here, but to pique folks' interest, they

are non-interactive.
do not use/store any metadata or database, outside of those pkgtools already use.
support multiple package sources, specified via a simple config file: which is just an ordered list of directories/URLS.
support local and remote package sources, which may be mixed.
will gpg verify if requested.
has a blacklist feature
has a per-package post-install-script feature.

So, if any of that sounds appealing then please point your browsers at my github repo and see if you want to give them a try.

I'm open to questions if anyone has any, and feedback is welcome.

Cheers.

marav · 10-02-2021, 04:56 AM

Hi GazL,

I just tried (just slackscan for now)

my feedbacks:

- eventually, to put also a blacklist for slackscan

It could be interesting :

- to put /var/lib/slackpkg by default in the conf
- and to automatically retrieve the uncommented mirror in /etc/slackpkg/mirrors
this way, even with an unmodified configuration file, the slackscan would work

- to display the package names (with a special mention) instead of "not available", even if they are external slackbuilds

GazL · 10-02-2021, 08:22 AM

Thanks for taking the time to have a look and to comment.

Quote:

Originally Posted by marav

- eventually, to put also a blacklist for slackscan

I intentionally kept slackscan light on features as per the UNIX philosophy:

Write programs that do one thing and do it well.
Write programs to work together.
Write programs to handle text streams, because that is a universal interface.

slackscan's "one thing" is to look what package files are available in its configured locations and then generate lists of packages to be added, updated or removed.

If you want to do a blacklisting then 'grep -v' will serve nicely.
For example, if I wanted to update/install everything except stuff in a blacklist and remove any removed files except stuff from SBo, then I could:

Code:

slackscan -i in.list -u up.list -r rm.list
grep -v -f blacklist.file in.list up.list | xargs -r upgradepkg --install-new
grep -v '_SBo$' rm.list | xargs -r removepkg

This is pretty much the same as an update, install-new, upgrade-all, clean-system run in slackpkg.

The above is pretty much what early versions of my 'slackup' script did before I expanded its feature set and made it output a command stream instead. Doing the above is likely a little unwieldy for the average user, which is why I wrote slackup with all those nice features like blacklists and filtering. all behind a more accessible command-line interface.

Quote:

Originally Posted by marav

It could be interesting :

- to put /var/lib/slackpkg by default in the conf
- and to automatically retrieve the uncommented mirror in /etc/slackpkg/mirrors
this way, even with an unmodified configuration file, the slackscan would work

Actually, slackscan might make a better back-end for slackpkg than the other way around.

Quote:

Originally Posted by marav

- to display the package names (with a special mention) instead of "not available", even if they are external slackbuilds

I'm not really sure what you mean here. The "not-available" stuff you see mentioned within the script itself is just used internally, the user will never see that. slackscan will output a list of packages whose package files have been removed to the file specified by the -r option.

For example, if I temporarily remove my hw-accel/ dir from my package locations to simulate some removals, it will look like this:

Code:

$ /usr/sbin/slackscan -r -
intel-gmmlib-19.2.3-x86_64-1_local
intel-media-driver-19.2.1-x86_64-1_local
libvdpau-va-gl-0.4.2-x86_64-2_local
$

marav · 10-02-2021, 08:47 AM

Quote:

Originally Posted by GazL

I'm not really sure what you mean here. The "not-available" stuff you see mentioned within the script itself is just used internally, the user will never see that. slackscan will output a list of packages whose package files have been removed to the file specified by the -r option.

For example, if I temporarily remove my hw-accel/ dir from my package locations to simulate some removals, it will look like this:

Code:

$ /usr/sbin/slackscan -r -
intel-gmmlib-19.2.3-x86_64-1_local
intel-media-driver-19.2.1-x86_64-1_local
libvdpau-va-gl-0.4.2-x86_64-2_local
$

Code:

blackstar ~ # slackscan -i -
not-available
not-available
not-available
not-available
https://slackware.nl/people/alien/sbrepos/current/x86_64/BeautifulSoup/BeautifulSoup-4.1.1-x86_64-1alien.tgz
https://slackware.nl/people/alien/sbrepos/current/x86_64/FAudio/FAudio-20.09-x86_64-1alien.tgz
http://local-mirror/slackware/slackware64-current/slackware64/xfce/Greybird-3.22.15-noarch-1.txz
....

After commenting

Code:

#    | join -a 1 -e 'not-installed' -o 1.1,2.2 - /dev/fd/$installed \
#    | join -a 1 -e 'not-available' -o 1.1,1.2,2.2,2.3 - /dev/fd/$available \

Code:

/var/lib/pkgtools/packages/cnijfilter2-5.90-1.x86_64
/var/lib/pkgtools/packages/duplicati-2.0.6.100-2.0.6.100_canary_20210811.noarch
/var/lib/pkgtools/packages/scangearmp2-3.90-1.x86_64
/var/lib/pkgtools/packages/swapinzram-1.1-noarch-Slackware
BeautifulSoup
Cython
FAudio
GConf
Greybird
...

GazL · 10-02-2021, 08:59 AM

You can't just comment out those joins like that, they're essential for the way slackscan works and it will corrupt slackscan's internal database table.

Are you saying that happened before you commented out those joins?
If so, that shouldn't happen. Please can you post your default.conf so I can try and debug. Also what $LANG are you running.

marav · 10-02-2021, 09:05 AM

Quote:

Originally Posted by GazL

Are you saying that happened before you commented out those joins?
If so, that shouldn't happen. Please can you post your default.conf so I can try and debug. Also what $LANG are you running.

Yep

This one :

Code:

blackstar ~ # slackscan -i -
not-available
not-available
not-available
not-available
...

Code:

blackstar :: ~ » echo $LANG
fr_FR.UTF-8

Code:

blackstar ~ # sed -e /^$/d -e /^#/d /etc/slackscan/default.conf
/var/lib/slackpkg
http://local-mirror/slackware/slackware64-current/CHECKSUMS.md5
blackstar ~ #

GazL · 10-02-2021, 09:07 AM

Please post your /etc/slackscan/default.conf for me.

edit: thanks I'll take a look.

GazL · 10-02-2021, 09:26 AM

If you remove /var/lib/slackpkg do you still get those lines?

Obviously I can't see your local mirror from here, but I tried with

Code:

/var/lib/slackpkg
http://slackware.uk/slackware/slackware64-current/CHECKSUMS.md5

and everything looks fine here, but my /var/lib/slackpkg is empty. What is in there on your system?

marav · 10-02-2021, 09:30 AM

Code:

blackstar ~ # ls /var/lib/slackpkg 
ChangeLogs     CHECKSUMS.md5.asc  SLACKPKGPLUS_alienbob-filelist.gz      current            pasture-filelist.gz  slackware64-filelist.gz
gpg            ChangeLog.txt      SLACKPKGPLUS_slackers-filelist.gz      extra-filelist.gz  patches-filelist.gz  testing-filelist.gz
CHECKSUMS.md5  PACKAGES.TXT       SLACKPKGPLUS_slackpkgplus-filelist.gz  install.log        pkglist

same as above without :
/var/lib/slackpkg in default.conf

Code:

blackstar ~ # sed -e /^$/d -e /^#/d /etc/slackscan/default.conf
http://local-mirror/slackware/slackware64-current/CHECKSUMS.md5
blackstar ~ # slackscan -i -                           
not-available
not-available
not-available
not-available
http://local-mirror/slackware/slackware64-current/slackware64/xfce/Greybird-3.22.15-noarch-1.txz
http://local-mirror/slackware/slackware64-current/slackware64/xap/MPlayer-20210418-x86_64-1.txz
...

GazL · 10-02-2021, 09:48 AM

One thing I can tighten up on is that I've not done anything to detect malformed package names. That could potentially be causing issues.

I'll add some code to make that more robust.

marav · 10-02-2021, 10:37 AM

Quote:

Originally Posted by GazL

One thing I can tighten up on is that I've not done anything to detect malformed package names. That could potentially be causing issues.

I'll add some code to make that more robust.

Let me know when you commit
I will test it again

GazL · 10-02-2021, 10:49 AM

Thanks marav. Just pushed to github.

marav · 10-02-2021, 11:18 AM

Quote:

Originally Posted by GazL

Thanks marav. Just pushed to github.

Code:

blackstar ~ # slackscan -i -
http://local-mirror/slackware/slackware64-current/slackware64/xfce/Greybird-3.22.15-noarch-1.txz
http://local-mirror/slackware/slackware64-current/slackware64/xap/MPlayer-20210418-x86_64-1.txz
http://local-mirror/slackware/slackware64-current/slackware64/xap/NetworkManager-openvpn-1.8.16-x86_64-1.txz
http://local-mirror/slackware/slackware64-current/pasture/apmd-3.2.2-x86_64-3.txz
...

GazL · 10-02-2021, 12:22 PM

Excellent. Thank you for helping to test.

I've tightened up the regex matching even further and tagged it as release 2.3.

marav · 10-02-2021, 12:42 PM

Quote:

Originally Posted by GazL

Excellent. Thank you for helping to test.

I've tightened up the regex matching even further and tagged it as release 2.3.

It's not a big deal,
but a small summary at the end of the list, for each, like :

XXX pkgs can be { installed / removed / upgraded }

could be nice