A great method for keeping Slackware up-to-date...without slackpkg.

Lufbery · 01-29-2010, 08:32 AM

Quote:

Originally Posted by gegechris99

Regarding security update of the kernel, I make a new initrd.gz image file and run "lilo" after the upgrade (i.e. I don't use the lilo option in slackpkg because I have to rebuild the initrd.gz file first).

Should your script include an automatic run of "lilo" (useless most of the time) or will the script grow more complicated (i.e. what about initrd.gz)?

I also use an initrd, so I'm not inclined to automatically rerun lilo at each update. In any event, kernel updates are rare in -stable.

Regards,

H_TeXMeX_H · 01-29-2010, 09:12 AM

Quote:

Originally Posted by Lufbery

Hm. That's an interesting idea. However, that would make the MD5sum checking more difficult.

Regards,

I see, but maybe you could only check the md5sum of files that have a size greater than 0, and also install only these files. If this can be implemented it would do exactly what I need it to. I'll look into it, maybe I can come up with a solution.

T3slider · 01-29-2010, 10:05 AM

Quote:

Originally Posted by H_TeXMeX_H

I see, but maybe you could only check the md5sum of files that have a size greater than 0, and also install only these files. If this can be implemented it would do exactly what I need it to. I'll look into it, maybe I can come up with a solution.

Arg, page expired and I just lost my last post...with my script. Anyway, here's attempt number 2. I am not at a Linux box and don't have access to one until I get home, but I would suggest using the `find` command with the -size parameter.

Code:

( cd patches/
  for filename in `find packages/ ! -size 0 -name "*.t?z"
  do
    if grep "$filename" CHECKSUMS.md5 | md5sum -c >/dev/null
    then
      upgradepkg "$filename" || exit 1
      echo "" > "$filename"
    else
      echo "MD5sum of $filename does not match -- aborting"
      exit 1
    fi
  done
)

Note that that code is very (very) rough and may not work in its current form -- my memory is not sufficient to write bash scripts without access to man pages and a Linux box. I am also not 100% positive that the output of the find command will match exactly what is in the CHECKSUMS.md5 file (it may have to be run through sed to get rid of a prefix or something). I don't know all the arguments to lftp so I'm not sure if your existing script touches existing files, but if it syncs everything then you would have to disable overwriting of existing files (so already-installed packages will remain at 0 bytes instead of being repopulated). You may also want to modify or remove entirely the || exit 1 after upgradepkg to account for any packages that are already installed but are not set to 0 bytes (for example, if you run this script for the first time) since I believe upgradepkg will send an error if you try to upgrade an existing package with the same package.

Good luck.

H_TeXMeX_H · 01-29-2010, 11:42 AM

Alright, here's my modification of the script:

* see below for newer version *

take what you will from it.

Also, I added the '-r' to lftp so that it won't get the kernel stuff (which I don't need), you can add it back in to download these too.

GrapefruiTgirl · 01-29-2010, 11:47 AM

Quote:

Originally Posted by Lufbery

...
. And besides, slackpkg is 476 lines lnog. My script (so far) is only 23.

That's good reason to play around with this, and here's another one (if you're interested):

Slackpkg is not POSIX compliant (it requires Bash to work) so when creating your own script, it's easier to avoid something like this from the start, than it is for (whoever, maybe me?) someone to fix Slackpkg so it will work in a non-Bash shell.

At this stage in the game, and given your admission to being pretty new at scripting, I doubt you care about POSIX compliance (and don't blame you

) and getting your script WORKING, and learning stuff, are more important.

I however, have been pondering repairing Slackpkg for this reason

Onward we go..

bgeddy · 01-29-2010, 12:47 PM

Quote:

Slackpkg is not POSIX compliant (it requires Bash to work) so when creating your own script, it's easier to avoid something like this from the start, than it is for (whoever, maybe me?) someone to fix Slackpkg so it will work in a non-Bash shell.

The shebang for slackpkg is "#!/bin/sh" which will run bash in posix mode. I understand there are still some differences in the intricacies of bash's posix mode and its "none POSIX compliance" but I'm not that much of a shell expert to know what all the differences between the Bourne shell and the Bash shell's posix compliance mode are! Suffice to say the slackpkg script does run bash in "posix mode".

GrapefruiTgirl · 01-29-2010, 12:56 PM

@ bgeddy,

yes, you're right, it does of course run in Bash's POSIX mode, which as you sort of correctly implied, is not the same as POSIX in a fully POSIX shell.
I don't know all the differences and subtleties either

But, AFAI-do-know, POSIX Bash (which is calling /bin/sh linked to /bin/bash, or /bin/bash --posix) still allows some bashisms and/or non-POSIX's to occur.

Simple little things too, like echo -e "some string\n" is not POSIX because the -e or -n switches to echo are not POSIX.

If you try running Slackpkg in a POSIX shell, it just fails with a syntax error:

Code:

POSIX shell:

root@reactor: slackpkg
/etc/slackpkg/slackpkg.conf: 92: Syntax error: "(" unexpected

BASH shell:
root@reactor: bash slackpkg
slackpkg - version 2.80.2

Usage: 	..blah blah...

Now, I do note that the error is coming from slackpkg.conf (apparently) and I haven't really looked into this-all yet.. But I plan to at some point

Anyhow, we may be drifting off of the OP's course here.

Cheers!
Sasha

Lufbery · 01-29-2010, 01:21 PM

Quote:

Originally Posted by GrapefruiTgirl

Anyhow, we may be drifting off of the OP's course here.

Cheers!
Sasha

Yeah, you are, Sasha, but that's okay -- as long as you provide some hints on how to make my script, or H_TeXMeX_H's modification of it POSIX compliant.

H_TeXMeX_H, I like your version of the script. I'll need to parse the script to see exactly what's going on, but it looks like it greps the CHECKSUMS.md5 and outputs the filtered results to a new file, which then is again filtered to match just the packages downloaded for the new updates.

I like that it does not (should not) be run as root.

I'm not sure I want to do the exact same thing because I want to maintain a full local mirror. I'm not sure how I feel about running my script as root, but I think it's an okay risk. Still . . .

Is is possible to add something to your script that would clear out the newly downloaded files and replace them with 0-byte placeholders after upgradepkg had been run?

Regards,

H_TeXMeX_H · 01-29-2010, 01:37 PM

Ok, here's the very latest version, I added support for a blacklist and fixed some bugs, so look at this one over the last one:

Code:

#!/bin/sh
# download packages to update slackware
# this script is NOT meant to be run as root

# Increase readability
tmpdir="/tmp/usu"
blacklist="$HOME/.usublacklist"
ftpmirror="ftp://ftp.slackware.at/slackware64-13.0/patches/"

# clean up from last time
rm -rf "$tmpdir"
mkdir -p "$tmpdir/packages"
cd "$tmpdir/packages"
grep -h 'PACKAGE LOCATION:' /var/log/packages/* | rev | cut -d / -f 1 | rev | xargs touch
cd ..

# get checksums
lftp -c "open $ftpmirror; get CHECKSUMS.md5"

# exclude packages in the blacklist
if test -f "$blacklist"
then
	cd packages
	grep -f "$blacklist" ../CHECKSUMS.md5 | rev | cut -d / -f 1 | rev | xargs touch
	cd ..
fi

# Sync and get checksums (the -r means NOT recursive)
lftp -c "open $ftpmirror; mirror -r -e -n -I *.t?z --ignore-size packages"

# Check for MD5 checksum errors
find ./packages -size 0 -type f -delete
find ./packages -type f > downloaded
grep -f downloaded CHECKSUMS.md5 | grep './packages/.*t.*z$' > final.md5

if test -s final.md5
then
	if md5sum -c final.md5
	then
		# Tell the user to update the packages as root
		echo 'Run the following as root to upgrade slackware'
		echo
		echo 'upgradepkg --install-new $tmpdir/packages/*.t?z'
	else
		echo 'ERROR: md5sums do not match'
		exit 1
	fi
else
	echo
	echo 'Your system is up to date !'
	echo
fi

exit 0

BTW, mine is POSIX compliant and yours should be too.

Yeah, you can add to my script something like:

Code:

cd packages
cat ../downloaded | xargs rm -f
cat ../downloaded | xargs touch

There might be a better way to do it. You would also need to stop it from deleting the temp directory.

What mine does is take all the packages you have installed and make zero byte files out of them in that directory, so that when lftp downloads them, the ones that already exist will not be downloaded (as per the '--ignore-size' option).

EDIT:
Almost forgot, the blacklist should be something like:

Code:

mozilla-firefox-.*t.*z$
bind-.*t.*z$

That's because it's input to grep.

grissiom · 01-30-2010, 12:23 AM

Hmm, guys, I have to admit that writing a update script from scratch is a good practice

But I have one thing to say: Only get updates from patch/ is not enough. Updates in current won't be placed in patch/. If you use stable version, it can be Ok. But if you stick with current, you will lost your updates. IMHO, the only way to do this is slackpkg's approach: maintain a local database(if you call it) to track the package names and versions on "remote" branch and compare it with the ones you have installed. Than update the differ ones.

Besides, the algorithm in current slackpkg to get update list is _extremely_ slow. I have patches for that.(it's on LQ but I couldn't find it at this moment) The patch will be included in next release of slackpkg.

H_TeXMeX_H · 01-30-2010, 04:29 AM

Hey Lufbery, I got a question. Can I take the modified script that I wrote, release it under GPL and put it on my site, I'll give you credit for coming up with the idea, because I would not have though of updating slackware in this exact way, it seems like a great yet simple idea.

Lufbery · 01-30-2010, 09:28 AM

Quote:

Originally Posted by H_TeXMeX_H

Hey Lufbery, I got a question. Can I take the modified script that I wrote, release it under GPL and put it on my site, I'll give you credit for coming up with the idea, because I would not have though of updating slackware in this exact way, it seems like a great yet simple idea.

Absolutely!

Warm regards,

Lufbery · 01-30-2010, 09:37 AM

Quote:

Originally Posted by grissiom

Hmm, guys, I have to admit that writing a update script from scratch is a good practice

But I have one thing to say: Only get updates from patch/ is not enough. Updates in current won't be placed in patch/. If you use stable version, it can be Ok. But if you stick with current, you will lost your updates.

Yes. This is a special purpose script to keep -stable up-to-date. The -current branch has its own special challenges, so slackpkg is exactly the right tool for that job.

Quote:

IMHO, the only way to do this is slackpkg's approach: maintain a local database(if you call it) to track the package names and versions on "remote" branch and compare it with the ones you have installed. Than update the different ones.

Besides, the algorithm in current slackpkg to get update list is _extremely_ slow. I have patches for that.(it's on LQ but I couldn't find it at this moment) The patch will be included in next release of slackpkg.

These are all good points. I'm sure there's a point where, in the development of an update script, one could add enough tasks, user input, and conditions that one would end up with slackpkg!

Regards,

H_TeXMeX_H · 01-30-2010, 09:43 AM

Quote:

Originally Posted by Lufbery

Absolutely!

Warm regards,

Alright, thanks, I'll do that soon.

grissiom · 01-30-2010, 10:05 AM

@Lufbery

Once I've know your purpose on this project, my aim being at here is not to start wars.(Do we? No, I think

) I just want to see some technical details. Your script is a good start and may end of something useful.