Local rsync

Yalla-One · 09-17-2006, 03:29 PM

Greetings,

I've got a full slackware mirror on my server, courtesy of Eric Hameleers' mirror-slackware-current.sh script. Let's for arguments sake say it's in /export/slackware/slackware-current

On my end-user machines, I don't want the complete mirror with source etc - just the packages I need to stay -current.

What is the best way to do this - rsync over NFS, or set up some rsyncd daemon? Also - do I need to build in lots of intelligence, or can I simply add ignore-statements to packages such as kdei and the kernel series etc that I don't want to touch?

Plan is to run mirror-slackware-current.sh on the server each night at let's say 0300, and then have the workstations update only the relevant files by cron a couple of hours later - around 0500...

Any insight?

-Y1

TSquaredF · 09-17-2006, 05:46 PM

I, too, have a -current mirror, using Eric's script. You are doing a bit more than I am, I only have a desktop & a loptop to update & I am doing them manually, so there are some significant differences in our requirements. For a while, I tried using another mirror on the HDD, with just those packages I wanted to upgrade, But, it got cumbersome & I thought it took up too much space. There are only 10 - 12 files in the -current tree that I don't want to upgrade, so I made a list of them, then wrote my upgrade script to read that list, rename those files so that they "no longer ended in 'tgz'", do the upgrade, then rename the files again. Not real elegant, but it works for me.
Regards,
Bill

Alien Bob · 09-18-2006, 04:35 AM

Hi Yalla-One

You may have told before why you want mirrors on every machine instead of allowing the clients to use one central mirror that you provide over an NFS mount. On any reasonable network, mounting the server mirror over NFS and upgrading packages that way would not be noticably slower than having the multiple local mirrors.

Having said that, you can easily cook up a rsync script that you run on the server after the "mirror-slackware-current.sh" script has run. That way you don't have to depend on the time that script needs to finish. If you run rsync scripts on the clients, you will need to add safe time margins to make sure the jobs won't collide with the master's update.

Something you need to provide in that case is "passwordless login" using SSH keys. The server can use a key to open a logon session to the client computer in an unattended script and use rsync to update just the relevant directories.
Something like

Code:

rsync -e "ssh -i ~/.ssh/slackrsync.id" -va --delete --exclude "pasture" --exclude "source" --exclude "extra/source" /export/slackware/slackware-current/ root@someclient:/clientmirror/

could work (don't use the exact code without testing first using the added rsync parameter "-n" - for "dry-run").
The "-i ~/.ssh/slackrsync.id" means to use a key or "identity file" that allows you to login as root (but you can pick any other account of course) without having to supply a password.

Eric

Yalla-One · 09-18-2006, 10:40 AM

Hi Eric,

Thanks again coming to the rescue once again!

The only reason for keeping the packages local is that I always do upgrades in single-user mode, which excludes NFS as viable solution. Furthermore doing upgradepkg --install-new /mnt/slackware/slackware-current/*/*.tgz gets a bit too much (kernel, kdei etc), so instead I figured I can just copy the exact packages each night over NFS to the local PC, and then when it's time for an update, I telinit 1, upgradepkg -install-new /root/slackware/*/*.tgz and that's it.

I have a sneaky feeling that I've misunderstood or forgotten something here, so any enlightenment would be _greatly_ appreciated :-)

-Y1

Alien Bob · 09-18-2006, 12:22 PM

Quote:

Originally Posted by Yalla-One

telinit 1, upgradepkg -install-new /root/slackware/*/*.tgz and that's it

If you carefully prune the "master mirror" so that only the packages are transfered to the clients that you'd really want, "upgradepkg -install-new /root/slackware/*/*.tgz" would get you where you want to be. But also check for any "*.new" configuration files that are added to your system - some of them might be required for correct functioning (like the many times rc.udev was re-written recently). Plus, sometimes (as is the case with the tcpip package afaik) some of your configuration files can be overwritten by those from the updated packages.

Blindly applying "upgradepkg -install-new /root/slackware/*/*.tgz" with slackware-current packages every day will someday break your machines, I'm willing to bet. Doing so with package updates for stable releases could be considered much safer.
For anyone using packages from slackware-current I would like to repeat the adagium: read the ChangeLog.txt before updating packages, and life will be much better.

Eric

Yalla-One · 09-19-2006, 01:33 PM

Thanks for answering,

I've been keeping with -current for a couple of years now, and as you say, as long as I read the Changelog there's no problems. However, I do not want to install-new kdei or kernel packages as they just fill the disk or polute /usr/src/linux* (not open for another discussion on kernel source locations :-) )

However, even when trying to follow your advise I somehow manage to get this wrong.

With the complete Slackware-current (including source and the whole nine yards) on let's say /mnt/maintenance/slackware/current through NFS, how do I use rsync to keep current in /root/slackware on the local PC, as the NFS volume is not available in single-user? Guess I've "read myself blind" on the subject, which normally happens when I fall-back to random manic experimentation :P

-Y1

PDock · 09-19-2006, 06:59 PM

Here's a really old thread I contributed to: http://www.linuxquestions.org/questi...d.php?t=273735

Of course the diff file is out of date; but it is/was possible to grab packages from the nfs mount/ store them locally/ init 1/ upgrade/ reboot

Your mileage may vary but with carefull inclusion to the /ourpatches directory everytime a machine on your lan rebooted it would be updated.

Yalla-One · 09-20-2006, 10:40 AM

Seems like the solution was easier than I thought - here's the solution - works like a charm:

Code:

#!/bin/sh
#slacksync by Yalla-One <yallaone@gmail.com>
#
SRCDIR=/mnt/maintenance/slackware/slackware-current/slackware
DSTDIR=/root/slackware-current
EXCLUDE=/root/slacksync-exclude.conf
SSPARM="-a"

# Check source directory availability
if [ ! -d ${SRCDIR} ]; then
  echo "Source directory ${SRCDIR} not available. Aborting..."
  exit 1
fi

# Check destination directory availability
if [ ! -d ${DSTDIR} ]; then
  echo "Destination directory ${DSTDIR} not available. Aborting..."
  exit 1
fi

# Check config-file availability
if [ ! -r ${EXCLUDE} ]; then
  echo "Config file  ${EXCLUDE} not available. Aborting..."
  exit 1
fi

case "$1" in
'-v')
  SSPARM="-av"
  ;;
'-vv')
  SSPARM="-avv"
  ;;
esac

# Perform actual sync
rsync ${SSPARM} --delete --delete-excluded --exclude-from ${EXCLUDE} ${SRCDIR} ${DSTDIR}

The config-file (rsync-exclude):

Code:

kdei
k
l/alsa-driver*
a/kernel-*
d/kernel-*

I do not want KDEI to bloat the system, and as I have my custom kernel tuned to each box' hardware, I do not want to contaminate the system with the default kernels, hence the exclusion of kernel-*

Works like a charm, especially when used together with Eric Hameleers' mirror-slackware-current.sh script

-Y1