LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Slackware (http://www.linuxquestions.org/questions/slackware-14/)
-   -   rsync slackware current (http://www.linuxquestions.org/questions/slackware-14/rsync-slackware-current-4175425879/)

elexwave 09-06-2012 02:06 AM

rsync slackware current
 
Hello LQ, :D
I just installed Slackware Current in my box. I have the problem to update the system cause limited bandwidth. :( So I extract the slackware-current ISO to create a local repo. my ISO was updated on 24 August. It's first time when I use rsync. whether the option of rsync can meet these needs :
- If the same file not deleted.
- If the same file but the date or size is different then removed and replaced (for downloading files rebuild slackware current).
- If there is a new file then the old file is deleted and replaced.

I want rsync to perform updates only the necessary packages. :?
thanks in advance :)

ponce 09-06-2012 02:16 AM

to sync a slackware current via rsync I usually use this syntax (choose the rsync mirror you prefeer and the local path to the slackware-current tree)
Code:

rsync -avP --delete-after rsync://ftp.osuosl.org/slackware/slackware-current/ /data/slackware/slackware-current
but maybe it's better for you to use Alien BOB's mirror-slackware-current shell script, it has many features (read it for easy instructions).

ruario 09-06-2012 03:43 AM

If you have no nearby/fast rsync mirrors, you could use lftp instead to achieve the same kind of effect, like so:

Code:

lftp -c 'open http://mirrors.slackware.com/slackware/; mirror -enx ".*\.mirrorlist" slackware-current'

elexwave 09-06-2012 04:50 AM

I create mirror for direct local access slackpkg, not to make an iso file. I want rsync just fetch some packages when it changed. can I do it? :study:

ruario 09-06-2012 05:04 AM

yes

Didier Spaier 09-06-2012 05:27 AM

I use the same command as ponce for that. Just add the -z option to compress files during the transfer, as you have a limited bandwidth:
rsync -avzP --delete-after

Of course reading the manual to check the meaning of each and every option is a must, as applying blindly a receipt is always risky. This was implicit in previous responses ;)

elexwave 09-06-2012 06:06 AM

@ruario : come on guy, i'm :newbie: in training. :D

@Didier Spaier : yes, its seems I must read a lot of manual program. :study:

thanks to Ponce, ruario and Didier Spaier. It's my solution :
$ rsync -avzP --delete-after --delete \
--exclude "pasture" \
--exclude "source" \
--exclude "usb-and-pxe-installers" \
--exclude "extra" \
--exclude "isolinux" \
--exclude "kernels" \
--exclude "patches" \
--exclude "testing" \
--exclude "slackware64/kde" \
--exclude "slackware64/kdei" \
--exclude "slackware64/e" \
--exclude "slackware64/t" \
--exclude "slackware64/tcl" \
--exclude "slackware64/xap" \
rsync://ftp.osuosl.org/slackware/slackware-current/ /data/slackware/slackware-current

one by one, just rsync core of slackware. ;)

ruario 09-06-2012 07:10 AM

Quote:

Originally Posted by elexwave (Post 4774022)
@ruario : come on guy, i'm :newbie: in training. :D

Sorry, didn't mean to be so sarcastic. ;) Ponce's answer was the right one though.

ruario 09-06-2012 07:12 AM

Quote:

Originally Posted by Didier Spaier (Post 4774002)
I use the same command as ponce for that. Just add the -z option to compress files during the transfer

-z, yes very neat!

Petri Kaukasoina 09-06-2012 07:32 AM

rsync uses zlib for compression. Most files in slackware mirrors are already compressed with xz, and zlib can't compress them, it just eats some additional cpu cycles.

ruario 09-06-2012 07:37 AM

Quote:

Originally Posted by Petri Kaukasoina (Post 4774089)
rsync uses zlib for compression. Most files in slackware mirrors are already compressed with xz, and zlib can't compress them, it just eats some additional cpu cycles.

Ah yes, another good point! The various text files will compress. Though none of them are that big, so I agree the benefits if any will probably be negligible.

Petri Kaukasoina 09-06-2012 08:57 AM

ok, I measured it. I transferred the slackware64-current directory (5.4GB) from an rsync server to another machine. Default configuration in both ends. Without -z the client received 5684687767 bytes and with -z it received 5674708207 bytes. So -z helps but only 0.18 percent.

Without compression the rsyncd server process used 22s cpu time and with compression 184s. That's why compression actually slows down the transfer if the connection has a large bandwidth: I have a 1Gb/s connection and it took 93s real time without compression and 230s with compression.

mRgOBLIN 09-06-2012 02:27 PM

You can also use
Code:

--exclude-from="exclude.list"
and just list all the excludes in exclude.list like this.
Code:

pasture/
source/
usb-and-pxe-installers/
extra/

etc.

Always good to let rsync know you intend to match directories only by including the trailing slash on the directory names too.

Franklin 09-06-2012 06:03 PM

I've used the following for years:

Code:


#!/bin/bash
#
# Check slackware-current
#

# Where to download from
# should script it so that the different sources can be listed and
# selected from the command line

SOURCE="rsync://rsync.osuosl.org/slackware/slackware-current/"

# Change as necessary

OPTIONS="-avzP --delete --delete-after"
EXCLUDE="--exclude=pasture --exclude=kdei \
--exclude=zipslack --exclude=source --exclude=testing/source \
--exclude=extra/source"

DEST="/home/slackware/slackware-current"

case "$1" in

 "-c" )
 echo "Checking..."
 /usr/bin/rsync $OPTIONS $EXCLUDE --dry-run $SOURCE $DEST
 ;;

 "-d" )
 echo "Downloading..."
 /usr/bin/rsync $OPTIONS $EXCLUDE $SOURCE $DEST
 ;;
 
 * )
 echo "Usage: `basename $0` {-c|-d}"
 echo -e "\t-c : Check for updates"
 echo -e "\t-d : Download updates"
 exit
 ;;

esac

check-current -c is a dry run

check-current -d downloads the new files and deletes the replaced files.


All times are GMT -5. The time now is 12:56 AM.