Linux - SoftwareThis forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Hello all its been a long time since I posted here, but I have a question about rsync. I have a script that I am trying to edit from jms1.net John Simpson if anyone knows him.
What I am after atm (end goal is a bit bigger will get to that on an other thread) is to only backup 1 directory and a set of files.
/usr/rx30 is the directory and i need all subdirectories too.
the list of files i need are as follows:
There will also become a time when I might need to grab additional directories off of /usr. Below is the script that I'm using with notes as to what I have modified. My issue is I'm getting an error due to my INCLUDES section.
Code:
# cat backup-servers
#!/bin/sh
#
# backup-servers
# John Simpson <jms1@jms1.net> 2005-09-12
# Edited for TDS by Ray Brunkow <rbrunkow@rx30.com> 2011-11-14
# Documentation: http://www.jms1.net/docs/rsync-backup.shtml
#
###############################################################################
#
# Copyright (C) 2005,2006,2007,2008 John Simpson.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 or version 3 of the
# license, at your option.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
###############################################################################
PATH="/usr/bin:/bin"
##############################################################################
#
# configuration
BACKUPDIR="/backup"
LOGFILE="$BACKUPDIR/backup.log" # in addition to syslog
SERVERS="10.10.4.77" # servers to be backed up
KEY="/root/.ssh/id_dsa_backup" # ssh key file
##############################################################################
# These can be uncommented out if you wish to perform a full system backup except these directories.
#
#EXCLUDES="/dev /proc /sys /tmp /var/tmp"
#EXCLUDES="$EXCLUDES /var/lib/pgsql/data /var/lib/mysql"
#EXCLUDES="$EXCLUDES /var/qmail/queue /var/qmail/simscan"
##############################################################################
#
# configured for TDS for INLCUDES for the rx30 directory and specific folders/files we need
INCLUDES="/etc/hosts /etc/exports /etc/dhcpd.conf /etc/resolv.conf /etc/cups/printers.conf /etc/sysconfig/static-routes /opt/ltsp/i386/etc/lts.conf /etc/sysconfig/network-scripts/ifcfg-eth0 /etc/sysconfig/network-scripts/ifcfg-eth0:0"
#INCLUDES="$INCLUDES "
# make sure cron's output gets mailed to the right place
MAILTO="postmaster@domain.xyz"
export MAILTO
##############################################################################
function log() {
logger -t backup -p local5.info "$@"
if [ -n "${LOGFILE:-}" ]
then
echo `date` "$@" >> $LOGFILE
fi
}
##############################################################################
##############################################################################
##############################################################################
# make rsync use ssh with the correct key
RSYNC_RSH="/usr/bin/ssh -i $KEY"
export RSYNC_RSH
# make sure agent keys don't override the backup key
unset SSH_AUTH_SOCK
########################################
# if we were called with "-v", pass the "-v" along to rsync
if [ "$1" == "-v" ]
then
OPTV="-v"
else
OPTV=""
fi
########################################
# build list of excludes
# uncomment for the EXCLUDES
#EXC=""
#for e in $EXCLUDES
#do
# EXC="$EXC --exclude=$e"
#done
#######################################
# build list of INCLUDES
EXC=""
for e in $INCLUDES
do
EXC="$EXC --includes=$e"
done
########################################
# let's do it
log '/===================='
log "backup-server $OPTV starting"
for SERVER in $SERVERS
do
########################################
# make sure target directory exists
TARGET="$BACKUPDIR/$SERVER"
if [ ! -d $TARGET ]
then
mkdir -p -m 0755 $TARGET
fi
########################################
# make sure an earlier backup isn't already running
if egrep -q "^rsync.*${TARGET}" /proc/*/cmdline > /dev/null 2>&1
then
log "$SERVER already in progress"
continue
fi
########################################
# do the deed
# uncomment the line with --bwlimit=1024 for WAN support.
log "$SERVER starting"
#rsync -aS $OPTV --delete $EXC --bwlimit=1024 $SERVER:/ ${TARGET}/
########################################
# use this line ONLY for LAN backups.
rsync -aS $OPTV --delete $EXC $SERVER:/ ${TARGET}/
rv=$?
log "$SERVER done rv=$rv"
done
log "backup complete"
log '\===================='
for the INCLUDES portion of this script do i need to put each file on its own INCLUDE? as it appears to not be grabbing the files i want. I have made some modifications, but get the same results:
Code:
INCLUDES="hosts exports dhcpd.conf resolv.conf cups/printers.conf /sysconfig/static-routes /opt/ltsp/i386/etc/lts.conf /sysconfig/network-scripts/ifcfg-eth0 /sysconfig/network-scripts/ifcfg-eth0:0"
#### Below is the first attempt at INCLUDES, both sets give the same error
#INCLUDES="/etc/hosts /etc/exports /etc/dhcpd.conf /etc/resolv.conf /etc/cups/printers.conf /etc/sysconfig/static-routes /opt/ltsp/i386/etc/lts.conf /etc/sysconfig/network-scripts/ifcfg-eth0 /etc/sysconfig/network-scripts/ifcfg-eth0:0"
INCLUDES="$INCLUDES --include-form"
First of all thanks for including complete shell script code (showing Simpsons' "INLCUDES" spelling error) and "-v" output, that helps a lot. While he uses a variable called "INCLUDES" the seemingly-rsync-equivalent "--includes=" doesn't exist: it's either "--include=" or "--include-from=". Same goes for your typo: it's not "--include-form=" but "--include-from=".
ahh thank you. those typo's are mine, not his . I am the one who added the INCLUDES... I'll make the adjustment. So in short its --include-from not form... interesting. Thank you.
Now that I have the dsa key fixed and the script is running. Next question. I only want to grab the files/directories listed above. I do not want an entire system backed up.
I only want to grab the files/directories listed above. I do not want an entire system backed up.
Last time you've included information and output. You've been verbally rewarded for doing that. That should trigger a Pavlov reaction to always do that.
* BTW rsync has a nifty "-n" switch (think "--dry-run") with which you can test how things will go w/o actually fscking up your backup data.
Again thank you for the reply, here is a directory list of what the first successful connection of the script grabbed and placed into the proper folder:
Code:
ls -laF /backup/10.10.4.77/
total 136
drwxr-xr-x 24 root root 4096 Nov 14 09:40 ./
drwxr-xr-x 3 root root 4096 Nov 14 11:15 ../
-rw-r--r-- 1 root root 0 Nov 14 09:40 .autofsck
-rw-r--r-- 1 root root 0 Nov 11 13:18 .autorelabel
-rw-r--r-- 1 root root 4634 Nov 11 13:18 .swatch_script.1629
-rw-r--r-- 1 root root 4634 Nov 14 09:40 .swatch_script.1645
-rw-r--r-- 1 root root 4634 Nov 11 17:21 .swatch_script.1647
drwxr-xr-x 2 root root 4096 Nov 14 11:00 bin/
drwxr-xr-x 4 root root 4096 Nov 30 2010 boot/
drwxr-xr-x 12 root root 4096 Nov 14 09:41 dev/
drwxr-xr-x 101 root root 12288 Nov 14 09:40 etc/
drwxr-xr-x 2 root root 4096 Mar 9 2009 home/
drwxr-xr-x 13 root root 4096 Nov 11 14:28 lib/
drwx------ 2 root root 4096 Nov 11 07:55 lost+found/
drwxr-xr-x 2 root root 4096 Mar 9 2009 media/
drwxr-xr-x 2 root root 4096 Nov 14 09:40 misc/
drwxr-xr-x 5 root root 4096 Nov 11 13:08 mnt/
drwxr-xr-x 2 root root 4096 Nov 14 09:40 net/
drwxr-xr-x 6 root root 4096 Apr 8 2011 opt/
dr-xr-xr-x 147 root root 4096 Nov 14 18:19 proc/
drwxr-x--- 5 root root 4096 Nov 11 17:03 root/
drwxr-xr-x 2 root root 12288 Nov 11 14:30 sbin/
drwxr-xr-x 2 root root 4096 Nov 11 12:56 selinux/
drwxr-xr-x 2 root root 4096 Mar 9 2009 srv/
drwxr-xr-x 11 root root 4096 Nov 14 04:39 sys/
drwxr-xr-x 3 root root 4096 Nov 11 13:08 tftpboot/
drwxrwxrwx 16 root root 4096 Nov 14 17:25 tmp/
drwxr-xr-x 16 root root 4096 Nov 11 13:24 usr/
drwxr-xr-x 21 root root 4096 Nov 11 13:07 var/
As you can clearly see this grabbed way more then I desire.
Hmm. I'll have to get back to you on that one. To me it would make sense to just concentrate on rsyncing the subset of "/" (basically here your first root would be "/usr/rx30") and then rsync the /etc/ and /opt/ files in a separate run for now. I haven't used something like "--exclude=/", include a subset of "/" and then made it work OK.
Instead of starting a new thread I will revive this one. I have a new script with -x -v output for some help.
the script:
Code:
#!/bin/sh
###################################################
# Atempt at a custom rsync for /usr/rx30
#
#
###############################################################################
#
# Copyright (C) 2005,2006,2007,2008 Ray Brunkow.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 or version 3 of the
# license, at your option.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
###############################################################################
##############################################################################
#
# configuration
BACKUPDIR="/backup"
LOGFILE="$BACKUPDIR/backup.log" # in addition to syslog
SERVERS="10.10.4.77" # servers to be backed up
#
#
###############################################################################
#
# Start the log function
#
##############################################################################
function log() {
logger -t backup -p local5.info "$@"
if [ -n "${LOGFILE:-}" ]
then
echo `date` "$@" >> $LOGFILE
fi
}
##########################################################
# make rsync use ssh with the correct key
RSYNC_RSH="/usr/bin/ssh -i $KEY"
export RSYNC_RSH
##########################################################
# make sure agent keys don't override the backup key
unset SSH_AUTH_SOCK
#########################################################
# if we were called with "-v", pass the "-v" along to rsync
if [ "$1" == "-v" ]
then
OPTV="-v"
else
OPTV=""
fi
########################################
# Getting started
log '/===================='
log "backup-server $OPTV starting"
for SERVER in $SERVERS
do
########################################
# make sure target directory exists
TARGET="$BACKUPDIR/$SERVER"
if [ ! -d $TARGET ]
then
mkdir -p -m 0755 $TARGET
fi
########################################
# make sure an earlier backup isn't already running
if egrep -q "^rsync.*${TARGET}" /proc/*/cmdline > /dev/null 2>&1
then
log "$SERVER already in progress"
continue
fi
########################################
##############################################################################
# rsync command without bandwidth limitations. If you with to limit the bandwidth
# uncomment out the line below:
##############################################################################
# rsync -avzpS --bwlimit=1024 -e "ssh -i /root/.ssh/id_dsa_backup" root@10.10.4.77:/usr/rx30 /backup/10.10.4.77/ -v
rsync -avzpS $OPTV /usr/rx30:/ ${TARGET}/ -v
# rsync -avzpS -e "ssh -i /root/.ssh/id_dsa_backup" root@10.10.4.77:/usr/rx30 /backup/10.10.4.77/ -v
rv=$?
log "$SERVER done rv=$rv"
done
log "backup complete"
log '\===================='
################################################################
# end log process and write file to backup up server
################################################################
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
. I am the one who added the INCLUDES... I'll make the adjustment. So in short its --include-from not form... interesting. Thank you.
