Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum. |
Notices |
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
Are you new to LinuxQuestions.org? Visit the following links:
Site Howto |
Site FAQ |
Sitemap |
Register Now
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
|
 |
02-10-2014, 03:03 PM
|
#1
|
Member
Registered: Sep 2008
Posts: 556
Rep:
|
script to run rsync process, stop and clean exit.
I'm a newbie with linux scripts so please go easy....
I'm using the following script on a centos 6 server to rsync a centos mirror to my own local mirror:
Quote:
#!/bin/bash
while [ 1 ]
do
/usr/bin/rsync -art --delete -vv --progress rsync://mirror.nsc.liu.se/centos-debuginfo/6/x86_64/ /var/www/html/debug.info/6/x86_64
if [ "$?" = "0" ] ; then
echo "rsync completed normally"
exit
else
echo "Rsync failure. Backing off and retrying..."
sleep 20
fi
done
|
this script works well in that it auto resumes rsyncing when the connection is broken. What I want to do however is make this process run for a set amount of time and then stop the rsync process and clean exit. Is there a way to do this?
I've tried putting this in at various aplaces:
Quote:
killall -SIGTERM rsync
exit 0
|
but it doesn't work, the script just carries on.
I've also tried the script this way:
Quote:
/usr/bin/rsync -art -vv --progress rsync://mirror.nsc.liu.se/centos-debuginfo/6/x86_64/ /var/www/html/debug.info/6/x86_64 &
sleep 20
killall -SIGTERM rsync
exit 0
|
and it does stop after 20 secs but it's messy, it doesn't clean exit and it doesn't contain any auto resume. What I ultimately want to do is run this script as a cron job during the night so it starts, runs for a set time and then stops clean and exits. Is there a way to have both these things in one script? Thanks for any help.
Last edited by tonj; 02-10-2014 at 03:12 PM.
|
|
|
02-11-2014, 04:00 AM
|
#2
|
LQ Newbie
Registered: Jun 2013
Posts: 14
Rep: 
|
Just thinking that maybe their is no clean exit for rsync if it is busy, the only clean exit is to let it finish.
|
|
|
02-11-2014, 04:26 AM
|
#3
|
Member
Registered: Sep 2008
Posts: 556
Original Poster
Rep:
|
but my problem is this:
I have a monthly download limit imposed by my isp, and the volume needed to build up a local mirror eats it away. However downloading between midnight and 8am isn't counted so I use a cron job to download during these hours, but so far it's messy trying to stop one download and start another during the night. I'm getting errors and failures to download.
|
|
|
02-11-2014, 09:46 AM
|
#4
|
LQ Veteran
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Rep: 
|
Quote:
Originally Posted by tonj
I'm getting errors and failures to download.
|
Better to diagnose the "errors and failures" and once those are corrected, schedule it as a late night cronjob, no?
What errors and failures are you getting?
60.4'ish G of data in that remote directory.
Do you have room for that?
Last edited by Habitual; 02-11-2014 at 09:50 AM.
|
|
|
02-11-2014, 02:38 PM
|
#5
|
Member
Registered: Sep 2008
Posts: 556
Original Poster
Rep:
|
the failures are because (using the second script) the internet connection sometimes gets broken and the stop and exit from the script is messy. The first script solves these problems, but the first script doesn't have a time limit in it - which I would like, if I could get some help on how to correctly script it.
Last edited by tonj; 02-11-2014 at 02:40 PM.
|
|
|
02-11-2014, 03:03 PM
|
#6
|
LQ Veteran
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Rep: 
|
http://wiki.centos.org/HowTos/CreateLocalMirror
says "find a mirror near you that supports rsync" and rsync://mirror.nsc.liu.se/CentOS is listed, but http://mirror.nsc.liu.se/centos-debuginfo is not.
I don't think that site allows rsync into that directory.
This however works for me here:
Code:
cd /var/www/html/debug.info/6/x86_64
lftp -c "open http://mirror.nsc.liu.se/centos-debuginfo/6/x86_64/ ; mirror"
and seems to supports a resume-type function
I watched a stat command on one of the existing files and killed the job and restarted it and the file *times weren't changed at all.
Last edited by Habitual; 02-11-2014 at 03:05 PM.
|
|
|
02-11-2014, 03:10 PM
|
#7
|
Member
Registered: Sep 2008
Posts: 556
Original Poster
Rep:
|
ok thanks but I'm fairly certain the problem here is in the script I'm using, not in the site I'm downloading from. The stop and clean exit has to be done using a time limit in the script (preferably in the first script I mentioned) not from a human pressing keys on the keyboard.
|
|
|
02-11-2014, 03:42 PM
|
#8
|
LQ Veteran
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Rep: 
|
I finally got it to "run" with this partial output:
Code:
/usr/bin/rsync -art --delete -vv --progress rsync://mirror.nsc.liu.se/centos-debuginfo/6/x86_64/
opening tcp connection to mirror.nsc.liu.se port 873
sending daemon args: --server --sender -vvlogDtpre.iLsf . centos-debuginfo/6/x86_64/
NSC public mirror server
Admin contact: mirror-admin@nsc.liu.se
receiving incremental file list
delta-transmission enabled
and a list of 2900 files, but NO files were downloaded.
I'm telling you lftp is the way to go 
Last edited by Habitual; 02-11-2014 at 03:44 PM.
|
|
|
02-11-2014, 03:55 PM
|
#9
|
LQ Veteran
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Rep: 
|
Now, this is working
Code:
rsync -art --delete -vv --progress rsync://mirror.nsc.liu.se/centos-debuginfo/6/x86_64/ .
notice the period? #ignore this comment as /var/www/html/debug.info/6/x86_64 satisfies.
Once you have a command that doesn't error out, then you can work on the timing/resume issue in your script.
Last edited by Habitual; 02-11-2014 at 05:00 PM.
|
|
|
02-11-2014, 04:16 PM
|
#10
|
LQ Veteran
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Rep: 
|
Something like this may work?
Code:
#!/bin/bash
SLEEP=20m
while : #infinite loop, jump out with break
do
/usr/bin/rsync -art --delete -vv --progress rsync://mirror.nsc.liu.se/centos-debuginfo/6/x86_64/ /var/www/html/debug.info/6/x86_64 && break
sleep $SLEEP
done
|
|
|
All times are GMT -5. The time now is 04:18 PM.
|
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
|
Latest Threads
LQ News
|
|