RSYNC Not Working

carlosinfl · 10-10-2007, 10:11 AM

Guys - I set up 7.04 to sync my home directory with an external usb drive every 5 minutes. I have added a new file to my home directory and it has yet to show up on /media/disk for the past hour and I don't understand why. The RSYNC is setup via CRON to run every 5 minutes so this makes no sense to me...

Can anyone please tell me what is wrong?

Here is my RSYNC command in crontab.

Code:

cwilliams@cwilliams:~$ crontab -l
# m h  dom mon dow   command
5 * * * * rsync --delete -rdtvu /home/cwilliams/ /media/disk
cwilliams@cwilliams:~$ ls
BEWS_11D.7170_LINUX-UNIX-MAC-NT4_AGENTS.2.tar.gz  idelab revised 8-8-06.vsd  permission.txt   wallpaper
contact.txt                                       linux_commands.txt         phonelist.txt    whitelist.txt
contact.txt~                                      mailman.txt                spam_lookup.txt  whitelist.txt~
create_calendar.txt                               mysql.txt                  stricom_vlk.txt
Desktop                                           old_users                  symantec
cwilliams@cwilliams:~$ ls /media/disk/
contact.txt          Desktop                    mailman.txt  permission.txt   stricom_vlk.txt  whitelist.txt
contact.txt~         idelab revised 8-8-06.vsd  mysql.txt    phonelist.txt    symantec         whitelist.txt~
create_calendar.txt  linux_commands.txt         old_users    spam_lookup.txt  wallpaper

As you can see /media/disk is missing the BEWS* tarball that is present in /home/cwilliams. That file has been there for two hours.

pronoy · 10-10-2007, 11:24 AM

It is working if you run the command directly from the command line?
Not sure, since the command will run in every 5 mins, they may get overlapped if the first command is not complete by that time.

carlosinfl · 10-10-2007, 11:31 AM

I can run the command manually which takes less than 4 seconds to run so I can't see it being overlapped in a 5 minute interval. When I run it manually the sync happens right away so I think the issue is cron or perhaps it does not like something but I have no idea what that is...

Code:

cwilliams@cwilliams:/share$ rsync --delete -rdtvu /home/cwilliams/ /media/disk
building file list ... done
deleting .mozilla/firefox/uj02mrmu.default/Cache/D36A5345d01
deleting .mozilla/firefox/uj02mrmu.default/Cache/904E844Dd01
deleting .mozilla/firefox/uj02mrmu.default/Cache/8D697FAEd01
./
skipping non-regular file ".DCOPserver_cwilliams_:0"
rsync: send_files failed to open "/home/cwilliams/.viminfo": Permission denied (13)
.gconf/apps/nautilus/preferences/
.gconf/apps/nautilus/preferences/%gconf.xml
skipping non-regular file ".kde/cache-cwilliams"
skipping non-regular file ".kde/socket-cwilliams"
skipping non-regular file ".kde/tmp-cwilliams"
skipping non-regular file ".mozilla-thunderbird/evcd6ud0.default/lock"
.mozilla-thunderbird/evcd6ud0.default/panacea.dat
.mozilla-thunderbird/evcd6ud0.default/ImapMail/mail.ideorlando.org/INBOX.msf
.mozilla-thunderbird/evcd6ud0.default/ImapMail/mail.ideorlando.org/Sent-1.msf
.mozilla-thunderbird/evcd6ud0.default/ImapMail/mail.ideorlando.org/Trash.msf
.mozilla-thunderbird/evcd6ud0.default/ImapMail/mail.ideorlando.org/INBOX.sbd/Logs.sbd/LT2FS2.msf
.mozilla/firefox/uj02mrmu.default/
.mozilla/firefox/uj02mrmu.default/cookies.txt
.mozilla/firefox/uj02mrmu.default/history.dat
skipping non-regular file ".mozilla/firefox/uj02mrmu.default/lock"
.mozilla/firefox/uj02mrmu.default/sessionstore.js
.mozilla/firefox/uj02mrmu.default/urlclassifier2.sqlite
.mozilla/firefox/uj02mrmu.default/Cache/
.mozilla/firefox/uj02mrmu.default/Cache/B0F65723d01
.mozilla/firefox/uj02mrmu.default/Cache/_CACHE_001_
.mozilla/firefox/uj02mrmu.default/Cache/_CACHE_002_
.mozilla/firefox/uj02mrmu.default/Cache/_CACHE_003_
.nautilus/metafiles/
.nautilus/metafiles/file:%2F%2F%2Fhome%2Fcwilliams.xml
skipping non-regular file ".openoffice.org2/user/config/arrowhd_en-GB.soe"
skipping non-regular file ".openoffice.org2/user/config/arrowhd_en-US_en-ZA.soe"
skipping non-regular file ".openoffice.org2/user/config/classic_en-GB.sog"
skipping non-regular file ".openoffice.org2/user/config/classic_en-US_en-ZA.sog"
skipping non-regular file ".openoffice.org2/user/config/hatching_en-GB.soh"
skipping non-regular file ".openoffice.org2/user/config/hatching_en-US_en-ZA.soh"
skipping non-regular file ".openoffice.org2/user/config/modern_en-GB.sog"
skipping non-regular file ".openoffice.org2/user/config/modern_en-US_en-ZA.sog"
skipping non-regular file ".openoffice.org2/user/config/palette_en-GB.soc"
skipping non-regular file ".openoffice.org2/user/config/palette_en-US_en-ZA.soc"
skipping non-regular file ".openoffice.org2/user/config/styles_en-GB.sod"
skipping non-regular file ".openoffice.org2/user/config/styles_en-US_en-ZA.sod"
zoo/

sent 21879479 bytes  received 408 bytes  43759774.00 bytes/sec
total size is 19473598011  speedup is 890.02
rsync error: some files could not be transferred (code 23) at main.c(977) [sender=2.6.9]

Above is the verbose output of my RSYNC command and it has permission errors on some folders which I don't care about. Can I exclude any files and or folders that are hidden? How would I execute that into my RSYNC command to leave out hidden files / folders?

tredegar · 10-10-2007, 12:00 PM

cron usually needs the full path to an executable. So try this:

Code:

5 * * * * /usr/bin/rsync --delete -rdtvu /home/cwilliams/ /media/disk

trickykid · 10-10-2007, 04:02 PM

Yes, put the full path. It's a good rule of thumb to always put the full path to an executable or script as the path might not always in the user who's running it own path. Make it a habit from now on.

carlosinfl · 10-10-2007, 06:05 PM

Quote:

Originally Posted by trickykid

Yes, put the full path. It's a good rule of thumb to always put the full path to an executable or script as the path might not always in the user who's running it own path. Make it a habit from now on.

Thanks - will do so for now on! As for if that was the problem or not, I don't know. I will post back after I look at some time stamps and diff the two destination and source folders.

To be continued...

carlosinfl · 10-12-2007, 02:59 PM

OK so I have updated my cron job to read as follows:

Code:

cwilliams@cwilliams:~$ crontab -l
# m h  dom mon dow   command
5 * * * * /usr/bin/rsync --delete -rdtvu /home/cwilliams/ /media/disk

I waited 5 minutes and never saw the file sync over to /media/disk however at some point with in the next hour or less the file will sync to /media/disk so my question is why is the sync not occurring every 5 minutes. Where can I check for errors and or issues that perhaps are not being visible.

**EDIT***

Exactly at 4:05 pm according to my system time, the sync went through so is there something wrong with my cron entry as to not syncing every 5 minutes?

cygnus-x1 · 10-12-2007, 03:11 PM

Quote:

Originally Posted by Carlwill

OK so I have updated my cron job to read as follows:

Code:

cwilliams@cwilliams:~$ crontab -l
# m h  dom mon dow   command
5 * * * * /usr/bin/rsync --delete -rdtvu /home/cwilliams/ /media/disk

I waited 5 minutes and never saw the file sync over to /media/disk however at some point with in the next hour or less the file will sync to /media/disk so my question is why is the sync not occurring every 5 minutes. Where can I check for errors and or issues that perhaps are not being visible.

**EDIT***

Exactly at 4:05 pm according to my system time, the sync went through so is there something wrong with my cron entry as to not syncing every 5 minutes?

Shouldn't your rsync be ??

*/5 * * * *

I think just saying 5 means 5 minutes after some hour which you do not have defined. If you had

5 10 * * *

This would say do this job at 10:05

I think I have this correct

carlosinfl · 10-12-2007, 04:09 PM

Thanks because I am very sure that its now running 5 past every hour every day. I will fix it. Thanks!

trickykid · 10-14-2007, 07:37 AM

Quote:

Originally Posted by Carlwill

Thanks because I am very sure that its now running 5 past every hour every day. I will fix it. Thanks!

Heh, I didn't even notice the discrepancy myself. You did in fact have it set to run 5 minutes after each hour.

tredegar · 10-14-2007, 08:02 AM

Beware:
You seem to be wanting to make a backup of your home to an external disk every so often.
OK. Nice idea

But you have rsync with the --delete option. Now, suppose you deleted a file from your home by mistake. Your cron job runs, the directories are synced and the file is also deleted from your "backup". Whoops! The file is truly gone.
This may not be what you want to achieve

cygnus-x1 · 10-15-2007, 02:17 PM

Quote:

Originally Posted by tredegar

Beware:
You seem to be wanting to make a backup of your home to an external disk every so often.
OK. Nice idea

But you have rsync with the --delete option. Now, suppose you deleted a file from your home by mistake. Your cron job runs, the directories are synced and the file is also deleted from your "backup". Whoops! The file is truly gone.
This may not be what you want to achieve

Well he does live dangerously. Even his sig line has 'rm -rf /' =)

Yes do be careful with --delete. It can be a good thing to have if you have a rolling backup system in place where you keep daily backup sets in different directories otherwise I would NOT include --delete.

carlosinfl · 10-19-2007, 08:26 AM

Yes - I am very careful when I remove something from my home folder simply because I want the --delete option to avoid useless clutter on my external drive. Thanks for the warning however.

Now my question is how do I avoid the RSYNC to copy over hidden files and folders from my home dire to my external drive? I am also copying a bunch of useless system data that are profile or system dependant. They all begin with a .* so I would assume I can do some kind of --exclude command in there but I don't know what the command should look like. I don't want to mess anything up if you know what I mean.

Anyone have any ideas?

carlosinfl · 10-24-2007, 01:10 PM

Anyone have any ideas how to run this rsync w/o or excluding the hidden files and directories?