LinuxQuestions.org
Review your favorite Linux distribution.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 12-16-2022, 04:33 PM   #1
alex4buba
Member
 
Registered: Jul 2020
Posts: 620

Rep: Reputation: Disabled
Cron job output result


I had recently a system CRASH and was forced to reinstall KDE Neon Plasma from scratch. It took a couple of days to restore and reinstall some apps and it seems to all work OK now.

I am using the built in Task scheduler to run a Backup cron script to an external hard disk. This also works OK, but the script output of the end, DOES no longer show up, it creates an EMPTY log file on the desktop, below is the command line that used to work, but now doesn't.

usr/bin/rsync -a /home/alexe/afolders/ /media/alexe/Elements/$ldow/ >/var/tmp/anajob.out

Nor does this command:

/usr/bin/rsync -a /home/alexe/Desktop/ /media/alexe/Elements/$ldow/ > /home/alexe/Desktop/dback.txt

Any help will be good
Many thanks
Alex
 
Old 12-16-2022, 06:32 PM   #2
computersavvy
Senior Member
 
Registered: Aug 2016
Posts: 3,345

Rep: Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484
Is it possible you are remembering the command incorrectly?

'rsync -a' does not produce an output to screen, but 'rsync -av' does.

If $ldow has an assigned value before the command is run rsync would create or use the directory. If $ldow has no value then the copy would be created in /media/alexe/Elements/ . There would be no entry made in the logfile without the -v option since rsync runs silently unless there is an error (which would not appear in the log) or the -v (verbose) option is used.
 
Old 12-16-2022, 07:22 PM   #3
alex4buba
Member
 
Registered: Jul 2020
Posts: 620

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by computersavvy View Post
Is it possible you are remembering the command incorrectly?

'rsync -a' does not produce an output to screen, but 'rsync -av' does.

If $ldow has an assigned value before the command is run rsync would create or use the directory. If $ldow has no value then the copy would be created in /media/alexe/Elements/ . There would be no entry made in the logfile without the -v option since rsync runs silently unless there is an error (which would not appear in the log) or the -v (verbose) option is used.
Hello,

Here is the complete script, yes that -av works, BUT it listed in the output 2036 lines of all the files that it copied. I used to get a single line of the action with date and time stamp....

Quote:
#get day of week
dow=$(/usr/bin/date "+%a")

# convert DOW to lower case i.e. Mon to mon
ldow=${dow,,}

/usr/bin/rsync -av /home/alexe/Desktop/ /media/alexe/Elements/$ldow/ > /home/alexe/Desktop/dback.txt
Any other suggestion?
Many thanks
 
Old 12-16-2022, 08:12 PM   #4
elgrandeperro
Member
 
Registered: Apr 2021
Posts: 415
Blog Entries: 2

Rep: Reputation: Disabled
Take out the redirect to the log file and run it a few times. You will see after the first run, all subsequent runs will have little or no output because rsync has already copied the data. Also you might throw a --delete to get rid of removed files and a real representation.
 
Old 12-16-2022, 08:52 PM   #5
alex4buba
Member
 
Registered: Jul 2020
Posts: 620

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by elgrandeperro View Post
Take out the redirect to the log file and run it a few times. You will see after the first run, all subsequent runs will have little or no output because rsync has already copied the data. Also you might throw a --delete to get rid of removed files and a real representation.
Ok, but if i remove the redirect, what file will it be, where will it be located?
I am confused
Cheers
 
Old 12-16-2022, 09:42 PM   #6
computersavvy
Senior Member
 
Registered: Aug 2016
Posts: 3,345

Rep: Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484
I think that suggestion to remove the redirect was so you can see on screen exactly what should be being written to the log file.

I used rsync to copy a directory tree and even with a large number of files the 'rsync -a' gave me nothing in the logfile. With -av it showed exactly what was copied.

After doing a backup that way IF you were writing to the same directory the -av would only display changes. Since you are creating a new directory each time it will always show the full list of files copied.

Copying all the files each time seems a major waste of time and resources when rsync takes care of only doing it as incremental when doing the backup into a previously used copy.

For example, I have about 5 TB of data and if I did a full backup each time it would take hours and a major amount of disk space for each backup. Since I do it as incremental it only takes a few minutes and the backup remains at the same size as the original, forever.
 
Old 12-16-2022, 10:30 PM   #7
elgrandeperro
Member
 
Registered: Apr 2021
Posts: 415
Blog Entries: 2

Rep: Reputation: Disabled
So I don't see a creation every run, sat dir would have sat, fri would have fri etc. But it would accumulate with deleted files and not a representation of a restore, that is why --delete. Yes, you would have everything, but if you stored a iso and then removed it, the image
would be your backup until you cleared it.

So using rsync in this manner at least not copy things already there and you could run it multi times a day and it would only copy updates. That is why I said don't use the redirect, the first time it copies everything and the next time nothing (or at least only the changes).
 
Old 12-16-2022, 11:46 PM   #8
alex4buba
Member
 
Registered: Jul 2020
Posts: 620

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by computersavvy View Post
I think that suggestion to remove the redirect was so you can see on screen exactly what should be being written to the log file.

I used rsync to copy a directory tree and even with a large number of files the 'rsync -a' gave me nothing in the logfile. With -av it showed exactly what was copied.

After doing a backup that way IF you were writing to the same directory the -av would only display changes. Since you are creating a new directory each time it will always show the full list of files copied.

Copying all the files each time seems a major waste of time and resources when rsync takes care of only doing it as incremental when doing the backup into a previously used copy.

For example, I have about 5 TB of data and if I did a full backup each time it would take hours and a major amount of disk space for each backup. Since I do it as incremental it only takes a few minutes and the backup remains at the same size as the original, forever.
Well, I never sit there and watch the backup, I just want to look at the LOG from time to time to see that my backup works. Yes, it is incremental, each day of the week it goes into another folder with the abbreviated day name - sun,mon,tue etc... those folders are only created on the first run.

I used to have a log file that showed ONE single line per each dayly backup - that would be fine for me.

So, How can I get back that this setup?
Thanks again
 
Old 12-16-2022, 11:49 PM   #9
alex4buba
Member
 
Registered: Jul 2020
Posts: 620

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by elgrandeperro View Post
So I don't see a creation every run, sat dir would have sat, fri would have fri etc. But it would accumulate with deleted files and not a representation of a restore, that is why --delete. Yes, you would have everything, but if you stored a iso and then removed it, the image
would be your backup until you cleared it.

So using rsync in this manner at least not copy things already there and you could run it multi times a day and it would only copy updates. That is why I said don't use the redirect, the first time it copies everything and the next time nothing (or at least only the changes).
My local storage device is only 512GB , my external drive is 3TB, I will not likely run out of space.

Cheers
Alex
 
Old 12-17-2022, 10:07 AM   #10
MadeInGermany
Senior Member
 
Registered: Dec 2011
Location: Simplicity
Posts: 2,796

Rep: Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201
-v lists all the copied files.
--stats gives a summary line.
Code:
/usr/bin/rsync -a --stats ...
 
Old 12-17-2022, 02:12 PM   #11
alex4buba
Member
 
Registered: Jul 2020
Posts: 620

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by MadeInGermany View Post
-v lists all the copied files.
--stats gives a summary line.
Code:
/usr/bin/rsync -a --stats ...
Hello,

I made the suggested change, what the log file has now is the following:

Code:
Number of files: 19,848 (reg: 16,178, dir: 3,670)
Number of created files: 2,091 (reg: 1,659, dir: 432)
Number of deleted files: 0
Number of regular files transferred: 1,664
Total file size: 24,699,946,665 bytes
Total transferred file size: 4,516,054,174 bytes
Literal data: 4,516,054,174 bytes
Matched data: 0 bytes
File list size: 393,194
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 4,517,819,136
Total bytes received: 48,137

sent 4,517,819,136 bytes  received 48,137 bytes  81,403,013.93 bytes/sec
total size is 24,699,946,665  speedup is 5.47
I used to get a single line only, with date and time. Is there such an option?

Thank you
 
Old 12-17-2022, 03:00 PM   #12
MadeInGermany
Senior Member
 
Registered: Dec 2011
Location: Simplicity
Posts: 2,796

Rep: Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201
Hmm, you can get this from the file's meta data:
ls -l filename
or
stat -c %y filename

Perhaps you want another command to create the file?
Code:
/usr/bin/rsync -a /home/alexe/Desktop/ /media/alexe/Elements/$ldow/; date > /home/alexe/Desktop/dback.txt
 
Old 12-17-2022, 03:23 PM   #13
michaelk
Moderator
 
Registered: Aug 2002
Posts: 25,710

Rep: Reputation: 5899Reputation: 5899Reputation: 5899Reputation: 5899Reputation: 5899Reputation: 5899Reputation: 5899Reputation: 5899Reputation: 5899Reputation: 5899Reputation: 5899
I don't know of an rsync option but as posted just write the timestamp of the date command to your file.
 
Old 12-17-2022, 04:00 PM   #14
alex4buba
Member
 
Registered: Jul 2020
Posts: 620

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by MadeInGermany View Post
Hmm, you can get this from the file's meta data:
ls -l filename
or
stat -c %y filename

Perhaps you want another command to create the file?
Code:
/usr/bin/rsync -a /home/alexe/Desktop/ /media/alexe/Elements/$ldow/; date > /home/alexe/Desktop/dback.txt
The above command is producing the following output:
Quote:
Sun 18 Dec 2022 08:51:46 AEDT
But, there is no indication if the backup worked or not. Is there a way to show this on the same line?

Many thanks
 
Old 12-17-2022, 04:37 PM   #15
michaelk
Moderator
 
Registered: Aug 2002
Posts: 25,710

Rep: Reputation: 5899Reputation: 5899Reputation: 5899Reputation: 5899Reputation: 5899Reputation: 5899Reputation: 5899Reputation: 5899Reputation: 5899Reputation: 5899Reputation: 5899
Check the rsync exit code. 0 means success.
 
  


Reply

Tags
backup, cron, kde



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] How can I run a Cron job manually or how can I sure my job executed? n00b_noob Linux - Newbie 43 02-24-2021 10:38 PM
Debian daily cron job won't run, but does run in cron.hourly. sandersch Linux - General 7 05-24-2012 01:50 AM
[SOLVED] BASH Script will not get use result of command when run as cron job dahweeds Linux - Newbie 3 07-01-2011 07:01 PM
linux cron job duplicate job question cpthk Linux - Newbie 4 09-11-2009 08:52 PM
adding a perl script to cron.daily / cron.d to setup a cron job CrontabNewBIE Linux - Software 6 01-14-2008 08:16 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 04:57 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration