LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Software (https://www.linuxquestions.org/questions/linux-software-2/)
-   -   Cron job output result (https://www.linuxquestions.org/questions/linux-software-2/cron-job-output-result-4175719861/)

alex4buba 12-16-2022 04:33 PM

Cron job output result
 
I had recently a system CRASH and was forced to reinstall KDE Neon Plasma from scratch. It took a couple of days to restore and reinstall some apps and it seems to all work OK now.

I am using the built in Task scheduler to run a Backup cron script to an external hard disk. This also works OK, but the script output of the end, DOES no longer show up, it creates an EMPTY log file on the desktop, below is the command line that used to work, but now doesn't.

usr/bin/rsync -a /home/alexe/afolders/ /media/alexe/Elements/$ldow/ >/var/tmp/anajob.out

Nor does this command:

/usr/bin/rsync -a /home/alexe/Desktop/ /media/alexe/Elements/$ldow/ > /home/alexe/Desktop/dback.txt

Any help will be good
Many thanks
Alex

computersavvy 12-16-2022 06:32 PM

Is it possible you are remembering the command incorrectly?

'rsync -a' does not produce an output to screen, but 'rsync -av' does.

If $ldow has an assigned value before the command is run rsync would create or use the directory. If $ldow has no value then the copy would be created in /media/alexe/Elements/ . There would be no entry made in the logfile without the -v option since rsync runs silently unless there is an error (which would not appear in the log) or the -v (verbose) option is used.

alex4buba 12-16-2022 07:22 PM

Quote:

Originally Posted by computersavvy (Post 6398371)
Is it possible you are remembering the command incorrectly?

'rsync -a' does not produce an output to screen, but 'rsync -av' does.

If $ldow has an assigned value before the command is run rsync would create or use the directory. If $ldow has no value then the copy would be created in /media/alexe/Elements/ . There would be no entry made in the logfile without the -v option since rsync runs silently unless there is an error (which would not appear in the log) or the -v (verbose) option is used.

Hello,

Here is the complete script, yes that -av works, BUT it listed in the output 2036 lines of all the files that it copied. I used to get a single line of the action with date and time stamp....

Quote:

#get day of week
dow=$(/usr/bin/date "+%a")

# convert DOW to lower case i.e. Mon to mon
ldow=${dow,,}

/usr/bin/rsync -av /home/alexe/Desktop/ /media/alexe/Elements/$ldow/ > /home/alexe/Desktop/dback.txt
Any other suggestion?
Many thanks

elgrandeperro 12-16-2022 08:12 PM

Take out the redirect to the log file and run it a few times. You will see after the first run, all subsequent runs will have little or no output because rsync has already copied the data. Also you might throw a --delete to get rid of removed files and a real representation.

alex4buba 12-16-2022 08:52 PM

Quote:

Originally Posted by elgrandeperro (Post 6398385)
Take out the redirect to the log file and run it a few times. You will see after the first run, all subsequent runs will have little or no output because rsync has already copied the data. Also you might throw a --delete to get rid of removed files and a real representation.

Ok, but if i remove the redirect, what file will it be, where will it be located?
I am confused
Cheers

computersavvy 12-16-2022 09:42 PM

I think that suggestion to remove the redirect was so you can see on screen exactly what should be being written to the log file.

I used rsync to copy a directory tree and even with a large number of files the 'rsync -a' gave me nothing in the logfile. With -av it showed exactly what was copied.

After doing a backup that way IF you were writing to the same directory the -av would only display changes. Since you are creating a new directory each time it will always show the full list of files copied.

Copying all the files each time seems a major waste of time and resources when rsync takes care of only doing it as incremental when doing the backup into a previously used copy.

For example, I have about 5 TB of data and if I did a full backup each time it would take hours and a major amount of disk space for each backup. Since I do it as incremental it only takes a few minutes and the backup remains at the same size as the original, forever.

elgrandeperro 12-16-2022 10:30 PM

So I don't see a creation every run, sat dir would have sat, fri would have fri etc. But it would accumulate with deleted files and not a representation of a restore, that is why --delete. Yes, you would have everything, but if you stored a iso and then removed it, the image
would be your backup until you cleared it.

So using rsync in this manner at least not copy things already there and you could run it multi times a day and it would only copy updates. That is why I said don't use the redirect, the first time it copies everything and the next time nothing (or at least only the changes).

alex4buba 12-16-2022 11:46 PM

Quote:

Originally Posted by computersavvy (Post 6398399)
I think that suggestion to remove the redirect was so you can see on screen exactly what should be being written to the log file.

I used rsync to copy a directory tree and even with a large number of files the 'rsync -a' gave me nothing in the logfile. With -av it showed exactly what was copied.

After doing a backup that way IF you were writing to the same directory the -av would only display changes. Since you are creating a new directory each time it will always show the full list of files copied.

Copying all the files each time seems a major waste of time and resources when rsync takes care of only doing it as incremental when doing the backup into a previously used copy.

For example, I have about 5 TB of data and if I did a full backup each time it would take hours and a major amount of disk space for each backup. Since I do it as incremental it only takes a few minutes and the backup remains at the same size as the original, forever.

Well, I never sit there and watch the backup, I just want to look at the LOG from time to time to see that my backup works. Yes, it is incremental, each day of the week it goes into another folder with the abbreviated day name - sun,mon,tue etc... those folders are only created on the first run.

I used to have a log file that showed ONE single line per each dayly backup - that would be fine for me.

So, How can I get back that this setup?
Thanks again

alex4buba 12-16-2022 11:49 PM

Quote:

Originally Posted by elgrandeperro (Post 6398403)
So I don't see a creation every run, sat dir would have sat, fri would have fri etc. But it would accumulate with deleted files and not a representation of a restore, that is why --delete. Yes, you would have everything, but if you stored a iso and then removed it, the image
would be your backup until you cleared it.

So using rsync in this manner at least not copy things already there and you could run it multi times a day and it would only copy updates. That is why I said don't use the redirect, the first time it copies everything and the next time nothing (or at least only the changes).

My local storage device is only 512GB , my external drive is 3TB, I will not likely run out of space.

Cheers
Alex

MadeInGermany 12-17-2022 10:07 AM

-v lists all the copied files.
--stats gives a summary line.
Code:

/usr/bin/rsync -a --stats ...

alex4buba 12-17-2022 02:12 PM

Quote:

Originally Posted by MadeInGermany (Post 6398492)
-v lists all the copied files.
--stats gives a summary line.
Code:

/usr/bin/rsync -a --stats ...

Hello,

I made the suggested change, what the log file has now is the following:

Code:

Number of files: 19,848 (reg: 16,178, dir: 3,670)
Number of created files: 2,091 (reg: 1,659, dir: 432)
Number of deleted files: 0
Number of regular files transferred: 1,664
Total file size: 24,699,946,665 bytes
Total transferred file size: 4,516,054,174 bytes
Literal data: 4,516,054,174 bytes
Matched data: 0 bytes
File list size: 393,194
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 4,517,819,136
Total bytes received: 48,137

sent 4,517,819,136 bytes  received 48,137 bytes  81,403,013.93 bytes/sec
total size is 24,699,946,665  speedup is 5.47

I used to get a single line only, with date and time. Is there such an option?

Thank you

MadeInGermany 12-17-2022 03:00 PM

Hmm, you can get this from the file's meta data:
ls -l filename
or
stat -c %y filename

Perhaps you want another command to create the file?
Code:

/usr/bin/rsync -a /home/alexe/Desktop/ /media/alexe/Elements/$ldow/; date > /home/alexe/Desktop/dback.txt

michaelk 12-17-2022 03:23 PM

I don't know of an rsync option but as posted just write the timestamp of the date command to your file.

alex4buba 12-17-2022 04:00 PM

Quote:

Originally Posted by MadeInGermany (Post 6398535)
Hmm, you can get this from the file's meta data:
ls -l filename
or
stat -c %y filename

Perhaps you want another command to create the file?
Code:

/usr/bin/rsync -a /home/alexe/Desktop/ /media/alexe/Elements/$ldow/; date > /home/alexe/Desktop/dback.txt

The above command is producing the following output:
Quote:

Sun 18 Dec 2022 08:51:46 AEDT
But, there is no indication if the backup worked or not. Is there a way to show this on the same line?

Many thanks

michaelk 12-17-2022 04:37 PM

Check the rsync exit code. 0 means success.


All times are GMT -5. The time now is 11:38 PM.