Linux - SoftwareThis forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
#!/bin/bash
#get day of week
dow=$(/usr/bin/date "+%a")
# convert DOW to lower case i.e. Mon to mon
ldow=${dow,,}
/usr/bin/rsync -av /home/alex/afolders/ /media/alex/Elements/$ldow/ >> /tmp/rsync.log
the rsync.log shows this morning the following single line
Code:
sending incremental file list
The syslog file shows as if all went OK, but when I check th eindividual files that I know were changed, I discover that they were not copied, here are some examples:
Try replacing your ">>" with "&>" so that both stdout AND stderr are redirected to the log file.
This way if it errors out before a file is synced, there will be evidence.
OR, you could redirect the error output to a "rsync.err" file. This would preserve the trace, but keep your log file clean.
Have you tested this script form the command line? ie, a manual run rather than scheduled.
Try replacing your ">>" with "&>" so that both stdout AND stderr are redirected to the log file.
This way if it errors out before a file is synced, there will be evidence.
OR, you could redirect the error output to a "rsync.err" file. This would preserve the trace, but keep your log file clean.
Have you tested this script form the command line? ie, a manual run rather than scheduled.
Sorry, I am still new to all this, you mean - instead of this line
Commands have two places where output can go - "standard output" (stdout) and "standard error" (stderr) - the latter is usually used for error messages, which are helpful when things are not working as expected.
The ">>" in your command appends the stdout of rsync to the file specified, but discards the stderr.
To append both stdout and stderr to your log file, use "&>>" instead of ">>".
(The process of sending stdout/stderr to a file is called redirection - more details on all the options are contained in the Redirections page of the manual. Reading manual pages is a great way to develop and reinforce understanding - even if it doesn't currently make sense, it will help things to click faster.)
as user from the CLI/Terminal type the following commands and paste the outputs here.
1. df -Th
2. ls -laF /media/alex/Elements
3. touch /media/alex/Elements/mon/foo.txt
I did what you instructed, below is the output of the 2 actions. The 3rd one - foo.txt was created but is empty.
I wish I could understand what are the results of those commands, hope you can help me understand
Thanks again
Alex
Code:
lex@alex-NUC8i7HNK:~$ df -Th
Filesystem ***************Type *****Size *Used Avail Use% Mounted on
udev *********************devtmpfs *7.8G ****0 *7.8G **0% /dev
tmpfs ********************tmpfs ****1.6G *2.1M *1.6G **1% /run
/dev/mapper/vgubuntu-root ext4 *****457G **76G *358G *18% /
tmpfs ********************tmpfs ****7.8G **14M *7.8G **1% /dev/shm
tmpfs ********************tmpfs ****5.0M *4.0K *5.0M **1% /run/lock
tmpfs ********************tmpfs ****7.8G ****0 *7.8G **0% /sys/fs/cgroup
/dev/loop1 ***************squashfs **55M **55M ****0 100% /snap/core18/1705
/dev/loop2 ***************squashfs **56M **56M ****0 100% /snap/core18/1885
/dev/loop3 ***************squashfs *241M *241M ****0 100% /snap/gnome-3-34-1804/24
/dev/loop5 ***************squashfs *256M *256M ****0 100% /snap/gnome-3-34-1804/36
/dev/loop4 ***************squashfs **97M **97M ****0 100% /snap/core/9804
/dev/loop6 ***************squashfs **63M **63M ****0 100% /snap/gtk-common-themes/1506
/dev/loop7 ***************squashfs **50M **50M ****0 100% /snap/snap-store/433
/dev/loop0 ***************squashfs *9.2M *9.2M ****0 100% /snap/canonical-livepatch/95
/dev/loop8 ***************squashfs **83M **83M ****0 100% /snap/scrcpy/254
/dev/loop9 ***************squashfs **31M **31M ****0 100% /snap/snapd/9279
/dev/loop10 **************squashfs **50M **50M ****0 100% /snap/snap-store/467
/dev/loop11 **************squashfs **20M **20M ****0 100% /snap/video-downloader/576
/dev/loop12 **************squashfs **30M **30M ****0 100% /snap/snapd/8790
/dev/loop13 **************squashfs **20M **20M ****0 100% /snap/video-downloader/580
/dev/sda1 ****************vfat *****511M *7.8M *504M **2% /boot/efi
tmpfs ********************tmpfs ****1.6G **24K *1.6G **1% /run/user/1000
/dev/sdb1 ****************fuseblk **2.8T *1.3T *1.6T *45% /media/alex/Elements
/dev/sde1 ****************exfat ****477G **82G *396G *18% /media/alex/New-SD-512
alex@alex-NUC8i7HNK:~$
alex@alex-NUC8i7HNK:~$ ls -laF /media/alex/Elements
total 120
drwxrwxrwx *1 alex alex *8192 Sep 13 16:48 *./
drwxr-x---+ 4 root root *4096 Sep 16 06:37 *../
drwxrwxrwx *1 alex alex ****0 Aug 28 00:11 '$RECYCLE.BIN'/
drwxrwxrwx *1 alex alex *4096 Sep 12 18:50 *A-DT_Ubuntu/
drwxrwxrwx *1 alex alex 45056 Aug 24 16:53 *A-FS_Backup/
drwxrwxrwx *1 alex alex *4096 Sep *7 16:24 *alexfolders/
drwxrwxrwx *1 alex alex *8192 Sep *7 22:23 *DirSync/
drwxrwxrwx *1 alex alex *4096 Sep *8 07:07 *fri/
drwxrwxrwx *1 alex alex *4096 Sep 15 08:46 *mon/
drwxrwxrwx *1 alex alex ****0 Aug 24 17:02 *Old-Things/
drwxrwxrwx *1 alex alex *4096 Sep *9 13:12 *OurVilla/
drwxrwxrwx *1 alex alex ****0 Sep 12 17:23 *sat/
drwxrwxrwx *1 alex alex *4096 Sep 13 06:47 *sun/
drwxrwxrwx *1 alex alex *4096 Aug 24 06:02 'System Volume Information'/
drwxrwxrwx *1 alex alex *4096 Sep 10 10:00 *Targz_Files/
drwxrwxrwx *1 alex alex *8192 Sep *9 16:54 *thu/
drwxrwxrwx *1 alex alex ****0 Aug 30 11:56 *.Trash-1000/
drwxrwxrwx *1 alex alex *8192 Sep 16 06:38 *tue/
drwxrwxrwx *1 alex alex *4096 Sep *7 16:21 *W-Desktop/
drwxrwxrwx *1 alex alex *4096 Sep 16 08:56 *wed/
drwxrwxrwx *1 alex alex ****0 Jun *4 09:36 *WindowsImageBackup/
alex@alex-NUC8i7HNK:~$
Commands have two places where output can go - "standard output" (stdout) and "standard error" (stderr) - the latter is usually used for error messages, which are helpful when things are not working as expected.
The ">>" in your command appends the stdout of rsync to the file specified, but discards the stderr.
To append both stdout and stderr to your log file, use "&>>" instead of ">>".
(The process of sending stdout/stderr to a file is called redirection - more details on all the options are contained in the Redirections page of the manual. Reading manual pages is a great way to develop and reinforce understanding - even if it doesn't currently make sense, it will help things to click faster.)
Sounds like a good idea, I will change the script before the next run.
Yes, I have a long way to go, thanks for pointing me to the manual.
The third line of my previous post describes how to append rsync's stderr into your existing "/tmp/rsync.log" file, which will be in the "/tmp" directory, (and can, if desired, be sent to another directory by changing the path appropriately).
Ok all of that looks good, what happened when you performed #3, the touch command? Any errors?
Here is my backup code, feel free to modify as you see fit:
Code:
#!/bin/bash
#
###########################################################
# Created by ME January 11, 2015
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 or version 3 of the
# license, at your option.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#############################################################
#
#
#############################################################
#
# Backup program for laptops to Linux server using
# rsync -aviS
#
#############################################################
# Veriables
RUSER=user
RHOST=xxx.xxx.xxx.xxx
WHO=`whoami`
DOW=`date +%A`
RDIR=/exports/backup
HOMEDIR="$HOME"
dtstamp="`date +%Y-%m-%d-%H:%M `"
log=${HOMEDIR}/logs/${dtstamp}-rsync.log
###############################################################
### Check for a logs directory, if not found create one
###############################################################
[ ! -d "${HOMEDIR}/logs" ] && mkdir -p ${HOMEDIR}/logs >> /dev/null 2>&1
###############################################################
#
# Rsync preserving permissions
#
###############################################################
rsync -aviS --exclude-from=${HOMEDIR}/excludes.txt ${HOMEDIR}/ ${RUSER}@${RHOST}:${RDIR}/${WHO}/${DOW}/ >> ${log} 2>&1
### Cleaning up log files
###############################
find ${HOMEDIR}/logs/*.log -mtime +30 -exec rm '{}' \;
exit
Quote:
Originally Posted by alex4buba
I did what you instructed, below is the output of the 2 actions. The 3rd one - foo.txt was created but is empty.
I wish I could understand what are the results of those commands, hope you can help me understand
Thanks again
Alex
Code:
lex@alex-NUC8i7HNK:~$ df -Th
Filesystem ***************Type *****Size *Used Avail Use% Mounted on
udev *********************devtmpfs *7.8G ****0 *7.8G **0% /dev
tmpfs ********************tmpfs ****1.6G *2.1M *1.6G **1% /run
/dev/mapper/vgubuntu-root ext4 *****457G **76G *358G *18% /
tmpfs ********************tmpfs ****7.8G **14M *7.8G **1% /dev/shm
tmpfs ********************tmpfs ****5.0M *4.0K *5.0M **1% /run/lock
tmpfs ********************tmpfs ****7.8G ****0 *7.8G **0% /sys/fs/cgroup
/dev/loop1 ***************squashfs **55M **55M ****0 100% /snap/core18/1705
/dev/loop2 ***************squashfs **56M **56M ****0 100% /snap/core18/1885
/dev/loop3 ***************squashfs *241M *241M ****0 100% /snap/gnome-3-34-1804/24
/dev/loop5 ***************squashfs *256M *256M ****0 100% /snap/gnome-3-34-1804/36
/dev/loop4 ***************squashfs **97M **97M ****0 100% /snap/core/9804
/dev/loop6 ***************squashfs **63M **63M ****0 100% /snap/gtk-common-themes/1506
/dev/loop7 ***************squashfs **50M **50M ****0 100% /snap/snap-store/433
/dev/loop0 ***************squashfs *9.2M *9.2M ****0 100% /snap/canonical-livepatch/95
/dev/loop8 ***************squashfs **83M **83M ****0 100% /snap/scrcpy/254
/dev/loop9 ***************squashfs **31M **31M ****0 100% /snap/snapd/9279
/dev/loop10 **************squashfs **50M **50M ****0 100% /snap/snap-store/467
/dev/loop11 **************squashfs **20M **20M ****0 100% /snap/video-downloader/576
/dev/loop12 **************squashfs **30M **30M ****0 100% /snap/snapd/8790
/dev/loop13 **************squashfs **20M **20M ****0 100% /snap/video-downloader/580
/dev/sda1 ****************vfat *****511M *7.8M *504M **2% /boot/efi
tmpfs ********************tmpfs ****1.6G **24K *1.6G **1% /run/user/1000
/dev/sdb1 ****************fuseblk **2.8T *1.3T *1.6T *45% /media/alex/Elements
/dev/sde1 ****************exfat ****477G **82G *396G *18% /media/alex/New-SD-512
alex@alex-NUC8i7HNK:~$
alex@alex-NUC8i7HNK:~$ ls -laF /media/alex/Elements
total 120
drwxrwxrwx *1 alex alex *8192 Sep 13 16:48 *./
drwxr-x---+ 4 root root *4096 Sep 16 06:37 *../
drwxrwxrwx *1 alex alex ****0 Aug 28 00:11 '$RECYCLE.BIN'/
drwxrwxrwx *1 alex alex *4096 Sep 12 18:50 *A-DT_Ubuntu/
drwxrwxrwx *1 alex alex 45056 Aug 24 16:53 *A-FS_Backup/
drwxrwxrwx *1 alex alex *4096 Sep *7 16:24 *alexfolders/
drwxrwxrwx *1 alex alex *8192 Sep *7 22:23 *DirSync/
drwxrwxrwx *1 alex alex *4096 Sep *8 07:07 *fri/
drwxrwxrwx *1 alex alex *4096 Sep 15 08:46 *mon/
drwxrwxrwx *1 alex alex ****0 Aug 24 17:02 *Old-Things/
drwxrwxrwx *1 alex alex *4096 Sep *9 13:12 *OurVilla/
drwxrwxrwx *1 alex alex ****0 Sep 12 17:23 *sat/
drwxrwxrwx *1 alex alex *4096 Sep 13 06:47 *sun/
drwxrwxrwx *1 alex alex *4096 Aug 24 06:02 'System Volume Information'/
drwxrwxrwx *1 alex alex *4096 Sep 10 10:00 *Targz_Files/
drwxrwxrwx *1 alex alex *8192 Sep *9 16:54 *thu/
drwxrwxrwx *1 alex alex ****0 Aug 30 11:56 *.Trash-1000/
drwxrwxrwx *1 alex alex *8192 Sep 16 06:38 *tue/
drwxrwxrwx *1 alex alex *4096 Sep *7 16:21 *W-Desktop/
drwxrwxrwx *1 alex alex *4096 Sep 16 08:56 *wed/
drwxrwxrwx *1 alex alex ****0 Jun *4 09:36 *WindowsImageBackup/
alex@alex-NUC8i7HNK:~$
command >file1 2>file2
you specify [the name of] file1 and file2 and they will contain the stdout and stderr output of the command. command is the full command line (without >, >> or similar).
command >file1 2>file2
you specify [the name of] file1 and file2 and they will contain the stdout and stderr output of the command. command is the full command line (without >, >> or similar).
Sory, I am new here, not clear on your instructions
A full example, based on my current code will really help
command >file1 2>file2
you specify [the name of] file1 and file2 and they will contain the stdout and stderr output of the command. command is the full command line (without >, >> or similar).
Quote:
Originally Posted by alex4buba
Sory, I am new here, not clear on your instructions
A full example, based on my current code will really help
Emphasis added.
I don't see how that could be any clearer...you have to do some working at learning.
Emphasis added.
I don't see how that could be any clearer...you have to do some working at learning.
OK, Let me ask you somethings (no need to answer)
1) How long have you been using Linux? Me, just over 2 months
2) How old are you? I am 75
If someone is asking again, it means that they didn't get it - so, there are two options:
1) You "Smile" in your mind and this they are funny, OR
2) You understand that your initial hint was not enough, and if you have time - you give them a full example
I know, it is very hard for veterans, to get down into the minds of newbies...
It's not about being a newbie, it's about you making some effort to understand what you're being told.
We shouldn't have to spoon feed you.
command means your rsync command
>file1 means the file to capture the stdout
2>file2 means the file to capture the stderr
I don't see how that could be put any clearer, unless we simply type your commands for you. You'll learn better if you figure it out yourself.
Remember that the /full/path/to/file1 (the "absolute path") needs to be specified because you're working on a cron job.
We are of a similar age. Yes, I have been programming computers for a very long time (> 40 years), but I've always avoided calling myself an "expert," even when I was the resident expert.
The way to learn how to use Linux (or anything) is to learn how to look up how.
On Linux, that's the man pages, which are available online if you aren't comfortable reading them in a terminal. Simply put man command into your favorite search engine and you'll get several links to online copies of the documentation.
You need to answer the specific questions you've been asked. Show us the ls -l of the source and destination files that you think should be the same but aren't.
You should be getting a summary, either in the log files you're creating or in the emails being sent to root.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
