Latest LQ Deal: Linux Power User Bundle
Go Back > Forums > Linux Forums > Linux - Newbie
User Name
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!


  Search this Thread
Old 04-29-2011, 07:12 AM   #1
LQ Newbie
Registered: Aug 2006
Posts: 12

Rep: Reputation: 0
Help with Rsync and DU commands output to file and mail

I have a simple task I run to copy via rsync directories from one machine to another. The command I run for each machine/directory is this:
rsync -vvaz -e ssh root@DS1:/var/log/ /Network/DS/var/log > /Network/DS1/log_1_Monday.txt


rsync -vvaz --progress -e ssh root@DS2:/var/etc/ /Network/DS2/var/etc
Afterwards I have this run:
du -h --max-depth=1 /Network/DS1 | tail -n 5 | sort -n > /Network/DS_log.txt

I put them all together and added this to CRON (via Webmin) and it works perfectly. Each night the backup is made, and in the morning a new log is there for the rsync and du output.

Well since I am in the "Linux - Newbie" forum, I suppose you can guess that I have pretty much reached the limits here. As it is I have done a lot of Google searches to get to where I am at right now, but alas Google may be at the end of its ropes in terms of helping me.

My first problem is when I issue a series of commands like:

du -h --max-depth=1 /Network/DS1 | tail -n 5 | sort -n > /Network/DS_log.txt
du -h --max-depth=1 /Network/DS2 | tail -n 5 | sort -n >> /Network/DS_log.txt
du -h --max-depth=1 /Network/DS3 | tail -n 5 | sort -n >> /Network/DS_log.txt

I would like to have each individual output spaced, or separated. So DS1 creates the file DS_log.txt, then DS2's output just adds, ditto DS3. Is there a way to possibly put some output of a command in between to separate in the file?

du -h --max-depth=1 /Network/DS1 | tail -n 5 | sort -n > /Network/DS_log.txt
du -h --max-depth=1 /Network/DS2 | tail -n 5 | sort -n >> /Network/DS_log.txt
du -h --max-depth=1 /Network/DS3 | tail -n 5 | sort -n >> /Network/DS_log.txt

Eventually I would like to get the rsync and disk usage commands mailed to me (I work remotely so that it would a real nice addition). I believe this is how I would do it:

du -h --max-depth=1 /Network/DS1 | tail -n 5 | sort -n > /Network/DS_log.txt
cat /Network/DS_log.txt | mail -s"Directory Size"

So if I add the "cat /..." at the end of the file it will simply mail that out. OK, now here is my second issue. With the rsync I have 3 machines with around 4 to 5 directories each that I transfer. I want to get the total output (log) of the rsync command sent to me. Problem is if I have each individual rsync command logged to a single file I find that while the simple newb script (if you can call it that) is finished, the file itself is basically empty. If I slowly run each rsync command at a time and give the time to have the file output written then fine. Good lord I hope I am making sense here. Should I have added a warning that this post will probably cause headaches, or fits of laughter ?
Old 04-29-2011, 08:09 AM   #2
Senior Member
Registered: Oct 2003
Location: Northeastern Michigan, where Carhartt is a Designer Label
Distribution: Slackware 32- & 64-bit Stable
Posts: 3,528

Rep: Reputation: 1056Reputation: 1056Reputation: 1056Reputation: 1056Reputation: 1056Reputation: 1056Reputation: 1056Reputation: 1056
OK, here's something you might try.

Let's say you have a shell program to do your logging. Start it with something like this
#     Clear the existing log file
> /Network/DS_log.txt
#     Make a DU entry in the log (note the 2>&1 -- you get both standard out and standard error that way,
#     also note the >> to append to the previously-emptied log file)
du -h --max-depth=1 /Network/DS1 | tail -n 5 | sort -n 2>&1 >> /Network/DS_log.txt
#     append a blank line to separate log entries (or put a line of hyphens or whatever)
echo "" >> /Network/DS_log.txt
#     Do other stuff, like your syncs, adding the 2>&1 and >> as necessary
#     Mail the log to yourself
mail -s "Directory Size" < /Network/DS_log.txt
One thing you should probably (always) do if this thing runs from cron is use absolute paths for the executable utilities you're using; e.g., /usr/bin/du rather than just du; the reason for that is that cron doesn't have much of a PATH environment and might not be able to find things you're trying to execute. Be sure to check where your utilities are located. You can find out what cron's PATH is by inserting
echo ${PATH} >> /Network/DS_log.txt
at the top of the shell program, say right after the line that clears the existing log file.

If your rsync command works manually, it'll also work in a shell program or a cron job -- just make sure to use absolute path, 2>&1 (so you'll see any errors) and >> into the log file.

Hope this helps some.


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
Find command with standard error output and mail error file montblanc Programming 2 11-11-2010 09:06 PM
write output of two commands to a file pgb205 Programming 1 10-12-2010 01:13 PM
How to concatenate output of two commands to a file starmonche Linux - General 3 05-01-2009 11:33 AM
varied output for linux df and du commands on same file system bzlaskar Linux - Newbie 1 12-11-2007 09:41 PM
using commands to output path, filename and info to a txt file bob_man_uk Linux - General 3 05-11-2006 02:31 PM

All times are GMT -5. The time now is 07:18 PM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration