you might try something a bit more complicated, but in the end it is a simple command:

you can always manually create the log directory, then just run the rsync command without the variables:

mkdir -p /home/user/logs
rsync -aviS /source/ /destination/ >> /home/user/logs/foo.log

Then not only can you verify what files are missing using vi, or your favorite editor, but you can monitor the rsync as well.

-a = archiving as mentioned above
-v = verbose, this will show you everything that rsync is doing.
-i = itemize-changes output a change-summary for all updates
-S = Try to handle sparse files efficiently so they take up less space on the destination. Conflicts
with --inplace because it’s not possible to overwrite data in a sparse fashion.

NOTE: Don’t use this option when the destination is a Solaris “tmpfs” filesystem. It doesn’t seem
to handle seeks over null regions correctly and ends up corrupting the files.

This is my choice for both local to local and remote rsync. I have found it very fast and effective. in addition when you combine the output into the log file it is a very powerful tool to find what files failed to transfer if any and why.

You mean make a bash file and the above are the contents, right?
 

You mean make a bash file and the above are the contents, right?

yes, or you can do the job manually as well with just the rsync -aviS /source/ /destination/ >> foo.log

Umm ... what!? That is not how you use tar. It is not "SOURCE TARGET" it is "ARCHIVE FILES". In addition you are missing the creation and file switches and one of the '-', of the front of the format switch. The entire method of using tar does not change just because you use --format, it is just an optional, extra switch. What you wanted was:

By using it differently (incorrectly) the letters in what you call "SOURCE" were probably interpreted as switches, causing your issues. I am glad you don't blame me as I can't possibly see how you could interpret any difficulties as my fault. I was just offering a quick thought (tip), you still have the responsibility to read the documentation (e.g. the man or info page) to understand how tar is supposed to work.

As a side note I only mentioned --format as a minor thing to consider. It was/is unlikely to be the source of the problems. Debian-based distros (which it appears you are using) tend to default to the gnu format in any case, others such as openSUSE default to pax. I mentioned --format on the off chance you were using something more obscure that set it to something else (e.g. ustar).

rsync -aviS still a similar error
 

Thanks for the new options, but still I run into a similiar error:

a@SAM:/media/a/AC/bckup$ sudo rsync -aviS /AC/ /media/a/AC/2013 01 01/ >>foo.log
bash: foo.log: Input/output error
a@SAM:/media/a/AC/bckup$

do me a favor, try to touch a file in the directory that you are in before you run the rysnc command. do you have write permissions?

also is the destination a local, a mounted drive, or an external system, if external, what is the OS?

The Source directory is /AC, that is I put the folder with all my personal information in the root directory "/"'s folder "AC". Concerning write permissions, I open a file in /AC/Career/Analysis_Salary.xlsx and make changes, it even shows up in the modified column (pls see attached picture), thus I seem to have permissions. The destination above is a mounted drive(/media/a/AC).

andrew, if you could perform the touch from the command line and copy/paste the results.

I just ran the exact command i gave you on my system for a small directory. now if i run the same command and try to use a destination my user, ray, does not have access to i will get the following error:

That command is trying to create a directory in the / directory, something the normal user can not do.

Progress made: sudo rsync -aviS -uE --delete /home/a/AC /media/a/AC/Recent/ >>foo.log
 

lleb,
I had no idea that it's not wise to put ones personal directories in the “/” folder. So this is why the programmers at the PCManFM file manager have the directory tree open up to /home/[username] by default and not the “/” directory. I guess I don't understand the security measures for this OS yet.
Anyway, I was able to move my personal directory “AC” over to the home/a directory, and I moved the largest folder “bckup” out of this directory. I used PCManFM to do this, and mysteriously enough when moving the folder AC it the it created an error message with 8 files' full pathnames. I checked them out and all were previously saved webpages with absurdly long filenames. After remedying this situation I then executed the sudo rsync -aviS -uE --delete /home/a/AC /media/a/AC/Recent/ >>foo.log command(external, removable, non-bootalbe HDD), and no more error messages were encountered. Finally via the “ls -R | wc -l” command I verified that the source and destination folders have an equal amount of files.
I still doubt the use of foo.log. By using this it implies that one would have to look at every single line in the file(46,939 lines). This really wastes time, and another bad thing about this is that it blocks the user from seeing the progress that terminal program is making, I mean the only thing you see with the >>foo.log is a cursor that blinks for several hours. Before taking my Backup file out of my personal file AC I did this right before going to sleep at ~11:30pm, and the next morning I am about to leave for work at 8:10am, and there it was, still blinking. Was terminal making progress, was it stuck? Who knows.
I look forward to learn how to learn scripts and bash files, but haven't got the chance yet. Does this have to involve “vi”, or will the regular text editor Leafpad that comes with Lubuntu do the job?
However, part of computers should be quality assurance, and I will conduct four more backups over the next month. If all are successful I will mark this question as solved sometime around February 15, 2013.
Thanks for your advice, and may GPL software/OS spread like ivy on a stone house.

Andrew

1. Here's a good tutorial http://rute.2038bug.com/index.html.gz

2. vim is the most common editor, but there are others. (FYI, the vi cmd is often symlinked to vim on Linux)
I don't know Leafpad, but a quick google implies it should be fine. Just don't use a proper doc editor like MSWord or OO/LibreOffice Writer; these add formatting cmds that the cmd interpreter will fail on.

3. checking processes: you can open multiple terminals simultaneously, so for instance you can have your 'flashing cursor' on one and tail the log file in another.

4. for fire-and-forget cmds (ie non-interactive) you should be able to use this style
which uses '&' to put the cmd in the background, and 'nohup' to disconnect it from the terminal session, enabling you to log off and it will continue running just fine.

HTH

[Solved]
 

2013 02 13
lleb,
I have completed the four follow-up backups over the past 4 weeks successfully. Screen shots verifying matching quantity of files attached(for past 3 weeks).

Thanks so much for your help!

Andrew

glad you were successful andrew.comly. also as noted above for monitoring the log file just tail -f foo.log in an other terminal window and all will be good.

rsync is very powerful.

also as mentioned you DO NOT have to use vi, but i prefer it for one simple reason. 99.9% of all Linux distros will have it installed as part of their base system. this means I do not have to install anything on a customers system in order to be comfortable editing any file. unlike say nano or pico or any other editor out there that may or may not be on the computer. that is not to say that i dont enjoy the power those other editors have over vi, but meh vi is most likely going to be on the system so i chose to learn it.

Yeah, vi is the default on *nix generally, so not just Linux (any flavour), but also all the *BSDs, Solaris, HP-UX, AIX etc (even OS/X I think).