LinuxQuestions.org
Visit Jeremy's Blog.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 07-16-2013, 09:51 PM   #1
vjramana
Member
 
Registered: Sep 2009
Posts: 89

Rep: Reputation: 0
Folder and file permissions in back-up not the same as in the source


I am using ubuntu 11.10. I have several files and folders in a particular folder. The list of files and folders in that folder is shown below.

I have backed-up all these folders and files into external hard-drive. I used rsync command to do the back up. The command I used is 'rsync -av --progress --size-only [source] [destination].

Quote:
total 56
drwxrwxr-x 2 vijay vijay 4096 2013-06-26 15:09 dyna-21-chain-dynamics/
drwxrwxr-x 2 vijay vijay 4096 2013-06-26 15:09 dyna-22-head-dynamics/
drwxrwxr-x 2 vijay vijay 4096 2013-06-26 15:10 dyna-23-exocyclic-rotation/
drwxrwxr-x 2 vijay vijay 4096 2013-04-29 11:27 dyna-24-lipid-diffusion/
drwxrwxr-x 2 vijay vijay 4096 2013-04-29 11:27 dyna-25-water-diffusion/
drwxrwxr-x 2 vijay vijay 4096 2013-06-26 15:11 dyna-26-water-redident-time/
-rwxrwxr-x 1 vijay vijay 86 2013-07-11 13:27 make-directory.sh*
drwxrwxr-x 6 vijay vijay 4096 2013-06-28 15:11 struc-01-local_density_profile/
drwxrwxr-x 2 vijay vijay 4096 2013-05-02 10:36 struc-02-area_per_lipid/
drwxrwxr-x 7 vijay vijay 4096 2013-07-03 16:16 struc-03-hydrogen_bond/
drwxrwxr-x 6 vijay vijay 4096 2013-07-11 13:27 struc-04-order_parameter/
drwxrwxr-x 2 vijay vijay 4096 2013-04-20 20:49 struc-05-head_tilt_angle/
drwxrwxr-x 2 vijay vijay 4096 2013-04-20 20:49 struc-06-tail_tilt_angle/
drwxrwxr-x 2 vijay vijay 4096 2013-04-20 20:52 struc-07-Raman-plot/
Below is the list of the files and folders copied into external hard drive.

Quote:
total 9
drwx------ 1 vijay vijay 0 2013-06-26 15:09 dyna-21-chain-dynamics/
drwx------ 1 vijay vijay 0 2013-06-26 15:09 dyna-22-head-dynamics/
drwx------ 1 vijay vijay 0 2013-06-26 15:10 dyna-23-exocyclic-rotation/
drwx------ 1 vijay vijay 0 2013-04-29 11:27 dyna-24-lipid-diffusion/
drwx------ 1 vijay vijay 0 2013-04-29 11:27 dyna-25-water-diffusion/
drwx------ 1 vijay vijay 0 2013-06-26 15:11 dyna-26-water-redident-time/
-rw------- 1 vijay vijay 86 2013-07-11 13:27 make-directory.sh
drwx------ 1 vijay vijay 0 2013-06-28 15:11 struc-01-local_density_profile/
drwx------ 1 vijay vijay 4096 2013-05-02 10:36 struc-02-area_per_lipid/
drwx------ 1 vijay vijay 4096 2013-07-03 16:16 struc-03-hydrogen_bond/
drwx------ 1 vijay vijay 0 2013-07-11 13:27 struc-04-order_parameter/
drwx------ 1 vijay vijay 48 2013-07-17 10:00 struc-05-head_tilt_angle/
drwx------ 1 vijay vijay 0 2013-04-20 20:49 struc-06-tail_tilt_angle/
drwx------ 1 vijay vijay 0 2013-04-20 20:52 struc-07-Raman-plot/
The information on each file and folders that I copied into external hard drive is not the same. The first column, second column, and fifth column seems do not preserve all the attributes corresponding to each files. For example in fifth column I can see some folders have value 4096 and 48. This is not consistent with the source folders.

My question: Is this normal in doing files and folders back-up? Or, is there something wrong the way I transferred the data. Or something else.

I don't want to loose any data from my external hard drive. Could anyone explain what actually getting wrong here?

Many thanks in advance.
 
Old 07-16-2013, 10:19 PM   #2
lleb
Senior Member
 
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora/Pop!_OS
Posts: 2,983

Rep: Reputation: 551Reputation: 551Reputation: 551Reputation: 551Reputation: 551Reputation: 551
well from the man page:

Quote:
-a, --archive archive mode; equals -rlptgoD (no -H,-A,-X)
-r, --recursive recurse into directories
-l, --links copy symlinks as symlinks
-p, --perms preserve permissions
-g, --group preserve group
-o, --owner preserve owner (super-user only)
-D same as --devices --specials

-ignore-existing skip updating files that exist on receiver
-S, --sparse
Try to handle sparse files efficiently so they take up less space on the destination. Conflicts with
--inplace because it’s not possible to overwrite data in a sparse fashion.

NOTE: Don’t use this option when the destination is a Solaris “tmpfs” filesystem. It doesn’t seem to han-
dle seeks over null regions correctly and ends up corrupting the files.
did you run rsync as user or root, or as a different user then on your directory?

i typically will run rsync -aviS source destination, but that is just me. when i do that as user, i never have permissions issues as you did above.
 
1 members found this post helpful.
Old 07-16-2013, 10:24 PM   #3
evo2
LQ Guru
 
Registered: Jan 2009
Location: Japan
Distribution: Mostly Debian and CentOS
Posts: 6,724

Rep: Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705
Hi,

using '-a' should mean that you get the '-p' behaviour and hence preserve permissions, but we can see that the group and other permissions are not preserved. The fifth column should be the size of the files. So the fact the the directories show "0" is quite strange. Have you actually checked the files?

What file systems are you using?

Also you may find it useful to run rsync with a high level of verbosity and check what is written to the terminal.

Another point is, that I think you should probably not use the --size-only option as it would cause rsync to skip modified files if their size is unchanged.

Evo2.
 
1 members found this post helpful.
Old 07-16-2013, 10:26 PM   #4
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian sid
Posts: 2,683

Rep: Reputation: 783Reputation: 783Reputation: 783Reputation: 783Reputation: 783Reputation: 783Reputation: 783
What filesystem is the external drive?
 
Old 07-16-2013, 10:34 PM   #5
vjramana
Member
 
Registered: Sep 2009
Posts: 89

Original Poster
Rep: Reputation: 0
Dear evo2,

When I check the contents of the folders and files, they are present as in the source files and folders. But their attributions are also not consistent with the source files.

In Ubuntu I am using ext4 file system. But my external hard drive file system is NTFS.

What is your advice on this could be?

Regards
 
Old 07-16-2013, 10:35 PM   #6
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian sid
Posts: 2,683

Rep: Reputation: 783Reputation: 783Reputation: 783Reputation: 783Reputation: 783Reputation: 783Reputation: 783
ntfs won't store all the file 'bits',
either format ext4, or use tar
 
Old 07-16-2013, 10:36 PM   #7
vjramana
Member
 
Registered: Sep 2009
Posts: 89

Original Poster
Rep: Reputation: 0
Dear Firerat,

My external hard drive file system is NTFS.

At times, I may plugin my external hard drive into Windows7 machine.

Thank you


Regards.
 
Old 07-16-2013, 10:40 PM   #8
vjramana
Member
 
Registered: Sep 2009
Posts: 89

Original Poster
Rep: Reputation: 0
Dear Firerat,

In the case of using 'tar', can I use compress option in rsync? Is that same or similar like using 'tar'?

Rgds
 
Old 07-16-2013, 11:04 PM   #9
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian sid
Posts: 2,683

Rep: Reputation: 783Reputation: 783Reputation: 783Reputation: 783Reputation: 783Reputation: 783Reputation: 783
compression with rsync is only during transport
i.e. compress file > send to dest > uncompress

It actually depends on how important the permissions are to you,

as it stands the only really difference is you have no 'group' or 'other' bits set
You will still be able to read them
.. Sorry just noticed you lost execute on make-directory.sh


Other options are partition the drive up into two
one ntfs and one ext4

You could also make a loopback filesystem, i.e. one bigfile, 'format' it ext4 and mount with loop option.

end of the day it is probably easier to use tar

Code:
tar -c ./ -f /path/to/backup.tar
optionally use compression ( see man tar for details )

But as I said earlier, it really depends on what these files are used for.
 
1 members found this post helpful.
Old 07-16-2013, 11:09 PM   #10
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian sid
Posts: 2,683

Rep: Reputation: 783Reputation: 783Reputation: 783Reputation: 783Reputation: 783Reputation: 783Reputation: 783
Just to clarify, the files themselves will be the 'same' it is just that extra information normally stored by the ext4 filesystem will be missing
 
Old 07-16-2013, 11:48 PM   #11
evo2
LQ Guru
 
Registered: Jan 2009
Location: Japan
Distribution: Mostly Debian and CentOS
Posts: 6,724

Rep: Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705Reputation: 1705
Hi,
Quote:
Originally Posted by vjramana View Post
Dear evo2,
When I check the contents of the folders and files, they are present as in the source files and folders. But their attributions are also not consistent with the source files.
OK.
Quote:
In Ubuntu I am using ext4 file system. But my external hard drive file system is NTFS.

What is your advice on this could be?
Avoid NTFS (and similar deficient filesystems) like the plague.

Cheers,

Evo2.
 
Old 07-17-2013, 05:29 PM   #12
lleb
Senior Member
 
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora/Pop!_OS
Posts: 2,983

Rep: Reputation: 551Reputation: 551Reputation: 551Reputation: 551Reputation: 551Reputation: 551
Sadly NTFS will cause these issues. Not much to be done about it other then not use it. Either split the drive or just deal with the permissions issue caused by the lack of control provided by NTFS
 
  


Reply

Tags
backup, copy, distribution, ubuntu 11.10



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Folder and file permissions DaftDave Linux - General 2 06-29-2008 10:37 AM
File/folder permissions Yig Linux - Newbie 3 11-28-2005 03:44 PM
File and folder permissions steve007 Linux - Newbie 3 06-30-2005 03:48 AM
File and Folder permissions seelk SUSE / openSUSE 9 01-28-2005 06:50 PM
File / folder Permissions bkbroil Linux - Newbie 3 10-15-2003 10:37 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 06:12 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration