Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game. |
Notices |
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
Are you new to LinuxQuestions.org? Visit the following links:
Site Howto |
Site FAQ |
Sitemap |
Register Now
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
|
|
09-14-2013, 12:13 PM
|
#1
|
Member
Registered: Dec 2010
Location: iran
Posts: 199
Rep:
|
File size increase after copy(scp,rsync)
Hello,
i was trying backup from my virtual machine (openxen) using rsync , after backup i used du -sh at destination and source , unbelievably amount of disk usage at destination was double time of amount of data at source, i checked , for duplication data , nothing found ,here is my situation
source :
linux lenny , ext3 ,
destination:
linux os , ext4 (tested with ext3)
command :
rsync -a -v --delete -e ssh /mbox root@IP:/mbox
also used
scp -r /mbox root@IP:/mbox
i used ::: du -sh ::: df -h ::: for disk usage
Any Idea....
Thanks
Last edited by hesisaboury; 09-14-2013 at 12:14 PM.
|
|
|
09-14-2013, 12:32 PM
|
#2
|
Senior Member
Registered: Aug 2009
Distribution: Rocky Linux
Posts: 4,801
|
Without more detail I can only guess, but perhaps the source had one or more sparse files. By default, rsync does not try to preserve sparseness (use the "-S" or "--sparse" option to do that). I don't see any option in scp to preserve sparseness.
|
|
|
09-15-2013, 12:56 PM
|
#3
|
Member
Registered: Dec 2010
Location: iran
Posts: 199
Original Poster
Rep:
|
Hi,
What details do u need rknichols?
i dont know what is going on , may be linux bug , the where i'm working used linux lenny 5.0.4 for most their projects, when i check some servers i see "du -sh" result has different value from"df -h" , it makes a big problem,
Last edited by hesisaboury; 09-15-2013 at 12:59 PM.
|
|
|
09-15-2013, 04:37 PM
|
#4
|
Senior Member
Registered: Aug 2009
Distribution: Rocky Linux
Posts: 4,801
|
Running du and df on a filesystem will very often yield different results. The du command walks through the directory tree and reports/sums the sizes of what it finds. The df command directly queries the filesystem and reports the number of blocks in use. Why would these be different? The most common cause is files that have been deleted from the directory tree but are still held open by some process(es). That would cause df to report higher usage than what du can see.
The other common cause is files that exist on the filesystem but are hidden under an active mount point. Let's say you decide to move /home from the root filesystem to a separate partition. You copy the current contents of /home to the new partition, and then mount the new partition on /home. All looks well, and /home now has lots of space from the new, presumably larger, partition. The catch is that you never deleted the files from the root filesystem. They still occupy space there, but "du --one-file-system" won't see them on the root partition while the new /home is mounted there.
I don't know whether either of these applies to your situation. Initially you reported different results for "du -sh" at the source and destination, which suggests a completely different issue. You would need to look at the output from "du --max-depth=N", with gradually increasing values for N, to try to see where the differences occur.
|
|
|
09-15-2013, 08:45 PM
|
#5
|
Moderator
Registered: Mar 2008
Posts: 22,229
|
Post the exact command you used and include where you ran it from.
|
|
|
09-29-2013, 11:53 AM
|
#6
|
Member
Registered: Sep 2012
Distribution: RHEL5/6, CentOS5/6
Posts: 218
Rep:
|
Quote:
Originally Posted by man du
The du utility displays the file system block usage for each file argument and for each directory in the file hierarchy rooted in each directory argument.
|
Du you have same file systems, block size etc on all your systems involved?
Block size usage and file size are not exactly the same. If you have a small text file with e.g. 20 bytes, it would still use a full block (e.g. 4096 bytes).
|
|
|
09-29-2013, 02:36 PM
|
#7
|
LQ Guru
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,573
|
Quote:
Originally Posted by dt64
Du you have same file systems, block size etc on all your systems involved?
Block size usage and file size are not exactly the same. If you have a small text file with e.g. 20 bytes, it would still use a full block (e.g. 4096 bytes).
|
^ this
And I noticed you didn't have -H set in your rsync command, which means any hard links on the source will be duplicated on the destination, resulting in double, triple, quadruple, etc. the disk usage (depending on how many hard links you had on the source).
Last edited by suicidaleggroll; 09-29-2013 at 02:38 PM.
|
|
|
09-30-2013, 01:49 AM
|
#8
|
Member
Registered: Dec 2010
Location: iran
Posts: 199
Original Poster
Rep:
|
Hi,
source location is /mbox and it has only mail in it's directories , there is no hard link there , the strange thing is the size of folder inside and outside of that , different size , for example :
du -sh folder == 60M
cd folder
du -dh * ==> 40M
Thanks suicidaleggroll
|
|
|
09-30-2013, 09:04 AM
|
#9
|
Senior Member
Registered: Aug 2009
Distribution: Rocky Linux
Posts: 4,801
|
There are 2 possibilities. Either (a) the directory file itself is huge (unlikely, unless you have tens of thousands of users), or (b) there are dotfiles in that directory (the expansion of "*" will not include any filenames that begin with ".").
|
|
|
All times are GMT -5. The time now is 09:50 PM.
|
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
|
Latest Threads
LQ News
|
|