Why does my data occupy 3x as much space after being copied?
Linux - GeneralThis Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Why does my data occupy 3x as much space after being copied?
Look at the following console output:
Alfa applejuice # ll -h
insgesamt 2,8G
-rw-r--r-- 1 fuchs users 793M 19. Jun 14:22 10.data
-rw-r--r-- 1 fuchs users 14K 19. Jun 14:22 10.part
-rw-r--r-- 1 fuchs users 583M 19. Jun 16:32 12.data
-rw-r--r-- 1 fuchs users 11K 19. Jun 16:32 12.part
-rw-r--r-- 1 fuchs users 736M 19. Jun 16:31 14.data
-rw-r--r-- 1 fuchs users 13K 19. Jun 16:31 14.part
-rw-r--r-- 1 fuchs users 677M 19. Jun 16:31 1.data
-rw-r--r-- 1 fuchs users 12K 19. Jun 16:31 1.part
-rw-r--r-- 1 fuchs users 704M 19. Jun 16:29 2.data
-rw-r--r-- 1 fuchs users 13K 19. Jun 16:29 2.part
-rw-r--r-- 1 fuchs users 684M 19. Jun 16:31 3.data
-rw-r--r-- 1 fuchs users 12K 19. Jun 16:31 3.part
-rw-r--r-- 1 fuchs users 468M 19. Jun 16:32 4.data
-rw-r--r-- 1 fuchs users 8,1K 19. Jun 16:32 4.part
-rw-r--r-- 1 fuchs users 793M 19. Jun 16:27 5.data
-rw-r--r-- 1 fuchs users 14K 19. Jun 16:27 5.part
-rw-r--r-- 1 fuchs users 466M 19. Jun 16:32 6.data
-rw-r--r-- 1 fuchs users 8,0K 19. Jun 16:32 6.part
-rw-r--r-- 1 fuchs users 796M 19. Jun 16:17 7.data
-rw-r--r-- 1 fuchs users 14K 19. Jun 16:17 7.part
-rw-r--r-- 1 fuchs users 464M 19. Jun 16:28 8.data
-rw-r--r-- 1 fuchs users 8,0K 19. Jun 16:28 8.part
-rw-r--r-- 1 fuchs users 755M 19. Jun 16:31 9.data
-rw-r--r-- 1 fuchs users 13K 19. Jun 16:31 9.part
Alfa applejuice #
How does it come that if you add the filesizes you get about 8 GB, but ls says it's 2,8 GB?
Normally I wouldn't worry about that, but I had to transfer the whole stuff to another harddisk (did it with "cp -Rp"), and there it really occupied 8 GB.
Once I had the problem that i cancelled the ongoing copy of a single big file because the partition was full, the file then didn't show nowhere, so I couldn't remove it, but it was occupying lots of space (saw it with "df").
Two thinks I would like to know:
- how does things with temporary files work?
- is there a way to keep the size when copying?
Try without the -h ... chances are "cumulative rounding errors"
are borking up the result.
I'm not sure the output explains the problem you have
with copying?
Quote:
- how does things with temporary files work?
- is there a way to keep the size when copying?
I don't understand the first question.
As for "changing file sizes" - that should only be possible
if the file-systems are different, but even then it shouldn't
make too much difference with files that big.
You mean the -h creates a rounding error of >200% ?
LOL, if it was that I would instantly switch OS.
This aren't trivial displaying errors. Try to add the single file sizes and then compare with what it displays as overall size (insgesamt..).
There are more than 5GB of difference between that two. (yes, without -h the difference remains the same, just dispayed as bytes...)
What it has to do with copying?
If you copy a whole partition to another (blank, same size, same filesystem), you expect to end with the same amount of occupied space, so I did.
But there where 5 GB more and I discovered (with du and ls -l) that it's just this single directory to make the difference which like mentioned above makes sense 'cos this files are partially downloaded iso's.
Originally posted by didi156
You mean the -h creates a rounding error of >200% ?
LOL, if it was that I would instantly switch OS.
I'd change the OS if copying changed the file-size...
I could live with ignoring the output in "human readable" format ;)
Quote:
This aren't trivial displaying errors. Try to add the single file sizes and then compare with what it displays as overall size (insgesamt..).
There are more than 5GB of difference between that two. (yes, without -h the difference remains the same, just dispayed as bytes...)
What it has to do with copying?
Did you actually TRY without h rather than just LOLing at me?
Quote:
If you copy a whole partition to another (blank, same size, same filesystem), you expect to end with the same amount of occupied space, so I did.
But there where 5 GB more and I discovered (with du and ls -l) that it's just this single directory to make the difference which like mentioned above makes sense 'cos this files are partially downloaded iso's.
And here (again) you didn't mention whether the file-system types
are different...
Here some console output for non-believers:
Alfa is the machine with the "source-harddisk"
Alfa applejuice # ll
insgesamt 2851068
-rw-r--r-- 1 fuchs users 811404 19. Jun 14:22 10.data
-rw-r--r-- 1 fuchs users 14 19. Jun 14:22 10.part
-rw-r--r-- 1 fuchs users 596004 19. Jun 16:32 12.data
-rw-r--r-- 1 fuchs users 11 19. Jun 16:32 12.part
-rw-r--r-- 1 fuchs users 753155 19. Jun 16:31 14.data
-rw-r--r-- 1 fuchs users 13 19. Jun 16:31 14.part
-rw-r--r-- 1 fuchs users 692509 19. Jun 16:31 1.data
-rw-r--r-- 1 fuchs users 12 19. Jun 16:31 1.part
-rw-r--r-- 1 fuchs users 720392 19. Jun 16:29 2.data
-rw-r--r-- 1 fuchs users 13 19. Jun 16:29 2.part
-rw-r--r-- 1 fuchs users 700182 19. Jun 16:31 3.data
-rw-r--r-- 1 fuchs users 12 19. Jun 16:31 3.part
-rw-r--r-- 1 fuchs users 479090 19. Jun 16:32 4.data
-rw-r--r-- 1 fuchs users 9 19. Jun 16:32 4.part
-rw-r--r-- 1 fuchs users 811627 19. Jun 16:27 5.data
-rw-r--r-- 1 fuchs users 14 19. Jun 16:27 5.part
-rw-r--r-- 1 fuchs users 477178 19. Jun 16:32 6.data
-rw-r--r-- 1 fuchs users 8 19. Jun 16:32 6.part
-rw-r--r-- 1 fuchs users 814910 19. Jun 16:17 7.data
-rw-r--r-- 1 fuchs users 14 19. Jun 16:17 7.part
-rw-r--r-- 1 fuchs users 474590 19. Jun 16:28 8.data
-rw-r--r-- 1 fuchs users 8 19. Jun 16:28 8.part
-rw-r--r-- 1 fuchs users 773062 19. Jun 16:31 9.data
-rw-r--r-- 1 fuchs users 13 19. Jun 16:31 9.part
Hogwart the "target machine"; the copy went over a smbfs mount.
hogwart applejuice # ll
total 8112264
-rw-r--r-- 1 dummy users 709129092 Jun 19 18:31 1.data
-rw-r--r-- 1 dummy users 11678 Jun 19 18:31 1.part
-rw-r--r-- 1 dummy users 830876928 Jun 19 16:22 10.data
-rw-r--r-- 1 dummy users 13854 Jun 19 16:22 10.part
-rw-r--r-- 1 dummy users 610308096 Jun 19 18:32 12.data
-rw-r--r-- 1 dummy users 10598 Jun 19 18:32 12.part
-rw-r--r-- 1 dummy users 771230252 Jun 21 21:16 14.data
-rw-r--r-- 1 dummy users 13270 Jun 19 18:31 14.part
-rw-r--r-- 1 dummy users 737681324 Jun 21 23:44 2.data
-rw-r--r-- 1 dummy users 12686 Jun 21 23:44 2.part
-rw-r--r-- 1 dummy users 716986076 Jun 21 23:45 3.data
-rw-r--r-- 1 dummy users 12302 Jun 21 23:45 3.part
-rw-r--r-- 1 dummy users 490587448 Jun 19 18:32 4.data
-rw-r--r-- 1 dummy users 8254 Jun 19 18:32 4.part
-rw-r--r-- 1 dummy users 831105072 Jun 19 18:27 5.data
-rw-r--r-- 1 dummy users 13742 Jun 19 18:27 5.part
-rw-r--r-- 1 dummy users 488630046 Jun 19 18:32 6.data
-rw-r--r-- 1 dummy users 8174 Jun 19 18:32 6.part
-rw-r--r-- 1 dummy users 834467060 Jun 19 18:17 7.data
-rw-r--r-- 1 dummy users 13718 Jun 19 18:17 7.part
-rw-r--r-- 1 dummy users 485979984 Jun 19 18:28 8.data
-rw-r--r-- 1 dummy users 8158 Jun 19 18:28 8.part
-rw-r--r-- 1 dummy users 791614824 Jun 19 18:31 9.data
-rw-r--r-- 1 dummy users 12790 Jun 19 18:31 9.part
And du says the same:
Alfa tmp # du applejuice/
2851072 applejuice/
hogwart tmp # du applejuice/
8112268 applejuice
This directory was just a small percent and is the only not matching in size, which makes obvious for me that it has to do with the fact that this are temporary files not filled entirely yet.
I just did some tests and discovered it was samba to create the size differences. Copying it locally didn't change nothing, but from an smb-mount it did... strange however.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.