LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 06-21-2004, 02:17 PM   #1
didi156
LQ Newbie
 
Registered: Feb 2004
Distribution: Debian, Ubuntu, uCLinux, Android
Posts: 15

Rep: Reputation: 0
Why does my data occupy 3x as much space after being copied?


Look at the following console output:

Alfa applejuice # ll -h
insgesamt 2,8G
-rw-r--r-- 1 fuchs users 793M 19. Jun 14:22 10.data
-rw-r--r-- 1 fuchs users 14K 19. Jun 14:22 10.part
-rw-r--r-- 1 fuchs users 583M 19. Jun 16:32 12.data
-rw-r--r-- 1 fuchs users 11K 19. Jun 16:32 12.part
-rw-r--r-- 1 fuchs users 736M 19. Jun 16:31 14.data
-rw-r--r-- 1 fuchs users 13K 19. Jun 16:31 14.part
-rw-r--r-- 1 fuchs users 677M 19. Jun 16:31 1.data
-rw-r--r-- 1 fuchs users 12K 19. Jun 16:31 1.part
-rw-r--r-- 1 fuchs users 704M 19. Jun 16:29 2.data
-rw-r--r-- 1 fuchs users 13K 19. Jun 16:29 2.part
-rw-r--r-- 1 fuchs users 684M 19. Jun 16:31 3.data
-rw-r--r-- 1 fuchs users 12K 19. Jun 16:31 3.part
-rw-r--r-- 1 fuchs users 468M 19. Jun 16:32 4.data
-rw-r--r-- 1 fuchs users 8,1K 19. Jun 16:32 4.part
-rw-r--r-- 1 fuchs users 793M 19. Jun 16:27 5.data
-rw-r--r-- 1 fuchs users 14K 19. Jun 16:27 5.part
-rw-r--r-- 1 fuchs users 466M 19. Jun 16:32 6.data
-rw-r--r-- 1 fuchs users 8,0K 19. Jun 16:32 6.part
-rw-r--r-- 1 fuchs users 796M 19. Jun 16:17 7.data
-rw-r--r-- 1 fuchs users 14K 19. Jun 16:17 7.part
-rw-r--r-- 1 fuchs users 464M 19. Jun 16:28 8.data
-rw-r--r-- 1 fuchs users 8,0K 19. Jun 16:28 8.part
-rw-r--r-- 1 fuchs users 755M 19. Jun 16:31 9.data
-rw-r--r-- 1 fuchs users 13K 19. Jun 16:31 9.part
Alfa applejuice #

How does it come that if you add the filesizes you get about 8 GB, but ls says it's 2,8 GB?
Normally I wouldn't worry about that, but I had to transfer the whole stuff to another harddisk (did it with "cp -Rp"), and there it really occupied 8 GB.
Once I had the problem that i cancelled the ongoing copy of a single big file because the partition was full, the file then didn't show nowhere, so I couldn't remove it, but it was occupying lots of space (saw it with "df").

Two thinks I would like to know:
- how does things with temporary files work?
- is there a way to keep the size when copying?
 
Old 06-21-2004, 02:40 PM   #2
Tinkster
Moderator
 
Registered: Apr 2002
Location: earth
Distribution: slackware by choice, others too :} ... android.
Posts: 23,067
Blog Entries: 11

Rep: Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928
Try without the -h ... chances are "cumulative rounding errors"
are borking up the result.

I'm not sure the output explains the problem you have
with copying?


Quote:
- how does things with temporary files work?
- is there a way to keep the size when copying?
I don't understand the first question.
As for "changing file sizes" - that should only be possible
if the file-systems are different, but even then it shouldn't
make too much difference with files that big.



Cheers,
Tink
 
Old 06-21-2004, 04:03 PM   #3
didi156
LQ Newbie
 
Registered: Feb 2004
Distribution: Debian, Ubuntu, uCLinux, Android
Posts: 15

Original Poster
Rep: Reputation: 0
You mean the -h creates a rounding error of >200% ?
LOL, if it was that I would instantly switch OS.

This aren't trivial displaying errors. Try to add the single file sizes and then compare with what it displays as overall size (insgesamt..).
There are more than 5GB of difference between that two. (yes, without -h the difference remains the same, just dispayed as bytes...)
What it has to do with copying?
If you copy a whole partition to another (blank, same size, same filesystem), you expect to end with the same amount of occupied space, so I did.
But there where 5 GB more and I discovered (with du and ls -l) that it's just this single directory to make the difference which like mentioned above makes sense 'cos this files are partially downloaded iso's.
 
Old 06-21-2004, 04:27 PM   #4
Tinkster
Moderator
 
Registered: Apr 2002
Location: earth
Distribution: slackware by choice, others too :} ... android.
Posts: 23,067
Blog Entries: 11

Rep: Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928
Quote:
Originally posted by didi156
You mean the -h creates a rounding error of >200% ?
LOL, if it was that I would instantly switch OS.
I'd change the OS if copying changed the file-size...
I could live with ignoring the output in "human readable" format ;)


Quote:
This aren't trivial displaying errors. Try to add the single file sizes and then compare with what it displays as overall size (insgesamt..).
There are more than 5GB of difference between that two. (yes, without -h the difference remains the same, just dispayed as bytes...)
What it has to do with copying?
Did you actually TRY without h rather than just LOLing at me?

Quote:
If you copy a whole partition to another (blank, same size, same filesystem), you expect to end with the same amount of occupied space, so I did.
But there where 5 GB more and I discovered (with du and ls -l) that it's just this single directory to make the difference which like mentioned above makes sense 'cos this files are partially downloaded iso's.
And here (again) you didn't mention whether the file-system types
are different...


Bis die Naechte,
Tink
 
Old 06-21-2004, 05:01 PM   #5
didi156
LQ Newbie
 
Registered: Feb 2004
Distribution: Debian, Ubuntu, uCLinux, Android
Posts: 15

Original Poster
Rep: Reputation: 0
Filesystem -> both ext2

Here some console output for non-believers:
Alfa is the machine with the "source-harddisk"

Alfa applejuice # ll
insgesamt 2851068
-rw-r--r-- 1 fuchs users 811404 19. Jun 14:22 10.data
-rw-r--r-- 1 fuchs users 14 19. Jun 14:22 10.part
-rw-r--r-- 1 fuchs users 596004 19. Jun 16:32 12.data
-rw-r--r-- 1 fuchs users 11 19. Jun 16:32 12.part
-rw-r--r-- 1 fuchs users 753155 19. Jun 16:31 14.data
-rw-r--r-- 1 fuchs users 13 19. Jun 16:31 14.part
-rw-r--r-- 1 fuchs users 692509 19. Jun 16:31 1.data
-rw-r--r-- 1 fuchs users 12 19. Jun 16:31 1.part
-rw-r--r-- 1 fuchs users 720392 19. Jun 16:29 2.data
-rw-r--r-- 1 fuchs users 13 19. Jun 16:29 2.part
-rw-r--r-- 1 fuchs users 700182 19. Jun 16:31 3.data
-rw-r--r-- 1 fuchs users 12 19. Jun 16:31 3.part
-rw-r--r-- 1 fuchs users 479090 19. Jun 16:32 4.data
-rw-r--r-- 1 fuchs users 9 19. Jun 16:32 4.part
-rw-r--r-- 1 fuchs users 811627 19. Jun 16:27 5.data
-rw-r--r-- 1 fuchs users 14 19. Jun 16:27 5.part
-rw-r--r-- 1 fuchs users 477178 19. Jun 16:32 6.data
-rw-r--r-- 1 fuchs users 8 19. Jun 16:32 6.part
-rw-r--r-- 1 fuchs users 814910 19. Jun 16:17 7.data
-rw-r--r-- 1 fuchs users 14 19. Jun 16:17 7.part
-rw-r--r-- 1 fuchs users 474590 19. Jun 16:28 8.data
-rw-r--r-- 1 fuchs users 8 19. Jun 16:28 8.part
-rw-r--r-- 1 fuchs users 773062 19. Jun 16:31 9.data
-rw-r--r-- 1 fuchs users 13 19. Jun 16:31 9.part

Hogwart the "target machine"; the copy went over a smbfs mount.

hogwart applejuice # ll
total 8112264
-rw-r--r-- 1 dummy users 709129092 Jun 19 18:31 1.data
-rw-r--r-- 1 dummy users 11678 Jun 19 18:31 1.part
-rw-r--r-- 1 dummy users 830876928 Jun 19 16:22 10.data
-rw-r--r-- 1 dummy users 13854 Jun 19 16:22 10.part
-rw-r--r-- 1 dummy users 610308096 Jun 19 18:32 12.data
-rw-r--r-- 1 dummy users 10598 Jun 19 18:32 12.part
-rw-r--r-- 1 dummy users 771230252 Jun 21 21:16 14.data
-rw-r--r-- 1 dummy users 13270 Jun 19 18:31 14.part
-rw-r--r-- 1 dummy users 737681324 Jun 21 23:44 2.data
-rw-r--r-- 1 dummy users 12686 Jun 21 23:44 2.part
-rw-r--r-- 1 dummy users 716986076 Jun 21 23:45 3.data
-rw-r--r-- 1 dummy users 12302 Jun 21 23:45 3.part
-rw-r--r-- 1 dummy users 490587448 Jun 19 18:32 4.data
-rw-r--r-- 1 dummy users 8254 Jun 19 18:32 4.part
-rw-r--r-- 1 dummy users 831105072 Jun 19 18:27 5.data
-rw-r--r-- 1 dummy users 13742 Jun 19 18:27 5.part
-rw-r--r-- 1 dummy users 488630046 Jun 19 18:32 6.data
-rw-r--r-- 1 dummy users 8174 Jun 19 18:32 6.part
-rw-r--r-- 1 dummy users 834467060 Jun 19 18:17 7.data
-rw-r--r-- 1 dummy users 13718 Jun 19 18:17 7.part
-rw-r--r-- 1 dummy users 485979984 Jun 19 18:28 8.data
-rw-r--r-- 1 dummy users 8158 Jun 19 18:28 8.part
-rw-r--r-- 1 dummy users 791614824 Jun 19 18:31 9.data
-rw-r--r-- 1 dummy users 12790 Jun 19 18:31 9.part

And du says the same:

Alfa tmp # du applejuice/
2851072 applejuice/

hogwart tmp # du applejuice/
8112268 applejuice


This directory was just a small percent and is the only not matching in size, which makes obvious for me that it has to do with the fact that this are temporary files not filled entirely yet.
 
Old 06-21-2004, 05:16 PM   #6
didi156
LQ Newbie
 
Registered: Feb 2004
Distribution: Debian, Ubuntu, uCLinux, Android
Posts: 15

Original Poster
Rep: Reputation: 0
I just did some tests and discovered it was samba to create the size differences. Copying it locally didn't change nothing, but from an smb-mount it did... strange however.
 
Old 06-21-2004, 05:24 PM   #7
Tinkster
Moderator
 
Registered: Apr 2002
Location: earth
Distribution: slackware by choice, others too :} ... android.
Posts: 23,067
Blog Entries: 11

Rep: Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928
Use nfs then ;)


Cheers,
Tink
 
Old 06-22-2004, 07:44 AM   #8
Guttorm
Senior Member
 
Registered: Dec 2003
Location: Trondheim, Norway
Distribution: Debian and Ubuntu
Posts: 1,450

Rep: Reputation: 446Reputation: 446Reputation: 446Reputation: 446Reputation: 446
Hi

Could be sparse files - I file containing mostly zeroes can use very little space on filesystem with this enabled.

Some programs can use this feature to reduce diskspace - e.g. xMule?

Some info:
http://intgat.tigress.co.uk/rmy/uml/sparsify.html

Hope this helps,
Guttorm
 
Old 03-29-2005, 10:26 PM   #9
fturcic
LQ Newbie
 
Registered: Jan 2005
Location: ACT, Australia
Posts: 16

Rep: Reputation: 0
Does the above save permissions, data etc etc for each database or just the schema?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
files being copied into root mercy420 Slackware 1 06-18-2005 07:23 AM
DELL monitor desktop does not occupy the entire screen akalliat Fedora 2 04-22-2005 06:20 AM
Syncing real hdd space with proftpd mod_quotatab_sql data qQsh Linux - Software 0 03-13-2005 10:48 AM
writable 'scratch space' for data between linux and winxp filesys questions... servnov Linux - General 2 02-11-2005 01:28 AM
Copied over rc.M startup script Tarts Slackware 0 12-03-2003 04:21 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 03:48 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration