LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 12-11-2012, 11:25 PM   #1
lucmove
Member
 
Registered: Aug 2005
Location: Brazil
Distribution: Lubuntu, Slackware
Posts: 575

Rep: Reputation: 64
Data won't fit in backup disk


I have lots of data on /mnt/data1. I want to copy it all to a partition on an external hard disk. Both partitions are the same size and I am almost sure they were formatted with the exact same parameters (ext4). But I am using rsync and running out of space on the target file system before the copy is done.

Some info on the issue:

Code:
# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/data1       230G  226G  1.5G 100% /mnt/data1
/dev/mapper/backup      230G  230G     0 100% /media/backup
Code:
# rsync -aHxvz --delete-before /mnt/data1/* /media/backup/
rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: Broken pipe (32)
rsync: write failed on "/media/backup/vms/Lin-Mint.vmdk": No space left on device (28)
rsync error: error in file IO (code 11) at receiver.c(322) [receiver=3.0.8]
rsync: connection unexpectedly closed (33 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(601) [sender=3.0.8]
Note the target partition is completely full, but there was more than 10 GB free before I ran the operation. Also, the --delete option in rsync should delete from the target FS everything that is no longer present in the source FS. The idea obviously is to update the target partition so it is a perfect clone of the source partition.

Now some info on the file systems themselves:

Code:
Filesystem volume name:   data1
Last mounted on:          /mnt/data1
Filesystem UUID:          b7b86639-whatever
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
Filesystem flags:         signed_directory_hash 
Default mount options:    (none)
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              15267328
Block count:              244275296
Reserved block count:     2442752
Free blocks:              9545177
Free inodes:              14404676
First block:              1
Block size:               1024
Fragment size:            1024
Reserved GDT blocks:      256
Blocks per group:         8192
Fragments per group:      8192
Inodes per group:         512
Inode blocks per group:   128
Flex block group size:    16
Filesystem created:       Mon Oct 15 01:18:34 2012
Last mount time:          Mon Dec  3 06:48:09 2012
Last write time:          Mon Dec  3 06:48:09 2012
Mount count:              14
Maximum mount count:      21
Last checked:             Wed Nov 21 07:03:13 2012
Check interval:           15552000 (6 months)
Next check after:         Mon May 20 06:03:13 2013
Lifetime writes:          237 GB
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               256
Required extra isize:     28
Desired extra isize:      28
Journal inode:            8
Default directory hash:   half_md4
Journal backup:           inode blocks
Journal features:         journal_incompat_revoke
Journal size:             32M
Journal length:           32768
Journal sequence:         0x000054f9
Journal start:            15732
Code:
Filesystem volume name:   backup
Last mounted on:          /media/backup
Filesystem UUID:          acf7d8b8-8457-whatever
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recover
y extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
Filesystem flags:         signed_directory_hash 
Default mount options:    (none)
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              15267328
Block count:              244275296
Reserved block count:     2442752
Free blocks:              244401
Free inodes:              14393940
First block:              1
Block size:               1024
Fragment size:            1024
Reserved GDT blocks:      256
Blocks per group:         8192
Fragments per group:      8192
Inodes per group:         512
Inode blocks per group:   128
Flex block group size:    16
Filesystem created:       Fri Oct 12 11:19:06 2012
Last mount time:          Wed Dec 12 02:39:35 2012
Last write time:          Wed Dec 12 02:39:35 2012
Mount count:              7
Maximum mount count:      25
Last checked:             Fri Oct 12 11:19:06 2012
Check interval:           15552000 (6 months)
Next check after:         Wed Apr 10 11:19:06 2013
Lifetime writes:          278 GB
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               256
Required extra isize:     28
Desired extra isize:      28
Journal inode:            8
Default directory hash:   half_md4
Journal backup:           inode blocks
Journal features:         journal_incompat_revoke
Journal size:             32M
Journal length:           32768
Journal sequence:         0x0000372c
Journal start:            9767
Any ideas?

Last edited by lucmove; 12-11-2012 at 11:27 PM.
 
Old 12-12-2012, 04:51 PM   #2
Nbiser
Member
 
Registered: Oct 2012
Location: Maryland
Distribution: Fedora, Slackware, Debian, Ubuntu, Knoppix, Helix,
Posts: 279
Blog Entries: 7

Rep: Reputation: 41
You can do several things: the first is to back up all files on your drive and then change the size on you other partition. Or you can get a bigger backup media.
 
Old 12-12-2012, 07:36 PM   #3
lucmove
Member
 
Registered: Aug 2005
Location: Brazil
Distribution: Lubuntu, Slackware
Posts: 575

Original Poster
Rep: Reputation: 64
You're saying that the target partition must be larger than the source partition?

So you're saying that data is naturally expected to swell during a file copy operation, thus not fit into a file system that is the exact same size as the file system it's coming from?

Can you please describe in interesting detail why or how exactly such swelling occurs? That is just the kind of information I need.
 
Old 12-13-2012, 02:34 PM   #4
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 1,342

Rep: Reputation: 575Reputation: 575Reputation: 575Reputation: 575Reputation: 575Reputation: 575
In your rsync command, you might need to use the "-S" (--sparse) option to handle sparse files efficiently and not expand them on the destination. Note that doing so will not convert files already expanded on the destination to their sparse equivalent.
 
1 members found this post helpful.
Old 12-14-2012, 08:49 PM   #5
lucmove
Member
 
Registered: Aug 2005
Location: Brazil
Distribution: Lubuntu, Slackware
Posts: 575

Original Poster
Rep: Reputation: 64
I have solved this issue. I had done something stupid, like

# rsync -aHxv --delete-before /mnt/data1 /media/backup/

which creates a "data1" directory inside /media/backup/, then I must have aborted and run something like

# rsync -aHxv --delete-before /mnt/data1/* /media/backup/

which will copy all of /mnt/data1/* directly into the root of /media/backup/, which is different from the first command line I had tried.

So I kind of sort of tried to fit the whole shebang from the source partition TWICE into the target partition. Of course it wouldn't fit.

Thank you, rknichols, for the tip regarding the -S option. My backup script never used that option, should do henceforth.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Backup script backs up more data than the size of the source disk? Mark_667 Programming 4 07-18-2012 02:57 AM
My external backup disk is corrupted by window registry how do I recover the data? cbn302 Linux - General 2 07-09-2006 10:01 PM
How much data can fit on a DVD? General General 6 06-25-2006 12:04 AM
ISO image won't fit on CD nleslie Linux - General 2 12-30-2005 10:49 AM
"Data does not fit on disk", from k3b bruno buys Debian 6 08-09-2005 07:51 AM


All times are GMT -5. The time now is 12:19 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration