LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 03-07-2008, 05:03 AM   #1
linuxhippy
Senior Member
 
Registered: Sep 2004
Location: Philadelphia, PA
Distribution: Xubuntu, Mythbuntu, Lubuntu, Picuntu, Mint 18.1, Debian Jessie
Posts: 1,207

Rep: Reputation: 47
copy a 6 GB file


I want to back up a 6 GB file to my external hard drive. Using cp with no switches I'm getting an error that the file is too large. How can I copy this file?
 
Old 03-07-2008, 05:11 AM   #2
billymayday
LQ Guru
 
Registered: Mar 2006
Location: Sydney, Australia
Distribution: Fedora, CentOS, OpenSuse, Slack, Gentoo, Debian, Arch, PCBSD
Posts: 6,678

Rep: Reputation: 122Reputation: 122
What file system is the external drive? If it FAT, then there is a 4g (?) limit on files. If you are simply transporting or backing up, you could use split to break it into 4g chunks.
 
Old 03-07-2008, 05:12 AM   #3
aus9
LQ 5k Club
 
Registered: Oct 2003
Location: Australia
Distribution: IceWM on Debian
Posts: 5,490

Rep: Reputation: Disabled
it may be easier to not use the command line.

do you have KDE or gnome or a windowmanager?
 
Old 03-07-2008, 05:34 AM   #4
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 680Reputation: 680Reputation: 680Reputation: 680Reputation: 680Reputation: 680
If there is a filesystem filesize limit or a limit in the transport mechanism, then copying from the GUI won't make a difference.

If this is a tar backup, you can pipe the output of tar through split to save the tar backup to an external usb drive. ( I would use slices under 2GB. Some versions of samba or some utilities may use a signed integer, so the limit may be 2GB in some cases. Better be safe than sorry.

You can extract from a split backup like: cat /media/disk/backup.tar.* | tar xvf - <filename or dir to restore>
This will extract the file(s) from the stdin stream. You don't need to reassemble the archive.

Note, that you could have also have produced a multi-part tar backup in the first place.
 
Old 03-07-2008, 06:01 AM   #5
linuxhippy
Senior Member
 
Registered: Sep 2004
Location: Philadelphia, PA
Distribution: Xubuntu, Mythbuntu, Lubuntu, Picuntu, Mint 18.1, Debian Jessie
Posts: 1,207

Original Poster
Rep: Reputation: 47
this is a vdi file that was produced in VirtualBox (it's a backup file). The file is on my ext3 Linux partition (when opened the file is NTFS and is a WinXP installation). The external drive has 2 partitions that are 80 GB each...1 is ext3 and 1 is vfat. I'd like to copy this vdi file to a vfat partition.

EDIT: I just tried copying it using konqueror and it wouldn't copy either. The gui way only copied 15 MB where the cli way copied 4 GB.

Last edited by linuxhippy; 03-07-2008 at 06:14 AM.
 
Old 03-07-2008, 07:50 AM   #6
Nathanael
Member
 
Registered: May 2004
Location: Karlsruhe, Germany
Distribution: debian, gentoo, os x (darwin), ubuntu
Posts: 940

Rep: Reputation: 33
IIRC 4GB is the limit of a single file on a standard fat32 file system

if you need to access the file on a windows machine, you might be better off copying to the ext3 partition, and installing ext3 drivers on the windows machine.

or scp from/to windows, writing to a ntfs partition.
 
Old 03-07-2008, 08:56 AM   #7
eantoranz
Senior Member
 
Registered: Apr 2003
Location: Costa Rica
Distribution: Kubuntu, Debian, Knoppix
Posts: 2,092
Blog Entries: 1

Rep: Reputation: 90
or use split to break the file into pieces, say 2GBs each.
 
Old 03-07-2008, 10:39 AM   #8
linuxhippy
Senior Member
 
Registered: Sep 2004
Location: Philadelphia, PA
Distribution: Xubuntu, Mythbuntu, Lubuntu, Picuntu, Mint 18.1, Debian Jessie
Posts: 1,207

Original Poster
Rep: Reputation: 47
I read a post when I was googling that said to use split to break the file up into parts and then cat to re-assemble. Is this reliable?
 
Old 03-07-2008, 01:22 PM   #9
billymayday
LQ Guru
 
Registered: Mar 2006
Location: Sydney, Australia
Distribution: Fedora, CentOS, OpenSuse, Slack, Gentoo, Debian, Arch, PCBSD
Posts: 6,678

Rep: Reputation: 122Reputation: 122
I use split/cat for a backup to an external drive all the time

something like

split -b 4000m infile.vdi /mnt/usb_drive/infile.vdi.split.

cat /mnt/usb_drive/infile.vdi.split.aa > /dest/infile.vdi
cat /mnt/usb_drive/infile.vdi.split.ab >> /dest/infile.vdi

Note the ">>" rather than ">" in second line

I also take an md5sum before the process and copy that to the external drive so when I cat the files back together, so I know it's worked
 
Old 03-07-2008, 05:44 PM   #10
linuxhippy
Senior Member
 
Registered: Sep 2004
Location: Philadelphia, PA
Distribution: Xubuntu, Mythbuntu, Lubuntu, Picuntu, Mint 18.1, Debian Jessie
Posts: 1,207

Original Poster
Rep: Reputation: 47
the split command worked fine but I had trouble re-assembling with cat on my vfat partition (I got an error once the file got over 4 GB). It worked fine on the other ext3 partition I had on my external drive...it checked out with md5sum. Thanks!
 
Old 03-07-2008, 06:02 PM   #11
eantoranz
Senior Member
 
Registered: Apr 2003
Location: Costa Rica
Distribution: Kubuntu, Debian, Knoppix
Posts: 2,092
Blog Entries: 1

Rep: Reputation: 90
As people told you before, VFAT won-t allow you to have a file bigger than 4 Gbs, that's why it will fail there.
 
Old 03-07-2008, 09:44 PM   #12
v00d00101
Member
 
Registered: Jun 2003
Location: UK
Distribution: Devuan Beowulf
Posts: 505
Blog Entries: 1

Rep: Reputation: 36
This should make things a lot clearer for our annoying friend.

Quote:
Originally Posted by Wikipedia

FAT32

In order to overcome the volume size limit of FAT16, while still allowing DOS real-mode code to handle the format without unnecessarily reducing the available conventional memory, Microsoft decided to implement a newer generation of FAT, known as FAT32, with cluster values held in a 32-bit field, of which 28 bits are used to hold the cluster number, for a maximum of approximately 268 million (228) clusters. This would allow for drive sizes of up to 8 tebibytes with 32KiB clusters, but the boot sector uses a 32-bit field for the sector count, limiting volume size to 2TiB on a hard disk with 512 byte sectors.

On Windows 95/98, due to the version of Microsoft's ScanDisk utility included with these operating systems being a 16-bit application, the FAT structure is not allowed to grow beyond around 4.2 million (< 222) clusters, placing the volume limit at 127.53 gibibytes.[6]. A limitation in original versions of Windows 98/98SE's Fdisk causes it to incorrectly report disk sizes over 64GiB.[7] A corrected version is available from Microsoft. These limitations do not apply to Windows 2000/XP except during Setup, in which there is a 32GiB limit.[8] Windows Me supports the FAT32 file system without any limits.[9]

FAT32 was introduced with Windows 95 OSR2, although reformatting was needed to use it, and DriveSpace 3 (the version that came with Windows 95 OSR2 and Windows 98) never supported it. Windows 98 introduced a utility to convert existing hard disks from FAT16 to FAT32 without loss of data. In the NT line, native support for FAT32 arrived in Windows 2000. A free FAT32 driver for Windows NT 4.0 was available from Winternals, a company later acquired by Microsoft. Since the acquisition the driver is no longer officially available.

Windows 2000 and Windows XP can read and write to FAT32 file systems of any size, but the format program included in Windows 2000 and higher can only create FAT32 file systems of 32 GiB or less. This limitation is by design[6] and was imposed because many tasks on a very large FAT32 file system become slow and inefficient.[10] This limitation can be bypassed by using third-party formatting utilities.

The maximum possible size for a file on a FAT32 volume is 4 GiB minus 1 Byte (232−1 bytes). Video capture and editing applications and some other software can easily exceed this limit. Larger files require another formatting type such as HFS+ or NTFS. Until mid-2006, those who run dual boot systems or who move external data drives between computers with different operating systems had little choice but to stick with FAT32. Since then, full support for NTFS has become available in Linux and many other operating systems, by installing the FUSE library (on Linux) together with the NTFS-3G application. Data exchange is also possible between Windows and Linux by using the Linux-native ext2 or ext3 file systems through the use of external drivers for Windows, such as ext2 IFS; however, Windows cannot boot from ext2 or ext3 partitions.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
squirrel Could not move/copy file. File not attached mrlinux2000 Linux - Software 4 05-24-2010 03:41 AM
how to rename a file and copy a file in a shell zach014 Linux - Newbie 6 11-23-2006 09:23 AM
copy vid tape to file and burn file to DVD jim mann Linux - Software 4 01-24-2006 09:57 PM
How to copy mysql file into text file lumba General 0 09-26-2005 05:08 AM
Is arrangement of file systems will differ if we copy a file from FAT 32 to ext 3 ? anindyanuri Linux - Software 2 02-20-2005 11:39 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 02:38 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration