Get around 2GB Tar Limit
 

Are there any suggestions or tips on getting around the 2.0GB Size limit for tar files in Debian Etch? Would like to use tar for backups but have some backups that well exceed 2.0GB and this causes problems with backup scripts.

Have thought about dd to an image file but prefer tar or some other similar utility that will work with my backup scripts. Suggestions are greatly appreciated.

Three things that effect file size limitations are the 1. the kernel, 2. the file system and 3. the application. Since Debian Etch uses a 2.6 kernel I would rule out 1. unless you have
compiled your own kernel and did not include large file support. 2. Would only be a problem
if you are trying to save the file to a FAT32 file system or using a kernel that did not include large file support and 3. would be a problem if using an old version of tar which does not handle large files.

So what kernel are you running, what file system are you trying to save the tar file on and what version of tar are you actually using? Doing some googling there appeared to be single file size limitation of 2GB but again I believe the latest version can handle a single file of 68 GB.

To see if your version of tar has large file support:
strings `which tar`|grep 64

If you are trying to save to a network share then you could have samba limitations depending on version.

So basically it is difficult to say what the problem is without additional information.

You can have bash configured to limit created file sizes. Check that by running:

$ ulimit -a

in a console. If you get a line of output that reads:

file size (blocks, -f) unlimited

Your OK there. I know the Bastille security program can be configured to limit the size of any created files as well. There's probably other examples, but the point is that there is no inherent limitation in tar itself limiting archives to 2GB. I routinely create much larger ones than that. It's probably not a tar problem; it's more likely something in the way your system is configured, unless you are running some really ancient version of tar for some reason.

kilgoretrout here is my output. Let me know what you think.


I am trying to have the file go over a samba share to a windows box that has a shared folder on an NTFS File System.

This command does not work for me. Strings command is not found. Is strings a package I need installed... here is output of tar version.

I will try to see if I bypass limit by taring to local file system. Using ext3 file system with stock kernel.

Yes, depending on the version samba does have a 2gb limitation if you do not mount the share with the lfs option. The latest uses cifs which should not have this problem.

mount -t smbfs -o lfs //server/share /localdir

How would I do and enable lfs via fstab? My fstab is below.

Try:
//cfsscdm1/SFTP /data smbfs rw,auto,credentials=/etc/samba/cred-file,uid=$i,gid=users,fmask=0770,dmask=0770,lfs 0 0

I was able to test the tar file to the local ext3 file system and it worked, backup came out at a nice 3.6GB. The problem lies with samba. So now I just need to get the samba share to allow for taring over 2GB files or tar to the local system and mv to remote share via my scripts. Installed Saber version is unknown, how would I find out samba version. Unable to parse via /etc/init.d/samba --version.

Well it should work (must be root):
/usr/sbin/smbd -V
or
smbstatus

Thanks Michael, Problem Solved and Thread/Issue can now be closed. Adding lfs option to smb mount really helped. I can now run my scripts and not worry about them ejecting because of a file limitation. Again, thanks so much. You made my life much easier. And in the process we've created a help doc for other users that have this problem.

The version of samba that requires lfs option is

Excellent.