Red HatThis forum is for the discussion of Red Hat Linux.
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
Distribution: Fedora Core 3, Red Hat 9, CentOS 4.2, Mandriva, Ret Hat Enterprise Linux 4.0
tar file size limit
I'm working on a logging infrastructure for an application. There is a directory in which log files are created. Logrotate is run to rotate the files based on size. However, when the date changes all the files from that day are archived into a tar.gz file.
What is largest file size that tar can handle i.e. archive and compress? I read at some places it's 2GB. I'm using RHEL 4.0 ES Server.
For historical reasons numerical values are encoded in octal with leading zeroes. The final character is either a null or a space. Thus although there are 12 bytes reserved for storing the file size, only 11 octal digits can be stored. This gives a maximum file size of 8 gigabytes on archived files. To overcome this limitation some versions of tar, including the GNU implementation, support an extension in which the file size is encoded in binary. Additionally, versions of GNU tar from 1999 and before pad the values with space characters instead of zero characters.
This comes from the Wikipedia tarball page. The GNU Tar Manual, section 8 seems to indicate no limit when using GNU Tar. Of course, you might run into problems on other systems not using GNU tar.
I am unaware of any file size limit with tar or gzip per se. I currently have tar.gz backups in excess of 7GB. AFAIK the only limits imposed are those of the filesystem you are backing up to, eg FAT32 has a max 4GB single file size limit. Also, if you are backing up to DVD there may be issues with single file sizes greater than 2GB IIRC.
I believe that the 2GB is due to the kernel not using large file support or something similar. RHEL 4 is pretty new and these issues should already be addressed. You shouldn't encounter the 2GB file size limit.