LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 09-05-2002, 09:36 PM   #1
hemlock
LQ Newbie
 
Registered: Aug 2002
Distribution: RedHat
Posts: 13

Rep: Reputation: 0
Question is there a limit to file size in Linux (RH 7.2)


Is there a limit to file size in Linux (RH 7.2)?

I did something like the following
$tar cvpzf /scsihd/mirror/zdata.tar.gz /datadir/data >>/scsihd/mirror/tarlog.txt

/datadir/data is a directory.
I did a test of this on a smaller dataset & once it completed I was able in Konqueror to click on the .tar.gz file & it would walk thru the directory structure like it was a complete filesystem (directory & subdirectory), opening files within this tar.gz file that were not directories.

When I did this on a large dataset (ie the directory /datadir/data & subdirs contain about 10Gb), there was ample room on the partition containing the tar file (16gb). There was no indication of a problem on the log file.
However, when I attempted to open/enter it as in above, I got a message 'the file or directory did not exist' (or similar). When I looked at the properties, it showed a large negative number for the size. (This all in Konqueror (sp?)). Just realized I didn't try an ls on it or if I did don't remember. I tried opening it w/ archiver & got the same message file not exist. But Konqueror definetly shows it there (with some size that seems unreasonable).

What gives?

Last edited by hemlock; 09-05-2002 at 09:37 PM.
 
Old 09-05-2002, 09:47 PM   #2
DavidPhillips
LQ Guru
 
Registered: Jun 2001
Location: South Alabama
Distribution: Fedora / RedHat / SuSE
Posts: 7,163

Rep: Reputation: 58
Maximum Sizes of the Filesystems
Filesystem File Size Limit Filesystem Size Limit
ext2 with 1 KiB blocksize 16448 MiB (~ 16 GiB) 2048 GiB (= 2 TiB)
ext2 with 2 KiB blocksize 256 GiB 8192 GiB (= 8 TiB)
ext2 with 4 KiB blocksize 2048 GiB (= 2 TiB) 16384 GiB (= 16 TiB)
ext2 with 8 KiB blocksize (Systems with 8 KiB pages like Alpha only) 65568 GiB (~ 64 TiB) 32768 GiB (= 32 TiB)
ReiserFS 3.5 4 GiB 16384 GiB (= 16 TiB)
ReiserFS 3.6 (as in Linux 2.4) 260 Bytes (= 1 EiB) 16384 GiB (= 16 TiB)
XFS 263 Bytes (= 8 EiB) 2048 GiB (= 2 TiB)(Linux kernel limitation)
JFS with 512 Bytes blocksize 4194304 GiB (= 4 PiB) 512 TiB
JFS with 4KiB blocksize 33554432 GiB (= 32 PiB) 4 PiB

Note Kernel Limitations: The table above describes limitations of the on-disk format. The maximal size of a file is limited to 2 TiB in Kernel 2.4.x. This might change during the development of Linux 2.5.x. Kernel 2.5.x might also bring changes to handle larger file systems.

Note in the above: 1024 Bytes = 1 KiB; 1024 KiB = 1 MiB; 1024 MiB = 1 GiB; 1024 GiB = 1 TiB; 1024 TiB = 1 PiB; 1024 PiB = 1 EiB (check http://physics.nist.gov/cuu/Units/binary.html)
Maximum Number of Partitions
An IDE disk has 64 minors, one is used for the full disk and therefore 63 partitions are possible. A SCSI disk has 16 minors and therefore only 15 partitions maximal.
Links

* XFS
http://oss.sgi.com/projects/xfs/

* ReiserFS
http://www.reiserfs.org

* JFS
http://oss.software.ibm.com/developer/opensource/jfs

Last edited by DavidPhillips; 09-05-2002 at 09:49 PM.
 
Old 09-05-2002, 11:31 PM   #3
hemlock
LQ Newbie
 
Registered: Aug 2002
Distribution: RedHat
Posts: 13

Original Poster
Rep: Reputation: 0
thanks for the answer to the first question.
Can you offer any surmises for the second, since it would appear that the tar (-z)'d file should be within the limits you list. I am a newbie & don't know what all that stuff is, but the file type is ext2, I don't know block size (how would I find that out). I didn't create the file system; who did probably took whatever default (RH 7.2) using fdisk & mke2fs.
 
Old 09-06-2002, 12:20 AM   #4
DavidPhillips
LQ Guru
 
Registered: Jun 2001
Location: South Alabama
Distribution: Fedora / RedHat / SuSE
Posts: 7,163

Rep: Reputation: 58
dumpe2fs /dev/hda1
 
Old 09-06-2002, 08:30 AM   #5
hemlock
LQ Newbie
 
Registered: Aug 2002
Distribution: RedHat
Posts: 13

Original Poster
Rep: Reputation: 0
thanks for how to find the block size. Do you have any surmise about why I'm getting file doesn't exist message?
If you don't know, I won't think less of you. In fact, nothing gets my respect more than someone willing to admit they don't know something.
 
Old 09-06-2002, 10:48 AM   #6
DavidPhillips
LQ Guru
 
Registered: Jun 2001
Location: South Alabama
Distribution: Fedora / RedHat / SuSE
Posts: 7,163

Rep: Reputation: 58
not sure about konq but try ls

try putting a dash in front of the arguments for tar

tar -cvpzf
 
Old 09-06-2002, 07:06 PM   #7
hemlock
LQ Newbie
 
Registered: Aug 2002
Distribution: RedHat
Posts: 13

Original Poster
Rep: Reputation: 0
dume2fs said block size is 4096, 4,421,631 free blocks. (after I deleted that file). so I can't do ls on it now. I'll probably try to create it again in pieces.

I would be shocked! shocked! if a - did anything different as the log (>> of v output) indicated it was traversing the directory structure just fine. Somehow I couldn't get it to redirect error output to that file as well, so it could have been trying to tell me something & if it's on the system somewhere I don't know how to find it yet.

Last edited by hemlock; 09-06-2002 at 07:10 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
File Size limit? compu73rg33k Linux - Software 1 02-27-2005 10:11 PM
limit file size ust Linux - General 2 02-24-2005 07:33 AM
File Size Limit?? Manuel-H Linux - Software 1 04-16-2004 03:23 AM
File size limit petway56 Linux - General 4 05-07-2003 11:19 PM
file size limit jaysan Slackware 5 07-12-2002 03:43 AM


All times are GMT -5. The time now is 06:45 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration