LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Programming (http://www.linuxquestions.org/questions/programming-9/)
-   -   Creating files larger than 2GB (http://www.linuxquestions.org/questions/programming-9/creating-files-larger-than-2gb-337523/)

Nerox 06-27-2005 01:14 AM

Creating files larger than 2GB
 
Hello there, i'm trying to create a 8GB file in size, but i cannot do that with the traditional fopen() function. Any ideas?

Thanks in Adavance.

freegianghu 06-27-2005 01:59 AM

Re: Creating files larger than 2GB
 
Quote:

Originally posted by Nerox
Hello there, i'm trying to create a 8GB file in size, but i cannot do that with the traditional fopen() function. Any ideas?

Thanks in Adavance.

Whats kind of your partition? Does your system support 64-bit file operations (large file support)? Is there a quota limitation?

Hopes this help
GH

Artik 06-27-2005 02:10 AM

Two potencial problems:
  1. You must use one of XFS, ReiserFS, JFS to create files lager then 2GB. Ext2/3 support only 2GB maximum file size.
  2. For big files read info pages running info libc -> stdio64
    I think you need to use fopen64() but not completly sure.



Nerox 06-28-2005 06:15 AM

Thanks, i was using Ext2 fs. I will try it with one of those. I think fopen64 is not in my system, should i install a newer glib version?

janctil 08-02-2007 11:27 AM

Another issue with files greater than 2GB
 
:newbie:
I am running on Red Hat Enterprise 3 OS and using the gcc compiler, I can generate files greater than 2GB by compiling the code with the following:

gcc -o fsl.exe -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE fsl.c

but, if I write a similar program in c++ and compile it with the exact same options using g++, then I receive a 'file size limit exceeded' message when the output file reaches the 2GB limit.

Can anyone recommend either a compiler option or some other option in which I can generate large files (>2GB) using the C++ compiler (in this case the g++ compiler).

Thank you,

John

ta0kira 08-02-2007 11:52 AM

Most of the glibc file functions have a 64-bit version with "64" appended to the name. I think this will alter the size type used, so you might need to use a 64-bit integer for size and offset values. If you look at info libc and find a function such as open you should see a variant such as open64.
ta0kira

janctil 08-02-2007 12:31 PM

I do not completely understand
 
I am opening the file using ofstream, and when I write to this file, at the point where it exceeds the 2GB limit, I get the error "filesize limit exceeded".

I have tried all the options as presented in my original post, but I can not seem to get the g++ compiler to generate the correct code to create large datafiles.

Thanks,

John

ta0kira 08-02-2007 01:36 PM

Quote:

Originally Posted by janctil
I am opening the file using ofstream, and when I write to this file, at the point where it exceeds the 2GB limit, I get the error "filesize limit exceeded".

I'm pretty sure ofstream is stuck with the register size of the architecture. While it's a portable way to deal with files in the C++ realm, with glibc available I think you're better off just using C to deal with huge files.
ta0kira

exvor 08-02-2007 02:53 PM

ext2 and ext3 support files larger then 2gb your thinking of fat32 not ext2

janctil 08-03-2007 09:44 AM

How to check
 
How do I check for either ext2 or ext3 on my system?

SciYro 08-03-2007 10:46 AM

Just type "mount" to see a list of mounted file systems.

ta0kira 08-03-2007 06:09 PM

If you still have problems creating files larger than 2GB, you might want to check to see that you are authorized to create one larger. See man getrlimit.
ta0kira


All times are GMT -5. The time now is 10:42 PM.