LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 06-27-2005, 01:14 AM   #1
Nerox
Member
 
Registered: Jul 2004
Location: Spain
Posts: 111

Rep: Reputation: 20
Creating files larger than 2GB


Hello there, i'm trying to create a 8GB file in size, but i cannot do that with the traditional fopen() function. Any ideas?

Thanks in Adavance.
 
Old 06-27-2005, 01:59 AM   #2
freegianghu
Member
 
Registered: Oct 2004
Location: somewhere in the street
Distribution: Window$
Posts: 192

Rep: Reputation: 30
Re: Creating files larger than 2GB

Quote:
Originally posted by Nerox
Hello there, i'm trying to create a 8GB file in size, but i cannot do that with the traditional fopen() function. Any ideas?

Thanks in Adavance.
Whats kind of your partition? Does your system support 64-bit file operations (large file support)? Is there a quota limitation?

Hopes this help
GH
 
Old 06-27-2005, 02:10 AM   #3
Artik
Member
 
Registered: Apr 2005
Location: Israel
Distribution: Debian 3.1 Sarge, Ubuntu Breezy
Posts: 223

Rep: Reputation: 30
Two potencial problems:
  1. You must use one of XFS, ReiserFS, JFS to create files lager then 2GB. Ext2/3 support only 2GB maximum file size.
  2. For big files read info pages running info libc -> stdio64
    I think you need to use fopen64() but not completly sure.



Last edited by Artik; 06-27-2005 at 02:16 AM.
 
Old 06-28-2005, 06:15 AM   #4
Nerox
Member
 
Registered: Jul 2004
Location: Spain
Posts: 111

Original Poster
Rep: Reputation: 20
Thanks, i was using Ext2 fs. I will try it with one of those. I think fopen64 is not in my system, should i install a newer glib version?
 
Old 08-02-2007, 11:27 AM   #5
janctil
LQ Newbie
 
Registered: Aug 2007
Posts: 3

Rep: Reputation: 0
Question Another issue with files greater than 2GB


I am running on Red Hat Enterprise 3 OS and using the gcc compiler, I can generate files greater than 2GB by compiling the code with the following:

gcc -o fsl.exe -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE fsl.c

but, if I write a similar program in c++ and compile it with the exact same options using g++, then I receive a 'file size limit exceeded' message when the output file reaches the 2GB limit.

Can anyone recommend either a compiler option or some other option in which I can generate large files (>2GB) using the C++ compiler (in this case the g++ compiler).

Thank you,

John
 
Old 08-02-2007, 11:52 AM   #6
ta0kira
Senior Member
 
Registered: Sep 2004
Distribution: FreeBSD 9.1, Kubuntu 12.10
Posts: 3,078

Rep: Reputation: Disabled
Most of the glibc file functions have a 64-bit version with "64" appended to the name. I think this will alter the size type used, so you might need to use a 64-bit integer for size and offset values. If you look at info libc and find a function such as open you should see a variant such as open64.
ta0kira
 
Old 08-02-2007, 12:31 PM   #7
janctil
LQ Newbie
 
Registered: Aug 2007
Posts: 3

Rep: Reputation: 0
I do not completely understand

I am opening the file using ofstream, and when I write to this file, at the point where it exceeds the 2GB limit, I get the error "filesize limit exceeded".

I have tried all the options as presented in my original post, but I can not seem to get the g++ compiler to generate the correct code to create large datafiles.

Thanks,

John
 
Old 08-02-2007, 01:36 PM   #8
ta0kira
Senior Member
 
Registered: Sep 2004
Distribution: FreeBSD 9.1, Kubuntu 12.10
Posts: 3,078

Rep: Reputation: Disabled
Quote:
Originally Posted by janctil
I am opening the file using ofstream, and when I write to this file, at the point where it exceeds the 2GB limit, I get the error "filesize limit exceeded".
I'm pretty sure ofstream is stuck with the register size of the architecture. While it's a portable way to deal with files in the C++ realm, with glibc available I think you're better off just using C to deal with huge files.
ta0kira
 
Old 08-02-2007, 02:53 PM   #9
exvor
Senior Member
 
Registered: Jul 2004
Location: Phoenix, Arizona
Distribution: Gentoo, LFS, Debian,Ubuntu
Posts: 1,537

Rep: Reputation: 87
ext2 and ext3 support files larger then 2gb your thinking of fat32 not ext2
 
Old 08-03-2007, 09:44 AM   #10
janctil
LQ Newbie
 
Registered: Aug 2007
Posts: 3

Rep: Reputation: 0
Question How to check

How do I check for either ext2 or ext3 on my system?
 
Old 08-03-2007, 10:46 AM   #11
SciYro
Senior Member
 
Registered: Oct 2003
Location: hopefully not here
Distribution: Gentoo
Posts: 2,038

Rep: Reputation: 51
Just type "mount" to see a list of mounted file systems.
 
Old 08-03-2007, 06:09 PM   #12
ta0kira
Senior Member
 
Registered: Sep 2004
Distribution: FreeBSD 9.1, Kubuntu 12.10
Posts: 3,078

Rep: Reputation: Disabled
If you still have problems creating files larger than 2GB, you might want to check to see that you are authorized to create one larger. See man getrlimit.
ta0kira
 
  


Reply

Tags
file, linux, support


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Samba and 2GB + files TheRealDeal Linux - Networking 7 08-22-2005 08:24 PM
problem with files larger then 100Mbyte++ carboncopy Slackware 8 04-26-2005 10:40 AM
apache > 2GB files RDove Linux - Software 1 03-11-2005 10:53 AM
Files larger than 4 GB Hamsjael Linux - Software 1 03-09-2005 01:51 PM
Can't copy files larger than 2GB? Rotwang Linux - General 6 03-09-2005 12:17 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 06:39 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration