Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place! |
Notices |
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
Are you new to LinuxQuestions.org? Visit the following links:
Site Howto |
Site FAQ |
Sitemap |
Register Now
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
|
 |
12-21-2004, 03:50 AM
|
#1
|
Member
Registered: Dec 2004
Location: Vienna
Posts: 46
Rep:
|
Large file size support for linux
I have a problem with my htdig-database-file...the file eats up more than 2 Gigs harddisk, which is the natural file size limit on standard-linux distros.
Now I heard about LFS(Large file size support) for linux...
Is it easy to enable this feature?
Do I have just to recompile my kernel with LFS-support?
Or isn't that so easy at all?
Otherwise I have to reduce the search-capability of my search enginge, which really isn't the finest way...
|
|
|
12-21-2004, 03:54 AM
|
#2
|
LQ Guru
Registered: Mar 2002
Location: Salt Lake City, UT - USA
Distribution: Gentoo ; LFS ; Kubuntu ; CentOS ; Raspbian
Posts: 12,613
Rep:
|
The filesystem determines the filesize limit, not the OS. XFS, JFS, EXT3 all allow for files larger than 2GB. I believe it was just a limitation of ext2... I personally haven't had to 'enable' anything (maybe I have in the kernel and simply don't remember as it is just a matter of fact...) since ext3 came about.
HTH
Cool
|
|
|
12-21-2004, 05:04 AM
|
#3
|
Member
Registered: Dec 2004
Location: Vienna
Posts: 46
Original Poster
Rep:
|
if this would be the only problem, that I use ext2, I'd be really happy...
It's just because I've read the following tutorial about LFS from suse:
http://www.suse.de/~aj/linux_lfs.html
excerpt:
Quote:
LFS raises the limit of maximal file size. For 32-bit systems the limit is 231 (2 GiB) but using the LFS interface on filesystems that support LFS applications can handle files as large as 263 bytes.
For 64-bit systems the file size limit is 263 bytes unless a filesystem (like NFSv2) only supports less.
|
this seems to say, that it isn't the filesystem, but the architecture that it is depending on...
further more, there is following written:
Quote:
For using LFS in user programs, the programs have to use the LFS API. This involves recompilation and changes of programs. The API is documented in the glibc manual (the libc info pages) which can be read with e.g. "info libc".
In a nutshell for using LFS you can choose either of the following:
* Compile your programs with "gcc -D_FILE_OFFSET_BITS=64". This
forces all file access calls to use the 64 bit variants. Several types change
also, e.g. off_t becomes off64_t. It's therefore important to always use the
correct types and to not use e.g. int instead of off_t. For portability with
other platforms you should use getconf LFS_CFLAGS which will return
-D_FILE_OFFSET_BITS=64 on Linux platforms but might return something
else on e.g. Solaris. For linking, you should use the link flags that are
reported via getconf LFS_LDFLAGS. On Linux systems, you do not need
special link flags.
* Define _LARGEFILE_SOURCE and _LARGEFILE64_SOURCE. With
these defines you can use the LFS functions like open64 directly.
* Use the O_LARGEFILE flag with open to operate on large files.
|
thats what me made thinking about recompiling the kernel or the programs...
|
|
|
12-21-2004, 05:55 AM
|
#4
|
LQ Guru
Registered: Mar 2002
Location: Salt Lake City, UT - USA
Distribution: Gentoo ; LFS ; Kubuntu ; CentOS ; Raspbian
Posts: 12,613
Rep:
|
Off the top of my head, I cannot recall where I recently saw those types of things being dealt with, however, LFS was needing to be enabled to do... something. But from my experience, I have had great success in using several applications (a common example is video editing) that work with large files on my HD. I generally use ext3 but have recently switched to XFS on some of my newer systems to check out it's 'performance enhancement' but either way, since I have switched up to ext3 I have not had any problems with large files (bigger than 2GB).
Cool
|
|
|
12-21-2004, 06:06 AM
|
#5
|
Member
Registered: Dec 2004
Location: Vienna
Posts: 46
Original Poster
Rep:
|
so the question is for me now, is changing to ext3 solving my problem??
Maybe I should give a try to recompile this mandrake-kernel...
allthough it seems a little bit difficult for me...
|
|
|
12-21-2004, 07:07 AM
|
#6
|
LQ Guru
Registered: Mar 2002
Location: Salt Lake City, UT - USA
Distribution: Gentoo ; LFS ; Kubuntu ; CentOS ; Raspbian
Posts: 12,613
Rep:
|
To switch from ext2 to ext3, it's a tune2fs command:
tune2fs -j /dev/hdx
Where hdx is your device. However, I've been fortunate enough to be able to start with a fresh mke2fs -j so YMMV. However, it's certainly worth a shot, remember to remount the devices with ext3 rather than ext2 as your filesystem option:
mount -t ext3
OR
/etc/fstab entry change to ext3
HTH
If that is too vague, feel free to reply, I can go into more depth if you need it.
Cool
|
|
|
12-21-2004, 07:20 AM
|
#7
|
Member
Registered: Dec 2004
Location: Vienna
Posts: 46
Original Poster
Rep:
|
If you think it will help solving my problem I'll give it a try...
Are you sure that I don't loose my data with
tune2fs -j /dev/hda
?
It's my root partition...wouldn't be nice to loose it 
|
|
|
12-22-2004, 03:32 AM
|
#8
|
LQ Guru
Registered: Mar 2002
Location: Salt Lake City, UT - USA
Distribution: Gentoo ; LFS ; Kubuntu ; CentOS ; Raspbian
Posts: 12,613
Rep:
|
No it certainly wouldn't. No you are fine, if you run into problems, you can always revert back to mounting the device as ext2. You lose nothing but a bit of space for the journal.
Cool
|
|
|
12-23-2004, 04:33 AM
|
#9
|
Member
Registered: Dec 2004
Location: Vienna
Posts: 46
Original Poster
Rep:
|
I did this tune - comman dnow and there wasn't any errormessage...
Did it worked?
Is there a command to find out on which filesystem I'm running now?
|
|
|
12-23-2004, 04:56 AM
|
#10
|
LQ Guru
Registered: Mar 2002
Location: Salt Lake City, UT - USA
Distribution: Gentoo ; LFS ; Kubuntu ; CentOS ; Raspbian
Posts: 12,613
Rep:
|
mount
Just plain with no delimiters. This will show you where everything is mounted and with what filesystem.
You will have to remount the device(s) with the ext3 option, it will not default to this. You will also have to edit fstab to have them mounted with the ext3 option upon successive reboots.
Cool
|
|
|
01-10-2005, 09:10 AM
|
#11
|
Member
Registered: Dec 2004
Location: Vienna
Posts: 46
Original Poster
Rep:
|
Hi anybody, I figured something out now:
It's not that my filesystem does not support >2GB file size, because I can make files larger than this actually!
(tried it with "dd if/dev/zero of=/test bs=1M count=3500")
Could it be that the app uses the fseek command, that handles ints, and not longs so there is only 32-bit support???
(I think I've found an issue about htat somewhere)
This would be the only idea I have...
What makes me wonder is that htdig claims to have no size limit in the documentation...
maybe I have to recompile the program against another glibc library?
Does anybody know how to do this??
|
|
|
01-11-2005, 09:01 AM
|
#12
|
Member
Registered: Dec 2004
Location: Vienna
Posts: 46
Original Poster
Rep:
|
HEELLLLPPP PLEASE !!!!
Please help me !!!!
I don't know what to do...searched the whole net in meanwhile....
|
|
|
All times are GMT -5. The time now is 01:21 PM.
|
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
|
Latest Threads
LQ News
|
|