LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 03-15-2007, 11:54 AM   #1
rdelia
LQ Newbie
 
Registered: Jul 2006
Posts: 15

Rep: Reputation: 0
Is there a file size limitation for transfering files in Red Hat ES 4?


We are using an NFS mount and need to transfer 200 Mb and larger files via this mount. Both NFS and FTP cause the I/O to max out. We've tested several scenarios and have determined it is indeed LINUX that has the issue. Is there a file size limitation here? We have a case with Red Hat but need this resolved ASAP as this affects our financial systems.

Thanks for your help!
 
Old 03-15-2007, 12:03 PM   #2
macemoneta
Senior Member
 
Registered: Jan 2005
Location: Manalapan, NJ
Distribution: Fedora x86 and x86_64, Debian PPC and ARM, Android
Posts: 4,593
Blog Entries: 2

Rep: Reputation: 332Reputation: 332Reputation: 332Reputation: 332
Not seeing this issue on Fedora (I've transferred files 10's of GB over NFS), so there must be an implementation issue. 200MB files are pretty small by today's standards.
 
Old 03-15-2007, 12:10 PM   #3
rdelia
LQ Newbie
 
Registered: Jul 2006
Posts: 15

Original Poster
Rep: Reputation: 0
We've done this test on different Linux boxes to eliminate the hardware and installation methods. I installed Linux using all the defaults on the test box and our production box was installed by a vendor installer. Same issue on both boxes. The similarity between the boxes is that they are both using Red Hat 4. Do we need to enable large files in the kernel or is that the default setting?
 
Old 03-15-2007, 12:31 PM   #4
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 23,067
Blog Entries: 11

Rep: Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910
I'd guess that you're hitting a driver issue - either the
network-card or the disk subsystem one. Check whether there's
a newer kernel available. What hardware is this on? What
version is your kernel?



Cheers,
Tink
 
Old 03-15-2007, 02:00 PM   #5
rdelia
LQ Newbie
 
Registered: Jul 2006
Posts: 15

Original Poster
Rep: Reputation: 0
Tink,

We've tried this on 2 completely different hardware forms. Even the nics were at different speeds. Smaller files don't touch the I/O at all, but a 200 mb file puts I/O at 100% and locks up the system. We are running 2.6.9-42.0.2 which was the most recent Kernel as I understan di.
 
Old 03-15-2007, 02:23 PM   #6
macemoneta
Senior Member
 
Registered: Jan 2005
Location: Manalapan, NJ
Distribution: Fedora x86 and x86_64, Debian PPC and ARM, Android
Posts: 4,593
Blog Entries: 2

Rep: Reputation: 332Reputation: 332Reputation: 332Reputation: 332
Are you seeing anything in your syslog or dmesg just prior to the problem occurring?
 
Old 03-15-2007, 03:24 PM   #7
rdelia
LQ Newbie
 
Registered: Jul 2006
Posts: 15

Original Poster
Rep: Reputation: 0
Not seeing anything unusual. I found in another thread where this is occurring for other people as well. I wonder if it is a known issue and Red Hat is working on it?
 
Old 03-15-2007, 03:39 PM   #8
ramram29
Member
 
Registered: Jul 2003
Location: Miami, Florida, USA
Distribution: Debian
Posts: 848
Blog Entries: 1

Rep: Reputation: 47
You may want to do the following:

-Boot to a LiveCD on both systems with a newer kernel such as Knoppix. Mount the folders and then try the transfer. If it works then you'll know that it's your software and not your hardware. If it doesn't work then it could be your hardware.

-Use 1000 Mbit/Gigabit NIC and Switches.

-Try using a null network cable to narrow down any switch or network problems.

-Even better, try it over a fiber connection.

-Try compressing the file as you send, you can do this using ssh and rsync.

-Try compressing the file before you send.

-Try splitting the file in chunks of 2GB as you copy. Afterward you can contactenate each chunk into the one big file. You can do this using the dd and split commands.

-Copying one mega file this big may cause overruns and errors. Think about it. Your system has to verify a all the bits in a 200GB file. This is one way of making your system get on it's knees.
 
Old 03-15-2007, 04:02 PM   #9
rdelia
LQ Newbie
 
Registered: Jul 2006
Posts: 15

Original Poster
Rep: Reputation: 0
ramram29,

All good suggestions, but we need to resolve the issue as is because the NFS mount needs to go to a 10/100 nic on a VMS server. We've eliminated the hardware so far and have isolated it to Red Hat software. We just need to find a way to successfully transfer 200mb and larger files using NFS from our LINUX 4 to a VMS server using NFS 2.
 
Old 03-15-2007, 04:19 PM   #10
rdelia
LQ Newbie
 
Registered: Jul 2006
Posts: 15

Original Poster
Rep: Reputation: 0
MACEMONETA,

What is the kernel version of Fedora and what is the RPM for NFS that you are running? Might help us.
 
Old 03-15-2007, 05:08 PM   #11
macemoneta
Senior Member
 
Registered: Jan 2005
Location: Manalapan, NJ
Distribution: Fedora x86 and x86_64, Debian PPC and ARM, Android
Posts: 4,593
Blog Entries: 2

Rep: Reputation: 332Reputation: 332Reputation: 332Reputation: 332
Fedora tends to run well ahead of Redhat:

kernel-2.6.20-1.2925.fc6
nfs-utils-1.0.10-5.fc6
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Samba file size limitation enygma Linux - Networking 2 02-07-2005 02:59 PM
Is there a file size limitation in Linux? lazyboy0001 Linux - Software 12 08-12-2004 08:28 PM
mkisofs, is there a max file size limitation? tsw Linux - Software 4 07-27-2004 04:10 PM
Maximum Size of File in Red Hat Linux 2.1 imsajjadali Red Hat 0 07-02-2004 09:05 AM
File size limitation? jbrashear Linux - General 7 04-03-2004 01:53 AM


All times are GMT -5. The time now is 06:23 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration