LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices


Reply
  Search this Thread
Old 05-24-2004, 05:16 PM   #1
BrianK
Senior Member
 
Registered: Mar 2002
Location: Los Angeles, CA
Distribution: Debian, Ubuntu
Posts: 1,334

Rep: Reputation: 51
Which is "better" - NFS or Samba?


I will be making almost entirely Linux to Linux connections for sharing files. I have one big file server and lots of client machines that will read from & write to the file server at the same time - usually large amounts of data. I've been running NFS in sync mode, but have had BIG speed issues (way too slow), so I'm thinking about switching to Samba. I've tried running NFS in async mode for speed, but have run into sync problems as one machine expects a file that another machine (or even the same machine) has written, but it's not complete yet - causing the whole operation to fail.

Is there a reason that I shouldn't switch to Samba?

Is NFS superior or inferior in certain aspects, but not in others?

FYI: All my machines have GigE & run through a single GigE switch. The file server runs RAID 5 on 4 drives, has a separate system drive, and a good amount of RAM, so it should be able to keep up with pretty high network traffic.

Thanks!
 
Old 05-25-2004, 05:10 AM   #2
maxut
Senior Member
 
Registered: May 2003
Location: istanbul
Distribution: debian - redhat - others
Posts: 1,188

Rep: Reputation: 50
i have a very little 100mb network. u use samba cos there are win macines in my network. it works very good. maybe there is a trouble with your network not NFS. have u checked your network? at least type ifconfig to see if there are some error packets. so u will be sure if there s an trouble with NFS.
 
Old 05-25-2004, 05:18 PM   #3
BrianK
Senior Member
 
Registered: Mar 2002
Location: Los Angeles, CA
Distribution: Debian, Ubuntu
Posts: 1,334

Original Poster
Rep: Reputation: 51
no, there doesn't appear to be any packet loss.

Samba seems to operate at about the same speed as NFS in sync mode. fooey.

This is incredibly frustrating. I can transfer files via ftp or scp MUCH faster than through Samba or NFS - if NFS or Samba were on par with ftp or scp, I'd be happy. I'm sure there's some sort of something I'm not doing correctly that's killing my speed.
 
Old 05-26-2004, 12:37 AM   #4
MS3FGX
LQ Guru
 
Registered: Jan 2004
Location: NJ, USA
Distribution: Slackware, Debian
Posts: 5,852

Rep: Reputation: 357Reputation: 357Reputation: 357Reputation: 357
Well, FTP is usually faster than Samba of NFS, just because it is a simpler protocol.

But what kind of speed are we talking about here? How long does it take to transfer 100MB, and how long for 1 GB?

It is hard to say if anything is really wrong unless we can get an idea of how fast (or slow) transfers really are.
 
Old 05-26-2004, 12:53 AM   #5
ppuru
Senior Member
 
Registered: Mar 2003
Location: Beautiful BC
Distribution: RedHat & clones, Slackware, SuSE, OpenBSD
Posts: 1,791

Rep: Reputation: 50
NFS coupled with NIS; life would be a lot more easier than Samba. If you plan to have Microsoft products on your LAN, you would need Samba too.
 
Old 05-26-2004, 02:41 PM   #6
BrianK
Senior Member
 
Registered: Mar 2002
Location: Los Angeles, CA
Distribution: Debian, Ubuntu
Posts: 1,334

Original Poster
Rep: Reputation: 51
Quote:
Originally posted by MS3FGX
Well, FTP is usually faster than Samba of NFS, just because it is a simpler protocol.

But what kind of speed are we talking about here? How long does it take to transfer 100MB, and how long for 1 GB?

It is hard to say if anything is really wrong unless we can get an idea of how fast (or slow) transfers really are.
I've recently moved the file server from RH7 to RH9. On the RH7 box, the fastest speed I ever saw under NFS was about 22-25 seconds for 256 MB (I think that was in async mode as well). Now I'm seeing 8-9 seconds for the same file (in NFS sync mode) & just over 30 seconds for 1 GB.

That's not such a bad speed & quite a bit better than before, but still is running around 25-28MB/s. 25-28 MB/s on a 1 Gb/s network seems on the slow side. I understand that 1 Gb/s means 100 MB/s, but shouldn't I be seeing a little bit better than what I am?

Last edited by BrianK; 05-26-2004 at 02:43 PM.
 
Old 05-26-2004, 03:36 PM   #7
MS3FGX
LQ Guru
 
Registered: Jan 2004
Location: NJ, USA
Distribution: Slackware, Debian
Posts: 5,852

Rep: Reputation: 357Reputation: 357Reputation: 357Reputation: 357
There is some overhead, but it does seem like you should be getting a bit faster speeds than that. You aren't even hitting 50% of the theoretical maximum.

You are sure that all of the machines are using Cat6? Perhaps a Cat5 got in there?
 
Old 05-26-2004, 04:05 PM   #8
BrianK
Senior Member
 
Registered: Mar 2002
Location: Los Angeles, CA
Distribution: Debian, Ubuntu
Posts: 1,334

Original Poster
Rep: Reputation: 51
Quote:
Originally posted by MS3FGX
There is some overhead, but it does seem like you should be getting a bit faster speeds than that. You aren't even hitting 50% of the theoretical maximum.

You are sure that all of the machines are using Cat6? Perhaps a Cat5 got in there?
Everything is using CAT 5e excpet the file server, which is using CAT 6.

The switch is made by Netgear - it's unmanaged - just a basic 24 port GigE switch. Not the best in the world, but I would assume that it would at least hit 50%.
 
Old 05-26-2004, 08:43 PM   #9
mlp68
Member
 
Registered: Jun 2002
Location: NY
Distribution: Gentoo,RH
Posts: 333

Rep: Reputation: 40
I don't think you said this - do you run v2 or v3?

I'm running a similar setup, gig network, several file servers, and I have seen NFS transfer speeds of more than 50MB/s. I think with the complexity of NFS, that is close to the max you can get.

By the way, I use the async option, and the clients have the "hard" option, and I don't seem to have problems with NFS errors, ever, and I shuffle a lot of data.

Sounds like you have enough internal bandwidth in your server for the disks, but another possibility is that your network card and the raid card share a PCI bus, then you'd get some arbitration issues when you have lots of connections via NFS to different files (as opposed to a single file access with ftp which you said worked fine).

Finally, how many nfsd threads are you setting up?

For more advice, we'd need to see some config facts...

Good luck,
mlp
 
Old 05-27-2004, 12:47 PM   #10
BrianK
Senior Member
 
Registered: Mar 2002
Location: Los Angeles, CA
Distribution: Debian, Ubuntu
Posts: 1,334

Original Poster
Rep: Reputation: 51
Quote:
Originally posted by mlp68
I don't think you said this - do you run v2 or v3?

I'm running a similar setup, gig network, several file servers, and I have seen NFS transfer speeds of more than 50MB/s. I think with the complexity of NFS, that is close to the max you can get.

By the way, I use the async option, and the clients have the "hard" option, and I don't seem to have problems with NFS errors, ever, and I shuffle a lot of data.

Sounds like you have enough internal bandwidth in your server for the disks, but another possibility is that your network card and the raid card share a PCI bus, then you'd get some arbitration issues when you have lots of connections via NFS to different files (as opposed to a single file access with ftp which you said worked fine).

Finally, how many nfsd threads are you setting up?

For more advice, we'd need to see some config facts...
Thanks for helping out with this...

# rpcinfo -p | grep nfs
100003 2 udp 2049 nfs
100003 3 udp 2049 nfs

# ps -e | grep nfs
18363 ? 00:00:14 nfsd
18364 ? 00:00:18 nfsd
18365 ? 00:00:15 nfsd
18366 ? 00:00:15 nfsd
18367 ? 00:00:16 nfsd
18368 ? 00:00:19 nfsd
18369 ? 00:00:15 nfsd
18370 ? 00:00:14 nfsd

# cat /etc/exports
/mnt/farm 192.168.0.0/255.255.255.0(rw,no_subtree_check,sync)


// on the client side:
# cat /etc/fstab
uranium:/mnt/farm /mnt/farm nfs bg,noac,rw,hard,rsize=16384,wsize=16384 0 0



So...
nfs v2 & v3
8 nfsd threads

re: async - I didn't really run into any nfs errors with async, I ran into problems with files not being completely written before they were accessed.... This is a render farm. Some of our renders create a shadow pic for each frame which then gets immediately used by the frame being rendered. The shadow pics weren't done writing before the renderer locked the file to read it - which made the file invalid & unusable. Result: no shadows. Switching back to sync fixed the problem but made the nfs writes horribly slow. (this was on the old RH7 file server - haven't run the test on the new box yet).

re: pci bus - I suppose it's possible & I'm not quite sure how to check. The NIC is built into the motherboard. The mobo runs the 865G chipset with ICH5 & Intel 82547EZ PLC chip for Gigbit LAN. The RAID card is plugged into a PCI slot on the board.

Thanks again for taking a look at this. ANY suggestions are greatly appreciated (being that I don't know of many other resources to help at this point).

Last edited by BrianK; 05-27-2004 at 12:59 PM.
 
Old 05-28-2004, 01:42 PM   #11
mlp68
Member
 
Registered: Jun 2002
Location: NY
Distribution: Gentoo,RH
Posts: 333

Rep: Reputation: 40
Well, yes, I think the sync bit gets you. Just for completeness, see what you get by adding a nfsvers=3 to the clients, and increase the r/wsize to 32k. I think that when given a choice, the partners negotiate v3, but you want to make sure.

lspci -v will list who's who on which bus.

I don't know what renderer you are using, but maybe you can provide some external means to control the access? Some lock file, or moving it into a different directory or rename it to indicate a file is done, etc. Just out of curiosity, you seem to farm out individual frames. Could you process one frame beginnig to end on one machine and move the final image to the NFS area?

Hope it helps somewhat.
mlp
 
Old 05-28-2004, 06:41 PM   #12
BrianK
Senior Member
 
Registered: Mar 2002
Location: Los Angeles, CA
Distribution: Debian, Ubuntu
Posts: 1,334

Original Poster
Rep: Reputation: 51
Quote:
Originally posted by mlp68
Well, yes, I think the sync bit gets you. Just for completeness, see what you get by adding a nfsvers=3 to the clients, and increase the r/wsize to 32k. I think that when given a choice, the partners negotiate v3, but you want to make sure.

lspci -v will list who's who on which bus.

I don't know what renderer you are using, but maybe you can provide some external means to control the access? Some lock file, or moving it into a different directory or rename it to indicate a file is done, etc. Just out of curiosity, you seem to farm out individual frames. Could you process one frame beginnig to end on one machine and move the final image to the NFS area?
Thanks for the suggestions - I'll give 'em a try later this evening.

as far as rendering - we're using Mantra (Houdini's renderer). It is possible to move the scene description to the local machine, render it, then move the rendered frame to the file server. I'm not sure if that would help things out speedwise being that the moves would have to complete before the render or next frame could start as opposed to reading, writing & rendering at the same time & with scene description files that often get upwards of 50-60MB, it might be a considerable bottleneck (even worse if the frame requires multiple shadow pics, zdepth pics, etc.). Granted, I could use async in that case. Worth a try, for sure.

Again, thanks for the suggestions.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Telling people to use "Google," to "RTFM," or "Use the search feature" Ausar General 77 03-21-2010 11:26 AM
NFS problems - "/usr/sbin/exportfs" just hangs meeble Linux - Networking 4 07-03-2008 07:14 AM
"Xlib: extension "XFree86-DRI" missing on display ":0.0"." zaps Linux - Games 9 05-14-2007 03:07 PM
Unmounting "broken" NFS/SMB mounts safely darkfame Linux - Networking 1 12-11-2003 06:13 PM
NFS "request from unauthorized host" and Program not registred hrasmus Mandriva 5 10-25-2003 02:07 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Networking

All times are GMT -5. The time now is 04:26 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration