NAS drive slows to 3.5MB/s
 

I have a Buffalo Linkstation 210 here, got it for a good deal, and it works pretty good.

I have to mount the shares CIFS, either that or it's AFS. It does FTP and Web access so that's pretty nice too.

FTP loses mtimes and there's no way to set them. CIFS (SMB) loses Case distinction and owner/group info.

Both of these methods gets me about 35MB/s thoughput, which is entirely adequate for my needs.

I want timestamps AND owner/group permissions, so what I did was this:

I create a giant block device on the CIFS share, and then mount it as ext4 loop device. Works awesome, but the throughput is lame.

First 100MB or so gets about 25MB/s, then after that, it slows to about 3.5MB/s. This is on multi GB VOBs.

Any idea why the horrible throughput on files > 100MB? Can I do anything about it?

Sounds like something is backing up and your hardware drivers (or one of them) suck. 3.5MB/S is about the right speed for PIO4 disk access (from the last millennium), which means something is defaulting to a teetotally lame way of doing things.

Thanks for the reply.

Why after ~100MB? Some buffer on the NIC fills up at that point? The card is Intel Corporation 82566DM-2.

I added to the ext4 options field in /etc/fstab and throughput seems to have increased to ~7.5-10MB/s. That's a huge improvement already. Tolerable even. About 1/2 the speed of writing to the CIFS partition by itself.

I'll keep fiddling with options. I don't think the fact it's mounted as loop device makes much difference. loops are fast by nature I think.

As a comparison I have Synology NAS and copying a 512MB file over CIFS using rsync was about 14MB/s. Copying the same file with a ext4 loop device over CIFS was about 12MB/s. The loop device was mounted with the default options and the tested system runs CentOS 6.

All filesystem writes are over the network which is why when you lower ext4 overhead buy changing the options throughput goes up. There are two many variables to say why the difference on your system is much greater than mine. Switching to ext2 will probably get you the best results.

I just now created an EXT2 partition to test this theory, mounted it with defaults, and throughput dropped as low as 3.9MB/s. Then I remounted it with async,noatime and it was just as bad. Copy same file to the EXT4 partition and throughput never dropped below 7.9MB/s. Looks like EXT4 with those mount options are my best choice so far.

This raises a question: I created the block file with truncate, not dd. I just now heard of fallocate (which doesn't seem to work on my system. fallocate: fallocate failed: Operation not supported). mkfs should take care of any sparseness issues, I think. I wonder if that is some cause of this throughput issue?

So I test my theory:

I create 10G block file with dd on CIFS NAS share. It gets created at about 12.5MB/s. This looks like max speed for my NAS drive, although the first ~100MB goes at 30-40MB/s, so that looks like the network buffer filling up, maybe a 128MB chip on the NIC. Created EXT2 fs on the block file, mount with async. Same procedure and I still get throughput as low as 3.9MB/s, so using truncate instead of dd saves a lot of time, and works just as well.

And, for me, EXT4 gets 2x the speed of EXT2 in this situation. Enigmas wrapped in mysteries...

And, unfortunately, I cannot log into my nice NAS drive as root and tweak a few things. I'm stuck as some dummy named "admin" who makes sure the donuts and coffee are stocked up.

I tried another approach using two computers on my network. One with a GB network adapter and one with just a 100MB network adapter. A speed test between them using netcat was 11.9MB/s which is expected. Mounting a CIFS share on the 100MB adapter computer and creating a 1GB file took 11.7 MB/s. I then created a loop device with an ext4 filesystem. I mounted that and created a 512 MB file in the same manner and the results were 13.2MB/s. Doing the same thing with an ext2 filesystem the results were 17MB/s.

This test by using dd to create a file does not include the disk I/O of the client computer which might make my numbers higher then a real file transfer. Still working on why a remote loop filesystem is faster on top of CIFS...

There's details that are always hidden that affect the speed, such as block size of transfer, exsft mode used,vetc. It is possible to write one byte St a time but obviously this is not the fastest way to do things

I think it's the NAS drive. Maybe that's why it was on sale. With FTP, most servers support the MTIME extension, but this one doesn't. If it did, I wouldn't be doing this loopback/EXT4 thing. It's feature set looks like it's trying really hard to support Apple stuff, and they forgot about other people in the process.