tcp kernel setting for rmem_max and wmem_max not taking effect instantaneously

jzoudavy · 04-27-2016, 03:01 PM

Hi all

Recently I am working on some SFTP throughput issue and found that by increasing the rmem_max and wmem_max at the receiving server we were able to boost from a SFTP rate of 400KBps to 8.9MBps.

But what's weird is during the test, right after I ran echo command,

Code:

echo "16777216" > /proc/sys/net/core/rmem_max
echo "16777216" > /proc/sys/net/core/wmem_max

The speed increase was small. We got 1.5MBps. Then I had some other stuff to do and left the server along for like an hour. During which I know for sure no one touched it as it was a temp setup that I made just for that test. Then when we tried the test again we were able to get 8.9MBps now pretty consistently.

I was under the impression when I make such changes to kernal params it should take effect instantaneously. But clearly this is not the case here.

We are running Arch Linux, but when we tried the same thing on a SUSE10 server we got the same result. Immediately after applying the new kernel param we do not see the speed increase that we are expecting.

jpollard · 04-27-2016, 09:55 PM

Quote:

Originally Posted by jzoudavy

Hi all

Recently I am working on some SFTP throughput issue and found that by increasing the rmem_max and wmem_max at the receiving server we were able to boost from a SFTP rate of 400KBps to 8.9MBps.

But what's weird is during the test, right after I ran echo command,

Code:

echo "16777216" > /proc/sys/net/core/rmem_max
echo "16777216" > /proc/sys/net/core/wmem_max

The speed increase was small. We got 1.5MBps. Then I had some other stuff to do and left the server along for like an hour. During which I know for sure no one touched it as it was a temp setup that I made just for that test. Then when we tried the test again we were able to get 8.9MBps now pretty consistently.

I was under the impression when I make such changes to kernal params it should take effect instantaneously. But clearly this is not the case here.

We are running Arch Linux, but when we tried the same thing on a SUSE10 server we got the same result. Immediately after applying the new kernel param we do not see the speed increase that we are expecting.

Almost instantly - there is a delay imposed by existing open connections. There may be other things that can cause delays that I'm not familiar with.

The amount of delay depends on the parameter, and what may be active using the facility controlled by the parameter.