Recently I am working on some SFTP throughput issue and found that by increasing the rmem_max and wmem_max at the receiving server we were able to boost from a SFTP rate of 400KBps to 8.9MBps.
But what's weird is during the test, right after I ran echo command,
echo "16777216" > /proc/sys/net/core/rmem_max
echo "16777216" > /proc/sys/net/core/wmem_max
The speed increase was small. We got 1.5MBps. Then I had some other stuff to do and left the server along for like an hour. During which I know for sure no one touched it as it was a temp setup that I made just for that test. Then when we tried the test again we were able to get 8.9MBps now pretty consistently.
I was under the impression when I make such changes to kernal params it should take effect instantaneously. But clearly this is not the case here.
We are running Arch Linux, but when we tried the same thing on a SUSE10 server we got the same result. Immediately after applying the new kernel param we do not see the speed increase that we are expecting.