How fast should scp be?
 

scp seems incredibly slow, and I'm wondering why.

With an i3 laptop --> RazPi 4, I'm getting ~1MB/S, but the ETA for a 490MB file is 40 minutes :eek: Either way, the transfer rate is poor. top reports all processes running in single figures. It's dawdling along. Both are wifi connections over local network and capable on multiples of the speed. The router has never been an issue before. They are USB drives, but each is good for ≥20MB/S.

Does scp just suck?

(Slightly off-topic, but touches on the speed of scp as well)
Deprecating scp @LWN.net
Have you seen this article?

no, I don't think so. It depends on the network and also the storage devices too. Not that simple.

I agree it isn't that simple.

How fast is your LAN wifi? You can measure it using iperf between the two computers.

How fast is your WiFi i.e AC,N etc and are you using 5GHz on both or just 2.4GHz.

How many devices are on your LAN operating at the same time?

What about the physical factors? distance what what lies between the computer and the router, channel interference, other non computer 2.4GHz or 5GHz device interference etc.

scp does have some limitations due to its small buffering but I don't have a problem with small files. Have you tried using sftp or rsync as a comparison?

1. source: your transfer depends upon how fast you can read the data. What is the source and how fast can you read data from it? Have you enough memory buffers free to be able to mass load blocks and stage them for transfer? (more buffers cover for more delays in other parts of the process and result in faster overall transfer.)

2. Network: what is the connection? How many interfaces does the transfer have to traverse to the target? Are the MTU settings compatible along the way or are you getting packet fragmentation on larger packets? Fragmentation can slow your transfer significantly.

Are all of the interfaces negotiating the same communication standards (1000BT-Full is good, if one negotiates 1000BT-Half you will see errors pile up and the transfer slow WAY down.) This could slow things down MUCH more than a little fragmentation!

Are your channels loaded up with other traffic, or is traffic moving through a hub that is a packet collision zone? Is the load on one of the devices along the way (routers, switches, hubs) loading up that device until it throttles the transfer?

Are you, at one end or the other throttling data either by intent or as a side-effect of some other setting you have modified?

If speed or security is at issue, use wire. If you are using wifi a microwave, blender, the neighbors table saw, air conditioner, refrigerator, or other houses wifi networks can cause issues. Nothing filters out man-made lightning interference better than twisted pair behind a shield.

3. Target: in addition to the network settings, how fast can your target machine/device save that data to media? How are the buffers, are there enough to load up and cover for delays on the receiving end?

I was wondering about this, too. In my experience transfers through SSH start fast but the speed rapidly decays to be ridiculously slow even over LAN (I don't know if the apparent high speed at the beginning is a miscalculation of the algorithm involved).

I would like to use SSH for secure transfers but especially for transfers from/to outside of LAN I find myself using vsftpd.

@mmichaek and @wpeckham posted their messages while I was writing mine. In my case, using ftp on the same target consistently gives high speed, so I think it must be something about the protocol.

FTP is not a secure protocol, but if you are transferring over your own secured wire who cares. One might run a test.

I know, that's the tragedy here :). These are not critical files so I wouldn't worry too much if someone spied on them but transfer times with SSH are so horrible that I end up losing patience.

I use scp all the time. I get (depends) 80 - 120 MBps transfer rate. I think that some of the older machines that I have (dual core) won't deliver data from the hard drive as fast as the Hazwell + machines. And, I only have a mix of Cat5e and Cat5e 350Mhz cable on the LAN. That may be slowing things down too. CAT5e won't deliver as fast as CAT6. I have gigabit devices but only CAT5e cables.

Yes, the bandwidth is shared between all devices talking. If nothing else is on I can get 120MBps.
So it looks like that CAT5e isn't hurting things too much. That is about as fast as my devices will go. I haven't had any problem with scp. I don't have any 10GB devices.

I do have machines that will start transferring at 120MBps for a bit and then will drop to 80 or even 60 for a bit. I don't know why. I assumed that the machines hard drive simply won't deliver bits that fast, or that CAT5e cable is not good enough to allow that fast of a signal, and the emf is inducing itself into its neighbors, causing corruption, and the speed auto negotiates downward, which is built into the TCP/IP protocol.

I do still have a machine with a 100MBit network device in it and it will only receive data at 12MBps, but that is because it is only 100Mb.

As another reference my wired lan iperf3 speed is 940Mbits/s between two computers, one is a dual core and the other is an I3.

Transferring a ~2.3GB ISO file via scp was 99.1MB/s = 792Mbits/s.

Transferring the same file via sftp was 105.2MB/s = 841Mbits/s

Thanks for reply. I set up an rsync server, but that was no better - 1.1MB/S. I memory serves, I used rsync once for an online job and it flew along. So my setup sucks - Oh well. I'm banging around movies, but I'm actually better downloading them where I want them instead of transferring stuff about. The RazPi only had realVNC last time I checked, but I didn't set up for it, because I had to open an account, give them a very intrusive amount of details, and at some point a middle finger arose …

Took me a bit to find the Pi 4 benchmark numbers. Yes, your setup sucks...

A Pi 4 wireless adapter can connect either 2.4 GHz or 5 GHz 802.11b/g/n/ac
I would expect something close to the posted numbers of 58 Mbps for 2.4GHz or 114 Mbps for 5GHz for iperf3 results.

https://magpi.raspberrypi.org/articl...ecs-benchmarks

I'll throw a temporary ftpd server on one of the boxes here, and try that just to confirm that it's the router, which I'm pretty sure it is. It actually matters little. The RazPi actually situation is weird, but you hardly want the shaggy dog story.

If it's the router, that's not my headache.

My attention also turned to my RazPi, which actually showed 2 wifis. The onboard one is compromised, having a Faraday shield beside it, but was refusing to be disabled.

So nearly uniquely I went at it, not to make it work, but to make it not work. Renaming the firmware did the trick where a direct instruction in config.txt failed. It's now an amnesiac question mark looking for it's firmware and not finding it. I now have one respectable wifi unit, and my transfer speed has increased 2-3 fold, ignoring all clocks and progress indicators thrown up which only contradict each other anyhow.