I am going to try a lot of these suggestions when I get a chance. I appreciate the help.

For the moment I have realized that if I SSH into an external server (my old university's server) and then SSH from there into my local network server (by forwarding port through my router), then it works perfectly fine.

This is deeply disturbing, but it's usable

The plot thickens. I hope you'll keep us updated in this thread.

1 members found this post helpful.

I've done some preliminary testing and found something interesting.

The problem is not that packets are getting lost. The problem is that there appears to be a long delay when my router delivers packets to both my laptop and my local server. This delay is so long that it sometimes triggers re-transmission of packets creating the appearance of dropped packets. However, if you actually follow the trace, you find that every packet is eventually sent (more or less).

To figure this out, I did the following:

1. I opened an SSH session from my laptop to my server
2. I then ran tcpdump in three places: my laptop, my router, and my local server
3. I typed the letter 'h' in the SSH session and waited for it to appear in the terminal
4. I stopped the tcpdump processes.

I then opened the captured activity in wireshark and set the time to display the offset from the first packet sent by my laptop (I made sure the system clocks were all more or less synchronized so that this was possible). The resulting data makes it pretty clear what is happening:

Laptop tcp dump

Router tcp dump

Server tcp dump


- Packet A is passed from laptop to router normally
- There is a 0.8 sec delay before packet A is passed from router to server
- Packet B is passed from server to router normally
- There is a 0.9 sec delay before packet B is passed from router to laptop
- Packet C is passed from laptop to router normally
- There is a 0.9 sec delay before packet C is passed from router to server

So it looks like the router is delaying delivery of packets for some reason. The fact that the delivery delay happens both with laptop to server and server to laptop path is making me think that the problem lies with my router. No idea what could be causing this. Maybe my router de-prioritizes intra-network communication for some reason?

Some other things I still want to try:

1. Check what tcpdump looks like when I SSH into an external server (which has no delay)
2. Check if this delivery delay happens for TCP laptop-server communication on other ports

I mentioned QOS before...do you happen to have that enabled? What kind of router? And (to start with simple things first), have you just tried to reset your router, or flush the ARP tables on it?

1 members found this post helpful.

Sorry, I forgot to respond. It's a Synology RT1900ac. None of the QoS features are enabled as far as I can tell. I have restarted it and my problem still happens. I am not sure how to flush the ARP tables.

Typically those routers have web interfaces where you can reset things..not sure where it is on your router, though. From what I saw there is an "Intrusion Detection" system available/installed on those devices, and have seen several reports of that pegging the CPU during some operations. Again, grasping at straws...since you can SSH in from OUTSIDE your home network with no problems, that leaves things that are LAN only to look at, as a diagnostic start.

1 members found this post helpful.

Probably my last followup:

After some other tests:

- SSH is still slow when run over another port
- TCP packets are sent instantaneously in other contexts
- SSH is instantaneous when coming from an external server.

These three facts together seem impossible unless my router is intentionally slowing down SSH packets but only when they come from within the network. That seems very unlikely. I suspect something weird is happening with my router, but it's beyond my ability to fix. Thanks for the help