Hello,

Of course I can do that, but then I'll change the conditions. Keep in mind that if I disable SRC NAT and MASQUERADE action on it then I would not be able to go to different network. So I need to change whole configuration of network.
There are other options. My router itself has SSH server. Then I talk directly SSH client (172.16.5.109) <------> SSH server (172.16.5.254), but I get the same problem.
Also, if I use wired connection to router everything works just fine (even through router). But difference in this case is wired network and the fact that client and server are in the same network.
I don't know why you think this isn't normal when SSH server "talks" to router thinking that it is the one who initiated session. From SSH server point of view its no difference it just matter of IP addresses:

1. Client (172.16.3.109) sends packet out to 192.168.5.6
2. Router gets packet and make route decision
3. Router changes src address of packet to its own (192.168.5.254)
4. Router then sends this packet to SSH server (192.168.5.6)
5. Router responds to packet and sends response back to sender (192.168.5.254)
6. Router gets back packet and again make routing desicion
7. Router changes dst address to its original initiator (172.16.5.109)

Where you see problems here ? The same scenario is when you try to SSH to server which is on internet (WAN address). In that case SSH server will never talk to your internal network address (ie 192.168.x.x), but instead it will talk to router's WAN IP address and all other work will do router. For me its perfectly normal scenario. That's why we need routers.

Hello,

Now the happy end of long story if someone is interested....
The problem was not router, not packet routing, not SRC ir DST NAT, but WiFi drivers. Strange, but true.
I'm running Broadcom 4311 mini PCI card and it has lots of issues with Ubuntu. When I've detected my problem with SSH I had 'wl' drivers installed for this card. I had issue described in this thread. Later I've checked a lot of threads about this issue and I ran into installing 'ndiswrapper'. This is totally crap for Ubuntu 9.10. It is ok maybe for Hardy or older version, but not for Jaunty. So after a lot of searches I installed 'fwcutter bcm43xx' drivers and now it works like a charm. And I make absolutely no changes to router ! Not changes in firewall or NAT.
So I hope this thread will help for at least for some users running the same problems.