[SOLVED] rsync+ssh: chain 4 boxes together

chrism01 · 08-02-2012, 01:16 AM

Hi Guys,

network layout (all RHEL5 systems):

box1 ---> box2 ---> box3 ---> box4

Need to rsync+ssh ie transparent pipe from 1->4 (going via box2 & box3 as per diagram).
Same user on all and ssh-auth-keys enabled & tested ok.
Reckon I need to use ssh tunnels, but can't quite get it working.

I'm using setups like this eg on box1

Code:

nohup /usr/bin/ssh -nNTx -L 8890:box2:8890 user@box2 &

& so on for box2 & box3

Edit: cmd

Code:

rsync -avz /home/user/test.dat --progress --inplace  --rsh='ssh -p8890' user@box1:/home/user

Eventually it times out with

Code:

ssh: connect to host box1 port 8890: Connection timed out
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(600) [sender=3.0.6]

See also as run on box1

Code:

rsync -avz /home/user/test.dat --progress --inplace  --rsh='ssh -p8890' user@box2:/home/user
ssh: connect to host box2 port 8890: Connection refused
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(600) [sender=3.0.6]

Strangely enough, I've never had to implement an ssh tunnel and chaining 4 systems together is confusing me

I think I need to define a listener on box4, but I'm not sure what, if any?

Thanks for any solns

evo2 · 08-02-2012, 03:35 AM

Hi,

you seem to be trying to do this the "old fashioned" way with local port forwarding. It's much easier (especially with multihop) to used the "ProxyCommand" in your ~/.ssh/config
Here is how I would do what you are trying to achieve.

Add the following to your ~/.ssh/config on box1

Code:

Host box4
     ProxyCommand ssh -qax -W %h:%p box3

Host box3
     ProxyCommand ssh -qax -W %h:%p box2

Now from box1 you should be able to get to box4 just by doing:

Code:

ssh box4

and rsync should work just the same.

One caveat:
The -W option was introduced to ssh somewhat recently (sorry I don't know the version number).
So if you boxen don't have ssh with -W you need to use a slight hack with netcat.

In this case you would have lines like:

Code:

Host box4
     ProxyCommand ssh -qax box3 nc %h %p

If you have different usernames on the different boxen just declare the username for each box in its section. Eg

Code:

Host box4
   User box4username
   ProxyCommand ssh -qax -W %h:%p box3

HTH,

Evo2.

anomie · 08-02-2012, 11:40 AM

I like evo2's solution.

I was also going to suggest a simple TCP forwarding service (e.g. rinetd) to accept connections from box1 -> box2, from box2 -> box3, and from box3 -> box4. IMO, it's not strictly necessary to authenticate at every host along the way (and the channel is already encrypted). rinetd can be configured for IP-level access control.

chrism01 · 08-02-2012, 07:57 PM

I like the idea of evo2's soln as well.
So long as it gets encrypted before it starts being transmitted, that's good enough.
(As it happens, I've never had to actually setup rsync or ssh tunnels before... )

Anyway, I put the proxy cmds in (using nc as the target systems are RHEL5.0; test is on 5.7), which is simpler anyway.
I've also copied the (dsa) ssh keys ie user1@box1 to the same user on box2, box3, box4.
As this is a test LAN, the boxes can all see each other direct, so I used /etc/hosts.deny on box4 to deny sshd from box1, box2 to enforce/check I'm using the proxies.

It all good so thx a lot(!) to evo2.

PS as it happens, the App team will not be using this now, but I'm documenting the technique anyway, for future ref for myself

evo2 · 08-02-2012, 09:19 PM

Hi Chris,

you say that you have copied the "keys" plural. You should only need to copy one key file: the dsa public key, and append its contents to ~/.ssh/authorized_keys file on boxs 2-4. Using ssh-copy-id normally works well for this. Eg

Code:

ssh-copy-id -i ~/.ssh/id_dsa.pub box2

You may get the most pertinent information from the sshd logs. If you don't have access to these you can start up your own sshd instance in debug mode on an alternative port. Eg log into box3 and run

Code:

/usr/sbin/sshd -d -p 9999

then put a Port option section in your box1:~/.ssh/config

Code:

Host box3
  Port 9999
  IdentityFile ~/.ssh/id_dsa
  ProxyCommand ProxyCommand ssh -qax box2 nc %h %p

Then try again.

Finally (and I think this is a long shot) I wonder if it would make any difference if you were explicit about what private key you want to use. Eg

Code:

Host box3
  IdentityFile ~/.ssh/id_dsa
  ProxyCommand ProxyCommand ssh -qax box2 nc %h %p

HTH,

Evo2.

chrism01 · 08-02-2012, 10:24 PM

Aargh, must have left this thread open, so I edited the above ... sorry.
Anyway, as you can see, you are essentially right.
Actually, I'm now using a much simpler rsync cmd, with the above technique (as credited to you)

Code:

[user@box1 ~]$ rsync -av test.dat -e ssh --progress user@box4:/home/user
sending incremental file list
test.dat
          19 100%    0.00kB/s    0:00:00 (xfer#1, to-check=0/1)

sent 112 bytes  received 31 bytes  95.33 bytes/sec
total size is 19  speedup is 0.13

& it works a treat

Thx again

evo2 · 08-03-2012, 12:18 AM

Hi,

glad I was able to help, you can put it away in your bag of tricks for later.

It's refreshing to help with a "proper" question... trying to help out over on the Newbie forum can be become quite frustrating after a while.

Evo2.

chrism01 · 08-05-2012, 06:01 PM

I hear that

Stick around, I may have qns on KVM later; trying to teach myself at home and got messed up with the networking part; managed to cut myself off from the internet

Section 2.6 http://linux.dell.com/files/whitepap..._made_easy.pdf

I'll open a new thread for the qns..