rsync protocol version mismatch -- is your shell clean

Lucio Chiappetti · 11-01-2021, 05:28 PM

I'd like to set up periodical reciprocal synch for a small number of files under a service account between my machine A at home (xubuntu 20.04 rsync 3.1.3 protocol version 31) and my machine B at work (opensuse leap 42 rsync 3.1.0 protocol version 31). By reciprocal I mean that the most recent file shall be transferred A to B or B to A.

I do not want to setup a rsync daemon (which I did long ago Europe to Chile), and cannot use an NFS mount now (machine A has a dynamic IP) like I did when machine A was at work on the work LAN.

So I discovered in the man page the possibility to tunnel rsync via ssh.

I was actually pleasantly surprised when I tried this ls-like command logged as myself on A

rsync B:/path

this prompts for my account password on B, and lists the files. I was surprised because ssh myself@B is protected by a knock daemon, but I noticed in /var/log/messages on B rsync seems to be working on a dedicated port.

rsync B:/path .
correctly fails with a write error on ~service

So I tried both an ls-like and a local copy (so far still logged in as myself,
I expect the second to fail with a write permission ... service@A has still an empty home, I'd like just to import stuff from B before logging in as service)

rsync service@B:/path
rsync service@B:/path .

I provide the service password (which should be used in clear since it is unimportant with sshpass or some other tricks suggested on the net) but I get for both commands

protocol version mismatch -- is your shell clean?

Now the version of rsync is slightly different but protocol is 31 for both, so it is not clear to me how to proceed. I am also confused by the difference between rsync user service@B and ssh user -e "ssh -l service"

(currently on B the service account is invoked as su - service, and its .cshrc spawns an utility program; on A I'd like when all is setup to invoke it in the same way, but it should (1) rsync files frm B to A; (2) run the utility on A ; (3) rsync files from A to B ... on both machine the utility is interactive, namely it interacts with the terminal)

evo2 · 11-01-2021, 05:59 PM

Hi,

Quote:

Originally Posted by Lucio Chiappetti

protocol version mismatch -- is your shell clean?

sounds like one of the dot files for your shell on the remote host is outputing something even though it is not an interactive shell. What do you see if you just ssh into the remote machine? What shell are you using there?

Evo2.

HappyTux · 11-01-2021, 06:35 PM

Quote:

Originally Posted by Lucio Chiappetti

Now the version of rsync is slightly different but protocol is 31 for both, so it is not clear to me how to proceed. I am also confused by the difference between rsync user service@B and ssh user -e "ssh -l service"

(currently on B the service account is invoked as su - service, and its .cshrc spawns an utility program; on A I'd like when all is setup to invoke it in the same way, but it should (1) rsync files frm B to A; (2) run the utility on A ; (3) rsync files from A to B ... on both machine the utility is interactive, namely it interacts with the terminal)

The default for rsync is to use the ssh when going over a network no need to tell it to do so with option passed. What has always worked for me is rsync -avP /path/to/sync user_name@123.456.789.012:/path/to/ which would make an exact copy of the sync directory on the machine you are copying too. Only the file difference(s) will ever be copied by the command, this is how rsync works by default. I fail to see what the service in your commands is supposed to do.

Code:

MacUser2525:~$ mkdir test
MacUser2525:~$ touch test/file1.txt
MacUser2525:~$ touch test/file2.txt
MacUser2525:~$ ls -l test/
total 0
-rw-r--r--  1 MacUser2525  staff  0  1 Nov 20:20 file1.txt
-rw-r--r--  1 MacUser2525  staff  0  1 Nov 20:21 file2.txt
MacUser2525:~$ rsync -avP test zeus@192.168.0.116:
sending incremental file list
test/
test/file1.txt
              0 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=1/3)
test/file2.txt
              0 100%    0.00kB/s    0:00:00 (xfr#2, to-chk=0/3)

sent 200 bytes  received 58 bytes  516.00 bytes/sec
total size is 0  speedup is 0.00

The destination machine.

Code:

zeus@bullseye-raspi:~$ ls -l
total 8
drwxr-xr-x  3 zeus zeus 4096 Aug 20 13:23 bin
drwxr-xr-x 13 zeus zeus 4096 Sep 13 19:05 rtorrent
zeus@bullseye-raspi:~$ ls -l
total 12
drwxr-xr-x  3 zeus zeus 4096 Aug 20 13:23 bin
drwxr-xr-x 13 zeus zeus 4096 Sep 13 19:05 rtorrent
drwxr-xr-x  2 zeus zeus 4096 Nov  1 20:21 test
zeus@bullseye-raspi:~$ ls -l test/
total 0
-rw-r--r-- 1 zeus zeus 0 Nov  1 20:20 file1.txt
-rw-r--r-- 1 zeus zeus 0 Nov  1 20:21 file2.txt
zeus@bullseye-raspi:~$ touch test/file3.txt
zeus@bullseye-raspi:~$ ls -l test/
total 0
-rw-r--r-- 1 zeus zeus 0 Nov  1 20:20 file1.txt
-rw-r--r-- 1 zeus zeus 0 Nov  1 20:21 file2.txt
-rw-r--r-- 1 zeus zeus 0 Nov  1 20:25 file3.txt
zeus@bullseye-raspi:~$ rsync -avP test MacUser2525@192.168.0.101:
sending incremental file list
test/
test/file3.txt
              0 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=0/4)

sent 180 bytes  received 39 bytes  146.00 bytes/sec
total size is 0  speedup is 0.00

Now back on the original machine.

Code:

MacUser2525:~$ ll test/
total 0
-rw-r--r--  1 MacUser2525  staff  0  1 Nov 20:20 file1.txt
-rw-r--r--  1 MacUser2525  staff  0  1 Nov 20:21 file2.txt
-rw-r--r--  1 MacUser2525  staff  0  1 Nov 20:25 file3.txt

Now as you can see I created test directory on my Mac for the testing. It contained two files created using touch. The before and after is shown on the destination machine with now the test directory present when checking it after the sync operation containing the files. I make another file on the destination and do the rsync operation back to the original and only the third file is copied back. This is how you use rsync to copy files between machines. Oh you may have noticed the lack of a password requested to do the operation I had the ~/.ssh/id_rsa.pub for each machine in the ~/.ssh/authorized_keys of the other machine, the ~/.ssh/id_rsa.pub was created without setting a password and allows for a passwordless transfer between them.

Edit2: Oh almost forgot. The versions of rsync used a clear mismatch in the release numbers but not the protocol.

Code:

zeus@bullseye-raspi:~$ rsync --version
rsync  version 3.2.3  protocol version 31
Copyright (C) 1996-2020 by Andrew Tridgell, Wayne Davison, and others.
Web site: https://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, hardlink-specials, symlinks, IPv6, atimes,
    batchfiles, inplace, append, ACLs, xattrs, optional protect-args, iconv,
    symtimes, prealloc, stop-at, no crtimes
Optimizations:
    no SIMD, no asm, openssl-crypto
Checksum list:
    xxh128 xxh3 xxh64 (xxhash) md5 md4 none
Compress list:
    zstd lz4 zlibx zlib none

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.

MacUser2525:~$ rsync --version
rsync  version 3.1.3  protocol version 31
Copyright (C) 1996-2018 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, iconv, symtimes, no prealloc

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.

michaelk · 11-01-2021, 07:55 PM

Quote:

(currently on B the service account is invoked as su - service, and its .cshrc spawns an utility program;

I agree with evo2 and also guessing the error is due to the utility program running. Try disabling the utility program then try rsync.

Lucio Chiappetti · 11-02-2021, 07:06 AM

I confirm your and evo2's suggestion. The utility is indeed producing terminal output (the purpose of a captive account was indeed to run it). Disabling it, rsync behaves nicely. Now I guess I'll play with conditional running of the utility only in interactive mode (BTW, the shell is tcsh, I thought I said it).

I will study HappyTux example later, but it should be what I meant (so far I used rsync mainly in a master-slave approach, i.e. main to backup, or development to production, or master to mirror). BTW. the word "service" was a placeholder for the name of the captive account. What the utility does is irrlevant here.

Thanks everybody

HappyTux · 11-02-2021, 11:26 AM

It is simple to go both ways with the sync you just have to be certain you use it before any changes are done when preparing to work on files on the second machine. Or if having just finished working on the files use it to send to the other so you always have them ready to be worked on in an equal newly synced state.