[SOLVED] How to copy file from remote host to local host then delete from remote host

legendmac · 05-26-2015, 02:57 PM

I have an expect script to SSH to a remote host and obtain some user inputs and information about the server/network configuration. The responses are saved in a text file that I then need to copy to my local host so that I can read the lines into variables for use in the parent shell script.

Is there a way to do this without needing to enter the username and password for the local host to use function scp? I have tried the following in my expect script to no avail:

Code:

spawn scp $usr@$host:$flnm .
expect {
	-re "(.*)assword:" { 
		send -s "$pswd\r"
	}
}

I have also tried to directly scp the file and enter the username and password to try to debug the issue, and that doesn't work either:

Code:

spawn scp file.txt user@host:file.txt
expect {
	-re "(.*)assword:" {
		send -s "password\r"
	}
	"you sure you want to continue connecting" {
		send -s "yes\r"
		exp_continue
	}
}

In both scenarios I have used exp_internal 1, and there are no errors. But I do not end up with the file on my local host.

Following the copy, I would like to delete the file from the remote host. Any suggestions on how to accomplish this?

ShadowCat8 · 05-26-2015, 04:55 PM

Greetings,

How about just writing the file to a location that you can rsync from to the local machine? And, rsync can delete the file after it has synced it locally. To do so, you would first need to set up a location in the filesystem to set as the rsync point in /etc/rsyncd.conf.

Code:

[monitor]
     path = /tmp/monitor
     comment = Data for systems monitor

Then it's just a matter of adding the rsync line to your script:

Code:

...
   rsync -a --remove-source-files rsync://ip.of.host.system:873/monitor/data.file /local/path/ 
...

Hope that helps.

schneidz · 05-26-2015, 05:18 PM

Quote:

Originally Posted by legendmac

I have an expect script to SSH to a remote host and obtain some user inputs and information about the server/network configuration. The responses are saved in a text file that I then need to copy to my local host so that I can read the lines into variables for use in the parent shell script.

Is there a way to do this without needing to enter the username and password for the local host to use function scp? I have tried the following in my expect script to no avail:
...

you're doing it wrong. setting up ssh with key based authentication is the way to go.

jlinkels · 05-26-2015, 05:53 PM

It is waaay easier to accomplish.

You can ssh into a box and supply a command which is to be executed at the remote host, but the output is on your local terminal. Which you can redirect into a file. E.g.

Code:

ssh use@remotehost 'ls -l /home' > info.txt

In order to circumvent the password problem you should set up passwordless SSH, using keys to log in. Once this is set up, SSH, SFTP, SCP and RSYNC all work with the same keys, no passwords, in case you'd ever need those programs.

jlinkels

jpollard · 05-26-2015, 07:13 PM

Depending on what you really want to do:

1. nfs mount the remote directory tree - then use the "mv" command.

2. use sshfs to mount the remote directory tree, and then use the "mv" command.

Either may be simpler than trying to make up an error prone script that may fail with some file names (blanks or other characters embedded in the file name for instance).

legendmac · 05-27-2015, 09:39 AM

I cannot use SSH key based authentication. It is against company policy to do that for root user accounts.

schneidz · 05-27-2015, 09:44 AM

^ on my personal pc, i cannot even log in remotely as root (using either key or password). no offense, but why does your corporate security seem so kludgy ?

legendmac · 05-27-2015, 09:51 AM

I don't know. I'm just learning all of this since I transferred to a new role. Ultimately I'm trying to automate our software install process at client sites, but it's becoming quite the challenge to get the necessary information from each server at a client site prior to installation. Lots of back and forth between boxes needed at various steps in the installation process.

suicidaleggroll · 05-27-2015, 09:54 AM

Quote:

Originally Posted by legendmac

I cannot use SSH key based authentication. It is against company policy to do that for root user accounts.

But it's not against company policy to store the system's root password in PLAIN TEXT in a shell script?

Anyway, I have used scp with expect to send the password before, for a very specific application where keys were not practical and it was perfectly acceptable to store the password in plain text (initial setup of an embedded system with a password everybody knows and is published in the documentation anyway):

Code:

#!/usr/bin/expect

set timeout -1

if { $argc == 1 } {
   set ip [lindex $argv 0]
} else {
   puts "Usage: $argv0 ip"
   exit 1
}

spawn scp -p -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no file root@$ip:/path/to/location/
expect *assword:
send "Password\r"
expect eof

exit 0

pan64 · 05-27-2015, 09:59 AM

Only local root users are allowed, that is normal I think, because otherwise the system can be compromised (probably).
I would try to write an expect script or similar to do the task. Or use rsync, or another client side service. If possible.

legendmac · 05-27-2015, 01:20 PM

Quote:

Originally Posted by suicidaleggroll

But it's not against company policy to store the system's root password in PLAIN TEXT in a shell script?

Anyway, I have used scp with expect to send the password before, for a very specific application where keys were not practical and it was perfectly acceptable to store the password in plain text (initial setup of an embedded system with a password everybody knows and is published in the documentation anyway):

Code:

#!/usr/bin/expect

set timeout -1

if { $argc == 1 } {
   set ip [lindex $argv 0]
} else {
   puts "Usage: $argv0 ip"
   exit 1
}

spawn scp -p -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no file root@$ip:/path/to/location/
expect *assword:
send "Password\r"
expect eof

exit 0

I cannot get this to work inside my expect script. No errors are thrown and exp_internal 1 does not show anything being wrong, but the file does not come across. If I do it during an interact, however, and manually enter the scp command, it works fine. Any thoughts on why this might be happening? Same thing occurs with rsync. I can't get either to work within the expect script.

legendmac · 05-27-2015, 02:22 PM

With a little more digging, this is what I see in exp_internal 1 when I try the scp or rsync commands from within the expect script. It logs in to the remote machine fine, but cannot transfer the file. Like I said before, I can manually type in the exact same commands in an SSH session for scp and rsync and they work. But I get these issues when I try to execute those commands in the expect script.

Commands tried:

Code:

spawn scp -p root@remotehost:/root/file.txt user@localhost:/home/user/
spawn rsync -a -e ssh file.txt user@localhost:/home/user/

exp_internal 1 from SCP command:

Code:

tty_raw_noecho: was raw = 0  echo = 1
spawn id exp7 sent <\r\n>
 
spawn id exp7 sent <Permission denied, please try again.\r\r\n>
Permission denied, please try again.
spawn id exp7 sent <Permission denied, please try again.\r\r\n>
Permission denied, please try again.
spawn id exp7 sent <Permission denied (publickey,gssapi-with-mic,password).\r\r\n>
Permission denied (publickey,gssapi-with-mic,password).
spawn id exp7 sent <lost connection\r\n>
lost connection
interact: received eof from spawn_id exp7
tty_set: raw = 0, echo = 1
tty_set: raw = 5, echo = 0

exp_internal 1 from RSYNC command:

Code:

tty_raw_noecho: was raw = 0  echo = 1
spawn id exp7 sent <\r\n>
 
spawn id exp7 sent <rsync: link_stat "/home/user/file.txt" failed: No such file or directory (2)\r\n>
rsync: link_stat "/home/user/file.txt" failed: No such file or directory (2)
spawn id exp7 sent <rsync error: some files could not be transferred (code 23) at main.c(702)\r\n>
rsync error: some files could not be transferred (code 23) at main.c(702)
interact: received eof from spawn_id exp7
tty_set: raw = 0, echo = 1
tty_set: raw = 5, echo = 0

legendmac · 05-27-2015, 03:11 PM

Well, I don't know what the scp problem is/was, but I managed to fix the rsync issue so that I could transfer my file. It just needed the rsync command to be adjusted a little and an interact command to be added.

Here's what I ended up with:

Code:

spawn rsync -a -e ssh root@remotehost:/root/file.txt file.txt
expect {
	-re "(.*)assword:" { 
		send -s "$pswd\r"
	}
}
interact

legendmac · 05-27-2015, 03:47 PM

scp was missing the 'interact' command, as well. Now it works fine. Thanks everyone for your suggestions!