LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 05-26-2015, 03:57 PM   #1
legendmac
LQ Newbie
 
Registered: Apr 2015
Location: AZ
Posts: 14

Rep: Reputation: Disabled
How to copy file from remote host to local host then delete from remote host


I have an expect script to SSH to a remote host and obtain some user inputs and information about the server/network configuration. The responses are saved in a text file that I then need to copy to my local host so that I can read the lines into variables for use in the parent shell script.

Is there a way to do this without needing to enter the username and password for the local host to use function scp? I have tried the following in my expect script to no avail:
Code:
spawn scp $usr@$host:$flnm .
expect {
	-re "(.*)assword:" { 
		send -s "$pswd\r"
	}
}
I have also tried to directly scp the file and enter the username and password to try to debug the issue, and that doesn't work either:
Code:
spawn scp file.txt user@host:file.txt
expect {
	-re "(.*)assword:" {
		send -s "password\r"
	}
	"you sure you want to continue connecting" {
		send -s "yes\r"
		exp_continue
	}
}
In both scenarios I have used exp_internal 1, and there are no errors. But I do not end up with the file on my local host.

Following the copy, I would like to delete the file from the remote host. Any suggestions on how to accomplish this?
 
Old 05-26-2015, 05:55 PM   #2
ShadowCat8
Member
 
Registered: Nov 2004
Location: Ontario, CA
Distribution: Gentoo, Arch, (RedHat4.x-9.x, FedoraCore 1.x-4.x, Debian Potato-Sarge, LFS 6.0, etc.)
Posts: 226

Rep: Reputation: 51
Greetings,

How about just writing the file to a location that you can rsync from to the local machine? And, rsync can delete the file after it has synced it locally. To do so, you would first need to set up a location in the filesystem to set as the rsync point in /etc/rsyncd.conf.
Code:
[monitor]
     path = /tmp/monitor
     comment = Data for systems monitor
Then it's just a matter of adding the rsync line to your script:
Code:
...
   rsync -a --remove-source-files rsync://ip.of.host.system:873/monitor/data.file /local/path/ 
...
Hope that helps.
 
1 members found this post helpful.
Old 05-26-2015, 06:18 PM   #3
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,051

Rep: Reputation: 852Reputation: 852Reputation: 852Reputation: 852Reputation: 852Reputation: 852Reputation: 852
Quote:
Originally Posted by legendmac View Post
I have an expect script to SSH to a remote host and obtain some user inputs and information about the server/network configuration. The responses are saved in a text file that I then need to copy to my local host so that I can read the lines into variables for use in the parent shell script.

Is there a way to do this without needing to enter the username and password for the local host to use function scp? I have tried the following in my expect script to no avail:
...
you're doing it wrong. setting up ssh with key based authentication is the way to go.
 
Old 05-26-2015, 06:53 PM   #4
jlinkels
Senior Member
 
Registered: Oct 2003
Location: Bonaire
Distribution: Debian Wheezy/Jessie/Sid, Linux Mint DE
Posts: 4,493

Rep: Reputation: 636Reputation: 636Reputation: 636Reputation: 636Reputation: 636Reputation: 636
It is waaay easier to accomplish.

You can ssh into a box and supply a command which is to be executed at the remote host, but the output is on your local terminal. Which you can redirect into a file. E.g.
Code:
ssh use@remotehost 'ls -l /home' > info.txt
In order to circumvent the password problem you should set up passwordless SSH, using keys to log in. Once this is set up, SSH, SFTP, SCP and RSYNC all work with the same keys, no passwords, in case you'd ever need those programs.

jlinkels
 
Old 05-26-2015, 08:13 PM   #5
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,654

Rep: Reputation: 1255Reputation: 1255Reputation: 1255Reputation: 1255Reputation: 1255Reputation: 1255Reputation: 1255Reputation: 1255Reputation: 1255
Depending on what you really want to do:

1. nfs mount the remote directory tree - then use the "mv" command.

2. use sshfs to mount the remote directory tree, and then use the "mv" command.

Either may be simpler than trying to make up an error prone script that may fail with some file names (blanks or other characters embedded in the file name for instance).
 
1 members found this post helpful.
Old 05-27-2015, 10:39 AM   #6
legendmac
LQ Newbie
 
Registered: Apr 2015
Location: AZ
Posts: 14

Original Poster
Rep: Reputation: Disabled
I cannot use SSH key based authentication. It is against company policy to do that for root user accounts.
 
Old 05-27-2015, 10:44 AM   #7
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,051

Rep: Reputation: 852Reputation: 852Reputation: 852Reputation: 852Reputation: 852Reputation: 852Reputation: 852
^ on my personal pc, i cannot even log in remotely as root (using either key or password). no offense, but why does your corporate security seem so kludgy ?
 
Old 05-27-2015, 10:51 AM   #8
legendmac
LQ Newbie
 
Registered: Apr 2015
Location: AZ
Posts: 14

Original Poster
Rep: Reputation: Disabled
I don't know. I'm just learning all of this since I transferred to a new role. Ultimately I'm trying to automate our software install process at client sites, but it's becoming quite the challenge to get the necessary information from each server at a client site prior to installation. Lots of back and forth between boxes needed at various steps in the installation process.
 
Old 05-27-2015, 10:54 AM   #9
suicidaleggroll
LQ Guru
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,362

Rep: Reputation: 2004Reputation: 2004Reputation: 2004Reputation: 2004Reputation: 2004Reputation: 2004Reputation: 2004Reputation: 2004Reputation: 2004Reputation: 2004Reputation: 2004
Quote:
Originally Posted by legendmac View Post
I cannot use SSH key based authentication. It is against company policy to do that for root user accounts.
But it's not against company policy to store the system's root password in PLAIN TEXT in a shell script?

Anyway, I have used scp with expect to send the password before, for a very specific application where keys were not practical and it was perfectly acceptable to store the password in plain text (initial setup of an embedded system with a password everybody knows and is published in the documentation anyway):
Code:
#!/usr/bin/expect

set timeout -1

if { $argc == 1 } {
   set ip [lindex $argv 0]
} else {
   puts "Usage: $argv0 ip"
   exit 1
}

spawn scp -p -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no file root@$ip:/path/to/location/
expect *assword:
send "Password\r"
expect eof

exit 0

Last edited by suicidaleggroll; 05-27-2015 at 10:58 AM.
 
1 members found this post helpful.
Old 05-27-2015, 10:59 AM   #10
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 8,516

Rep: Reputation: 2436Reputation: 2436Reputation: 2436Reputation: 2436Reputation: 2436Reputation: 2436Reputation: 2436Reputation: 2436Reputation: 2436Reputation: 2436Reputation: 2436
Only local root users are allowed, that is normal I think, because otherwise the system can be compromised (probably).
I would try to write an expect script or similar to do the task. Or use rsync, or another client side service. If possible.
 
Old 05-27-2015, 02:20 PM   #11
legendmac
LQ Newbie
 
Registered: Apr 2015
Location: AZ
Posts: 14

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by suicidaleggroll View Post
But it's not against company policy to store the system's root password in PLAIN TEXT in a shell script?

Anyway, I have used scp with expect to send the password before, for a very specific application where keys were not practical and it was perfectly acceptable to store the password in plain text (initial setup of an embedded system with a password everybody knows and is published in the documentation anyway):
Code:
#!/usr/bin/expect

set timeout -1

if { $argc == 1 } {
   set ip [lindex $argv 0]
} else {
   puts "Usage: $argv0 ip"
   exit 1
}

spawn scp -p -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no file root@$ip:/path/to/location/
expect *assword:
send "Password\r"
expect eof

exit 0
I cannot get this to work inside my expect script. No errors are thrown and exp_internal 1 does not show anything being wrong, but the file does not come across. If I do it during an interact, however, and manually enter the scp command, it works fine. Any thoughts on why this might be happening? Same thing occurs with rsync. I can't get either to work within the expect script.
 
Old 05-27-2015, 03:22 PM   #12
legendmac
LQ Newbie
 
Registered: Apr 2015
Location: AZ
Posts: 14

Original Poster
Rep: Reputation: Disabled
With a little more digging, this is what I see in exp_internal 1 when I try the scp or rsync commands from within the expect script. It logs in to the remote machine fine, but cannot transfer the file. Like I said before, I can manually type in the exact same commands in an SSH session for scp and rsync and they work. But I get these issues when I try to execute those commands in the expect script.

Commands tried:
Code:
spawn scp -p root@remotehost:/root/file.txt user@localhost:/home/user/
spawn rsync -a -e ssh file.txt user@localhost:/home/user/
exp_internal 1 from SCP command:
Code:
tty_raw_noecho: was raw = 0  echo = 1
spawn id exp7 sent <\r\n>
 
spawn id exp7 sent <Permission denied, please try again.\r\r\n>
Permission denied, please try again.
spawn id exp7 sent <Permission denied, please try again.\r\r\n>
Permission denied, please try again.
spawn id exp7 sent <Permission denied (publickey,gssapi-with-mic,password).\r\r\n>
Permission denied (publickey,gssapi-with-mic,password).
spawn id exp7 sent <lost connection\r\n>
lost connection
interact: received eof from spawn_id exp7
tty_set: raw = 0, echo = 1
tty_set: raw = 5, echo = 0
exp_internal 1 from RSYNC command:
Code:
tty_raw_noecho: was raw = 0  echo = 1
spawn id exp7 sent <\r\n>
 
spawn id exp7 sent <rsync: link_stat "/home/user/file.txt" failed: No such file or directory (2)\r\n>
rsync: link_stat "/home/user/file.txt" failed: No such file or directory (2)
spawn id exp7 sent <rsync error: some files could not be transferred (code 23) at main.c(702)\r\n>
rsync error: some files could not be transferred (code 23) at main.c(702)
interact: received eof from spawn_id exp7
tty_set: raw = 0, echo = 1
tty_set: raw = 5, echo = 0

Last edited by legendmac; 05-27-2015 at 03:28 PM.
 
Old 05-27-2015, 04:11 PM   #13
legendmac
LQ Newbie
 
Registered: Apr 2015
Location: AZ
Posts: 14

Original Poster
Rep: Reputation: Disabled
Well, I don't know what the scp problem is/was, but I managed to fix the rsync issue so that I could transfer my file. It just needed the rsync command to be adjusted a little and an interact command to be added.

Here's what I ended up with:
Code:
spawn rsync -a -e ssh root@remotehost:/root/file.txt file.txt
expect {
	-re "(.*)assword:" { 
		send -s "$pswd\r"
	}
}
interact

Last edited by legendmac; 05-27-2015 at 04:12 PM.
 
Old 05-27-2015, 04:47 PM   #14
legendmac
LQ Newbie
 
Registered: Apr 2015
Location: AZ
Posts: 14

Original Poster
Rep: Reputation: Disabled
scp was missing the 'interact' command, as well. Now it works fine. Thanks everyone for your suggestions!
 
  


Reply

Tags
expect, linux, scp, shell script, ssh


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Pause processes in remote host and resume execution in another remote host Saeya Darsan Linux - General 5 10-22-2013 02:51 AM
telnet: Unable to connect to remote host: No route to host grob115 Linux - Networking 3 07-30-2010 11:18 AM
secure copy a latest data file from the remote host spkandy Linux - Networking 4 08-08-2009 03:46 AM
How to copy a file securly to remote host in JAVA waqasdaar Programming 1 02-03-2009 05:56 AM
Resolving <www.some remote host>.... failed: Host not found. koodoo Linux - Newbie 2 06-27-2005 09:48 AM


All times are GMT -5. The time now is 12:31 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration