LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 08-24-2009, 03:15 PM   #1
lunlun
Member
 
Registered: Jul 2005
Posts: 41

Rep: Reputation: 15
Smile *Perl* ssh into remote machine and execute parallel copy operation to network drive


Hi I have a remote machine that has a mounted network drive.

I am trying to create a script that ask me for the name of folder I wish to copy, ssh into the remote machine and have it start the copy operation.

parallel/concurrent copy is needed as I have a time constraint.

here is what i have for now in my script in "perl" if it matters....

print `ssh -t -x rocup@factory "cp -rf $folder1/ /NFS; cp -rf $folder2/ /NFS; cp -rf $folder3/ /NFS"';

I also tried

print `ssh -t -x rocup@factory "cp -rf $folder1/ /NFS & cp -rf $folder2/ /NFS & cp -rf $folder3/ /NFS &"';

and

print `ssh -t -x rocup@factory "nohup cp -rf $folder1/ /NFS;nohup cp -rf $folder2/ /NFS;nohup cp -rf $folder3/ /NFS"';

the above operations copy one by one and it takes over an hour to do it, Im hoping that I can trim down the time a bit by doing a concurrent copy

the host factory does not have private/public key setup, so i cannot do a

ssh -f -t -x rocup@factory "cp -rf $folder1/ /NFS"; as I would need to enter password 3 times.

Thanks for helping
 
Old 08-24-2009, 06:12 PM   #2
MBybee
Member
 
Registered: Jan 2009
Location: wherever I can make a living
Distribution: PC-BSD / FreeBSD / Debian / Ubuntu / Win7 / OpenVMS
Posts: 438

Rep: Reputation: 57
They are running serially because of the ";" - can you not just run them on different lines?
Example:
Code:
print `ssh -t -x rocup@factory "cp -rf $folder1/ /NFS"';
print `ssh -t -x rocup@factory "cp -rf $folder2/ /NFS"';
print `ssh -t -x rocup@factory "cp -rf $folder3/ /NFS"';
That should simply roll through them - perhaps use "&" to run them in the background if needed. If you want to specify an arbitrary list of folders each time, you can just read the input arguments and split them out from there. Let me know if you'd like a code example.
 
Old 08-24-2009, 06:16 PM   #3
MBybee
Member
 
Registered: Jan 2009
Location: wherever I can make a living
Distribution: PC-BSD / FreeBSD / Debian / Ubuntu / Win7 / OpenVMS
Posts: 438

Rep: Reputation: 57
Nevermind - I see why you want to do it in one line. Because of the password prompting.

You can try using expect to cache the password/pass the password, you could also try using "&" to launch them in the background. I'll check that quickly here and confirm.
 
Old 08-24-2009, 06:34 PM   #4
MBybee
Member
 
Registered: Jan 2009
Location: wherever I can make a living
Distribution: PC-BSD / FreeBSD / Debian / Ubuntu / Win7 / OpenVMS
Posts: 438

Rep: Reputation: 57
<edit - this didn't work, I just thought it did>

Ok, I tested this and it worked, so perhaps here's what you're looking for?
user@server1 $ ssh -x user@server2 'hostname >> /tmp/testing & whoami >> /tmp/testing'
This produced the following:
On server 1 it prompted once for password.
On server 2, /tmp/testing contained:
server2
user

Let me know if this works for you.

Last edited by MBybee; 08-24-2009 at 11:44 PM. Reason: It didn't work - I was just deluded.
 
Old 08-24-2009, 11:34 PM   #5
MBybee
Member
 
Registered: Jan 2009
Location: wherever I can make a living
Distribution: PC-BSD / FreeBSD / Debian / Ubuntu / Win7 / OpenVMS
Posts: 438

Rep: Reputation: 57
Ok, FINALLY, I figured this one out, I'm *almost* sure

This was fairly challenging, but when I got home I tested with timed events to make sure it was working, and sure enough, my previous method (like yours) wasn't really working. Hostname and whoami run too fast to be useful alone, so I was thinking it worked, and it did not.

This, however, I'm pretty certain works. The sleeps have date then host unless it runs parallel.
user@host1:~$ ssh user@host2 " sh -c 'sleep 60;date >> test' & sh -c 'sleep 10; hostname >> test' "
user@host2's password:
<60 seconds elapses>
user@host2:~$ cat test
Host2
Mon Aug 24 21:32:25 MST 2009

So now I think I have it. By spinning off a sub-shell, you can break off the whole command... I think. It sure looks like it works, after what, like 4 attempts? LOL
There is still Expect
 
Old 08-25-2009, 11:10 AM   #6
lunlun
Member
 
Registered: Jul 2005
Posts: 41

Original Poster
Rep: Reputation: 15
just wondering, why is expect still needed then?


I have never used expect, not really sure what it is for/how to use it for my case
 
Old 08-25-2009, 11:31 AM   #7
MBybee
Member
 
Registered: Jan 2009
Location: wherever I can make a living
Distribution: PC-BSD / FreeBSD / Debian / Ubuntu / Win7 / OpenVMS
Posts: 438

Rep: Reputation: 57
Expect is used primarily to handle things that prompt (like telnet/sftp/ssh), especially when they don't accept the normal styles of input (like < ).

There's a Perl module for it as well (this jumps right to the login section): http://search.cpan.org/~rgiersig/Exp...automate_login

I've used it on occasion when I needed to drive a stubborn session where I couldn't use the FTP or SFTP or whatever modules - often due to needing a password for each session or something.

In this case, you could use it to prompt for the password only once, then pass it to the commands as you called them.

Glad that worked for you, though!
 
Old 08-25-2009, 12:08 PM   #8
Sergei Steshenko
Senior Member
 
Registered: May 2005
Posts: 4,481

Rep: Reputation: 453Reputation: 453Reputation: 453Reputation: 453Reputation: 453
And why not 'scp' ?
 
  


Reply

Tags
command, linux, perl, remote, ssh


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Can't SSH to remote machine: Connection closed by remote host Avatar Linux - Networking 34 05-24-2013 09:28 AM
Need help to execute command on Remote machine tucs_123 Linux - Newbie 4 01-09-2009 02:03 AM
scp: copy a file from local machine to remote machine seran Linux - Newbie 8 10-30-2007 12:23 PM
perl script to execute my a c executable on remote machin sharad Linux - General 5 12-14-2006 07:56 AM
how to open a dilog on local machine when i do ssh to remote machine fahad26 Programming 3 05-03-2005 07:39 PM


All times are GMT -5. The time now is 05:04 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration