*Perl* ssh into remote machine and execute parallel copy operation to network drive
 

Hi I have a remote machine that has a mounted network drive.

I am trying to create a script that ask me for the name of folder I wish to copy, ssh into the remote machine and have it start the copy operation.

parallel/concurrent copy is needed as I have a time constraint.

here is what i have for now in my script in "perl" if it matters....

print `ssh -t -x rocup@factory "cp -rf $folder1/ /NFS; cp -rf $folder2/ /NFS; cp -rf $folder3/ /NFS"';

I also tried

print `ssh -t -x rocup@factory "cp -rf $folder1/ /NFS & cp -rf $folder2/ /NFS & cp -rf $folder3/ /NFS &"';

and

print `ssh -t -x rocup@factory "nohup cp -rf $folder1/ /NFS;nohup cp -rf $folder2/ /NFS;nohup cp -rf $folder3/ /NFS"';

the above operations copy one by one and it takes over an hour to do it, Im hoping that I can trim down the time a bit by doing a concurrent copy

the host factory does not have private/public key setup, so i cannot do a

ssh -f -t -x rocup@factory "cp -rf $folder1/ /NFS"; as I would need to enter password 3 times.

Thanks for helping

They are running serially because of the ";" - can you not just run them on different lines?
Example:
That should simply roll through them - perhaps use "&" to run them in the background if needed. If you want to specify an arbitrary list of folders each time, you can just read the input arguments and split them out from there. Let me know if you'd like a code example.

Nevermind - I see why you want to do it in one line. Because of the password prompting.

You can try using expect to cache the password/pass the password, you could also try using "&" to launch them in the background. I'll check that quickly here and confirm.

<edit - this didn't work, I just thought it did>

Ok, I tested this and it worked, so perhaps here's what you're looking for?
user@server1 $ ssh -x user@server2 'hostname >> /tmp/testing & whoami >> /tmp/testing'
This produced the following:
On server 1 it prompted once for password.
On server 2, /tmp/testing contained:
server2
user

Let me know if this works for you.

Ok, FINALLY, I figured this one out, I'm *almost* sure :D

This was fairly challenging, but when I got home I tested with timed events to make sure it was working, and sure enough, my previous method (like yours) wasn't really working. Hostname and whoami run too fast to be useful alone, so I was thinking it worked, and it did not.

This, however, I'm pretty certain works. The sleeps have date then host unless it runs parallel.
user@host1:~$ ssh user@host2 " sh -c 'sleep 60;date >> test' & sh -c 'sleep 10; hostname >> test' "
user@host2's password:
<60 seconds elapses>
user@host2:~$ cat test
Host2
Mon Aug 24 21:32:25 MST 2009

So now I think I have it. By spinning off a sub-shell, you can break off the whole command... I think. It sure looks like it works, after what, like 4 attempts? LOL
There is still Expect :D

just wondering, why is expect still needed then?


I have never used expect, not really sure what it is for/how to use it for my case

Expect is used primarily to handle things that prompt (like telnet/sftp/ssh), especially when they don't accept the normal styles of input (like < ).

There's a Perl module for it as well (this jumps right to the login section): http://search.cpan.org/~rgiersig/Exp...automate_login

I've used it on occasion when I needed to drive a stubborn session where I couldn't use the FTP or SFTP or whatever modules - often due to needing a password for each session or something.

In this case, you could use it to prompt for the password only once, then pass it to the commands as you called them.

Glad that worked for you, though!

And why not 'scp' ?