LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 03-14-2011, 07:21 PM   #1
mbeipi
LQ Newbie
 
Registered: Mar 2011
Posts: 4

Rep: Reputation: 0
Problem running an Expect script within a Bash script


Hello, first of all let me state that I am not a systems administrator, I am just a humble geophysicist trying to retrieve some data from a server in an automated way.

I wrote an Expect script to do this:

Code:
#!/usr/bin/expect 
set year [lindex $argv 0]
set jday [lindex $argv 1]
spawn scp username@server:/somepath/$year/$jday/* ./.
expect "password:"
send "MyPassword\r"
interact
As you can see the data are in a directory structure that includes the year and the julian day.

Now, I know that it's better to do an RSA authentication or whatever it is and I shouldn't put the password in the script... I don't think the people who are giving me access to their data will let me do the RSA thing. So, let's assume that I have to use expect.

OK, the previous script works (only if I put that interact at the end, I have no idea why, if I don't put interact there then it gets to where the server is asking for the password and then returns to the prompt). The problem is that I don't want to sit there forever waiting for the data to download and then call the Expect script again with the next year jday combination. So I wrote a neat little bash script like so:

Code:
#!/bin/bash

while read year jday 
do

myscript.exp $year $jday

done <dates.file
where dates.file is something like:
2009 004
2009 087
...
you get the idea.

That does not work. It shows the server asking for the password and then I'm back at the command prompt.

Please help. I hope I do not anger you with my ignorance, I tried really hard to find an answer in other similar threads before breaking down and starting a new thread. TIA

Max
 
Old 03-14-2011, 07:27 PM   #2
mbeipi
LQ Newbie
 
Registered: Mar 2011
Posts: 4

Original Poster
Rep: Reputation: 0
Update: I just tried removing the "interact" from the expect script, so now the bash script continues after the first year/julian daypair, gets to where it's asking for the password, and goes on to the next year/julianday pair and so on... without ever mgetting anything
 
Old 03-14-2011, 07:41 PM   #3
jlinkels
Senior Member
 
Registered: Oct 2003
Location: Bonaire
Distribution: Debian Lenny/Squeeze/Wheezy/Sid
Posts: 4,066

Rep: Reputation: 491Reputation: 491Reputation: 491Reputation: 491Reputation: 491
I am very bad at debugging other's scripts, but:
put this near the beginning of you expect script:
Code:
exp_internal 1
and be surprised by the amount of useful information on your screen.
Also run you Bash script with sh -x yourscript to see what Bash internals are.

Somehow I have the idea that you should expect "password\r" or "password\r\n"
Once you have exp_internal 1 you'll see exactly what your server returns and how Expect matches that or not.

jlinkels
 
Old 03-15-2011, 12:22 AM   #4
mbeipi
LQ Newbie
 
Registered: Mar 2011
Posts: 4

Original Poster
Rep: Reputation: 0
One more update:
I got rid of the "interact" and instead put in a "expect eof"

that KIND OF does the trick: it starts to perform the scp but then exits before it's done copying the files over!

I believe other people have run into this before and will look around tomorrow, but if anybody knows the answer it might save hours of time!

Thanks

BTW, thanks for the suggestion Jlinkles I tried that already, no luck!
 
Old 03-15-2011, 02:11 AM   #5
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,478

Rep: Reputation: 1888Reputation: 1888Reputation: 1888Reputation: 1888Reputation: 1888Reputation: 1888Reputation: 1888Reputation: 1888Reputation: 1888Reputation: 1888Reputation: 1888
I am not all that familiar with expect scripts, but it would appear that as your scp is at least starting that the issue is it returns to the bash script and the next call to
the expect script, or at least to scp, is killing the first one. To this end I would look into finding out when the scp is finished prior to allowing
the bash loop to continue.
Whilst I am sure there are superior methods, something simple like the following may work:
Code:
wait $(pgrep -n scp)
This will wait for the most recent scp to conclude. You could additionally include this in an if and print an appropriate message should it not complete.
 
1 members found this post helpful.
Old 03-15-2011, 02:21 AM   #6
jlinkels
Senior Member
 
Registered: Oct 2003
Location: Bonaire
Distribution: Debian Lenny/Squeeze/Wheezy/Sid
Posts: 4,066

Rep: Reputation: 491Reputation: 491Reputation: 491Reputation: 491Reputation: 491
Just a calculated guess: put wait instead of interact.

This way Expect will wait until the child process is done. This is in line with your experience to wait for an end of file and grail's advice to wait until scp is done.

Besides you should wait after a spawn anyway in order to have Expect clean op child processes nicely when they do not terminate properly. But you see I often forget that myself.

jlinkels
 
1 members found this post helpful.
Old 03-15-2011, 02:30 AM   #7
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,478

Rep: Reputation: 1888Reputation: 1888Reputation: 1888Reputation: 1888Reputation: 1888Reputation: 1888Reputation: 1888Reputation: 1888Reputation: 1888Reputation: 1888Reputation: 1888
Thanks jinkels ... that sounds much better
 
1 members found this post helpful.
Old 03-15-2011, 03:36 PM   #8
mbeipi
LQ Newbie
 
Registered: Mar 2011
Posts: 4

Original Poster
Rep: Reputation: 0
apparently solved!

Thanks guys, I appreciate your comments.
I gave

Code:
exp_internal 1
another shot after all and realized that my main problem was that expect was timing out, so I added this at the very beginning of my expect script:

Code:
set timeout -1
I then run scp in verbose mode and tell expect to...

Code:
expect "Exit status"
That works. It waits for the scp to complete and does not timeout. Using wait as you suggested may work too, but I think I may still have had the timeout issue. Thanks again for your suggestions!

PS I say "apparently solved" because it's running now and seems to be running well, it will be a long while before it actually finishes so I'll wait until then to declare victory.
 
Old 03-15-2011, 04:07 PM   #9
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,488

Rep: Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956
Quote:
Originally Posted by mbeipi View Post
Now, I know that it's better to do an RSA authentication or whatever it is and I shouldn't put the password in the script... I don't think the people who are giving me access to their data will let me do the RSA thing.
Just an aside note. As long as the remote server permits publickey authentication method, the process of creating the key pair and copying the public one onto the remote home directory is up to you. Nevertheless, you should have learned something useful about expect, today!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Expect script: how do i send function key F12 in an expect script alix123 Programming 4 09-01-2013 09:06 PM
[SOLVED] /usr/bin/expect : Script to check server load using both expect and bash Soji Antony Programming 1 07-27-2010 11:27 PM
Embed expect in bash script prpednek Linux - General 3 05-15-2010 06:36 AM
Unexpected curly braces in expect script spawn command & bash suid problem slinx Programming 1 05-02-2008 01:47 PM
can expect be called from within a bash script? johnpaulodonnell Programming 4 06-21-2007 09:42 AM


All times are GMT -5. The time now is 01:05 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration