LinuxQuestions.org
LinuxAnswers - the LQ Linux tutorial section.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices



Reply
 
Search this Thread
Old 11-20-2012, 06:58 PM   #1
phpguru
LQ Newbie
 
Registered: Nov 2012
Posts: 12

Rep: Reputation: Disabled
Post Automate FTP session interaction with Expect script


I need to write an expect script that connects to an FTP server on the command line. I need to traverse into a specific folder and download all the files in a certain directory and delete them when I'm done downloading them.

I can do this without expect using
Code:
ncftp -u username ftp.domain.com
cd downloads
get *.tgz
I am trying to solve two different problems.

The first is, I need to get a list of the files in the remote directory, and iterate over them, downloading each one and deleting it.

The second problem is, I need the script to not timeout. I figure once I have switched my stupid code below into a foreach, that's the perfect place to put exp_continue, but I am not sure how to code getting the file list and iterating on the files.

Where I'm stuck is this:
Code:
send "ls -a"
gives me a nice file list, but I have to send "ls -a" and store the resulting list in an array inside expect. How do you store a list of files returned from an FTP server in an array inside expect?

Here's what I have so far...

Code:
#!/usr/bin/expect
set timeout 300
# @TODO Change to a argv
cd /home/myusername/downloads
# Start ncftp
spawn ncftp -u $ftpusername $ftphost
expect "Password:"
# Send the password
send "$password\r"
# Get an FTP prompt
expect "*$ftpusername*"
# Change directories
send "cd downloads\r"
# Moved into directory
expect "*successfully changed*"
# Get an FTP prompt
expect "*downloads*"
# here's where I need a loop
send "ls -a\r"
send "get *.tgz\r"
expect "100%"
# exp_continue
# end loop
expect "*outgoing*"
send "bye"
puts "All done.\r"
 
Old 11-21-2012, 01:59 AM   #2
firstfire
Member
 
Registered: Mar 2006
Location: Ekaterinburg, Russia
Distribution: Debian, Ubuntu
Posts: 640

Rep: Reputation: 375Reputation: 375Reputation: 375Reputation: 375
Hi.

Consider this example in which I save output of shell command ls -1 to a list:
Code:
#!/usr/bin/expect
spawn sh
expect "$ "
send "ls -1 /\n"
expect "$ "

set results [regexp -all -inline {[^\r\n]+} $expect_out(buffer)]

for {set i 1} {$i<[llength $results]-1} {incr i} {
	puts "+++[lindex $results $i]"
}
I use for loop instead of foreach to ignore first and last elements of $expect_out(buffer), which are "ls -1 /" and "$ ".

PS: The strange [regexp ...] here (instead of just [split ...]) is because expect (unlike tcl) insist on using '\r\n' as line endings.

Last edited by firstfire; 11-21-2012 at 02:01 AM.
 
Old 11-21-2012, 02:42 AM   #3
phpguru
LQ Newbie
 
Registered: Nov 2012
Posts: 12

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by firstfire View Post
Hi.
Consider this example in which I save output of shell command ls -1 to a list:
Code:
#!/usr/bin/expect
spawn sh
expect "$ "
send "ls -1 /\n"
expect "$ "

set results [regexp -all -inline {[^\r\n]+} $expect_out(buffer)]

for {set i 1} {$i<[llength $results]-1} {incr i} {
	puts "+++[lindex $results $i]"
}
I use for loop instead of foreach to ignore first and last elements of $expect_out(buffer), which are "ls -1 /" and "$ ".
So, this mess:
Code:
set results [regexp -all -inline {[^\r\n]+} $expect_out(buffer)]
is where the magic comes from? You have to catch and parse the output buffer? It's a helpful example, thanks.

I thought that when you
Code:
ls -l
in bash, you get a list (array) of filenames automagically. It doesn't work that way when listing an FTP server? Or within Expect? I'm confused about that.

Quote:
Originally Posted by firstfire View Post
PS: The strange [regexp ...] here (instead of just [split ...]) is because expect (unlike tcl) insist on using '\r\n' as line endings.
Really? I find that odd. On what distro? You send just fine with "\r" at the end only, correct?
 
Old 11-21-2012, 03:47 AM   #4
firstfire
Member
 
Registered: Mar 2006
Location: Ekaterinburg, Russia
Distribution: Debian, Ubuntu
Posts: 640

Rep: Reputation: 375Reputation: 375Reputation: 375Reputation: 375
Honestly, I'm not expert in expect or tcl and do not know why this happens, but if I print $expect_out(buffer) to stdout and look at it through hexdump, I see 0d 0a (\r\n) after each line. Probably this quote from man expect explains this behaviour:
Quote:
Newlines are usually converted to carriage return, linefeed sequences when output by the terminal
driver. Thus, if you want a pattern that explicitly matches the two lines, from, say,
printf("foo\nbar"), you should use the pattern "foo\r\nbar".
I spent an hour trying to split such response into separate lines. Now I understand why they wanted to call it sex instead of expect (see man expect):
Quote:
BUGS
It was really tempting to name the program "sex" (for either "Smart EXec" or "Send-EXpect"), but good
sense (or perhaps just Puritanism) prevailed.


BTW, as you can see on each my post (small icon on the lower left side of message), I use Ubuntu.

Anyway, my main point was that you can find your file listing in $expect_out(buffer).

Last edited by firstfire; 11-21-2012 at 03:49 AM.
 
Old 11-21-2012, 11:01 AM   #5
phpguru
LQ Newbie
 
Registered: Nov 2012
Posts: 12

Original Poster
Rep: Reputation: Disabled
Unhappy

Quote:
Originally Posted by firstfire View Post
Code:
set results [regexp -all -inline {[^\r\n]+} $expect_out(buffer)]
for {set i 1} {$i<[llength $results]-1} {incr i} {
	puts "+++[lindex $results $i]"
}
I tried pasting this into my script and run it, and it gives:
Code:
command returned bad code: -101
    while executing
"for {set i 1} {$i<[llength $results]-1} {incr i} {
	puts "+++[lindex $results $i]"
        exp_continue
}"
I took out exp_continue and it ran fine. I figured I would want to plug the downloading inside this for loop, and call exp_continue inside there.

Last edited by phpguru; 11-21-2012 at 11:05 AM.
 
  


Reply

Tags
expect


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Expect script to automate ftp upload mabin Linux - Newbie 1 10-18-2008 12:36 PM
Expect script to automate ftp upload mabin Linux - Security 1 10-18-2008 12:29 PM
Expect script to automate ftp upload mabin Linux - General 1 10-18-2008 08:08 AM


All times are GMT -5. The time now is 03:59 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration