LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 09-16-2014, 09:29 AM   #1
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 132

Rep: Reputation: Disabled
Expect SFTP Rename


Hey Guys,

Having a bit of trouble trying to get an Expect script to do something simple!

I need to rename a file after I've downloaded it, and although I've probably gone the long way round I've got an 'almost' working solution.

I download the files, and then do an ls and pass the contents to a variable, then find the files within that variable and run through each line in a loop... the problem is with the rename command:

Code:
/usr/bin/expect <<! > $FTPLIST
        set timeout -1
        spawn sftp -o$PORT $USER@$HOST
        expect "password:"
        send "$PASS\r"
        expect "sftp>"
        send "cd Output\r"
        expect "sftp>"
        send "ls -1 *$D*\r"
        send "bye\r"
        expect eof
!
awk '/MF_BAT/' $FTPLIST | while read i; do

/usr/bin/expect <<!
        set timeout -1
        spawn sftp -o$PORT $USER@$HOST
        expect "password:"
        send "$PASS\r"
        expect "sftp>"
        send "cd Output\r"
        expect "sftp>"
        send "rename $i PROCESSED_$i"
        expect "sftp>"
        send "bye\r"
        expect eof
!
done
The problem is with send "rename $i PROCESSED_$i"...

If I substitute the $i for the actual filename in txt it works fine, but as soon as I add the $i it tries to send the command immediately after the first $i and therefore the SFTP Session just moans that the rename command is unfinished.

Anyone have any ideas?

Thanks
Jon
 
Old 09-16-2014, 10:25 AM   #2
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,027

Rep: Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845
if the rename subcommand does what i think it does this would probably be simpler:
Code:
ssh user@host mv /whatever/floats /your/boat
 
Old 09-16-2014, 10:27 AM   #3
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 132

Original Poster
Rep: Reputation: Disabled
Unfortunately ssh is not an option. We have no ssh access to the server. The server belongs to someone else, and we have access only via SFTP.

I would have preferred not to use Expect for this at all, but unfortunately I've no other option really.
 
Old 09-17-2014, 04:22 AM   #4
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 132

Original Poster
Rep: Reputation: Disabled
Just a quick bump wondering if anyone has any ideas?

I don't understand why if I send the following command in my expect script:

Code:
send "rename file1 processed_file1"
It works and renames the file.

But when I do it with the loop:
Code:
awk '/MF_BAT/' $FTPLIST | while read i; do

/usr/bin/expect <<!
        set timeout -1
        spawn sftp -o$PORT $USER@$HOST
        expect "password:"
        send "$PASS\r"
        expect "sftp>"
        send "cd Output\r"
        expect "sftp>"
        send "rename $i PROCESSED_$i\r"
        expect "sftp>"
        send "bye\r"
        expect eof
!
done
And then run it, it outputs the following:

Code:
sftp> cd Output
sftp> rename MF_BAT_BB160914192302.csv.PGP
You must specify two paths after a rename command.
sftp>  PROCESSED_MF_BAT_BB160914192302.csv.PGP
Invalid command.
sftp> bye
So it picks up the filename from the loop... and adds it to the rename command.. and also it constructs the new filename... but it separates the commands into two!

If anyone has any ideas, it would be greatly appreciated.

Thanks
Jon

Last edited by jonnybinthemix; 09-17-2014 at 06:43 AM.
 
Old 09-17-2014, 09:21 AM   #5
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,027

Rep: Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845
maybe there is a \n character at the end of the file you are reading in. (did you by chance create the file on windows ?)
 
1 members found this post helpful.
Old 09-17-2014, 09:23 AM   #6
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 132

Original Poster
Rep: Reputation: Disabled
Not sure what you mean by the file I'm reading in? There's a chance the file was created by a windows box, but I'm not calling in the file really, I'm just renaming it on the remote SFTP server.
 
Old 09-17-2014, 09:59 AM   #7
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,027

Rep: Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845
what is $FTPLIST ?

Last edited by schneidz; 09-17-2014 at 10:02 AM.
 
1 members found this post helpful.
Old 09-17-2014, 10:00 AM   #8
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 132

Original Poster
Rep: Reputation: Disabled
It's a file I'm writing the output of the SFTP session to.
 
Old 09-17-2014, 10:02 AM   #9
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,027

Rep: Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845
output ?; you seem to be reading it in to your while loop via a pipe| ?

what is the output of:
Code:
cat -A $FTPLIST

Last edited by schneidz; 09-17-2014 at 10:04 AM.
 
1 members found this post helpful.
Old 09-17-2014, 10:03 AM   #10
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 132

Original Poster
Rep: Reputation: Disabled
Sorry I've not been clear..

So, before the bit of the script I've posted, theres another SFTP session that logs onto the server, and carries out the command ls -1 *.JPEG.PGP

Because the output of the whole session is written to $FTPLIST, the next bit finds all filenames that start with MF_BAT.

Then I want to loop through those filenames and rename them
 
Old 09-17-2014, 10:08 AM   #11
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,604

Rep: Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241
We sort of need to know what the contents are - Something appears to be including a newline as part of a file name.
 
Old 09-17-2014, 10:08 AM   #12
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 132

Original Poster
Rep: Reputation: Disabled
the result is:

Code:
spawn sftp -oPort=22 Login@XX.XX.XX.XX^M$
Connecting to XX.XX.XX.XX...^M$
reverse mapping checking getaddrinfo for xxxx-xxxx.xxxxxx.xxx-xx.com [XX.XX.XX.XX] ^M^M$
Login@XX.XX.XX.XX's password: ^M$
sftp> cd Output^M$
sftp> ls -1 *160914*^M$
MF_BAT_BB160914192302.csv.PGP^M$
sftp> bye^M$
just realised I changed the ls command.. it's sending ls -1 *$D*, $D is just $(date -d 'yesterday' +%d%m%y) so it's listing files with yesterdays date in the name.
 
Old 09-17-2014, 10:09 AM   #13
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,027

Rep: Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845
is MF_BAT_BB160914192302.csv.PGP the only file you are having problems with ?

where is it in $FTPLIST (first line / last line ?)
 
1 members found this post helpful.
Old 09-17-2014, 10:12 AM   #14
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,027

Rep: Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845
Quote:
Originally Posted by jonnybinthemix View Post
the result is:

Code:
spawn sftp -oPort=22 Login@XX.XX.XX.XX^M$
Connecting to XX.XX.XX.XX...^M$
reverse mapping checking getaddrinfo for xxxx-xxxx.xxxxxx.xxx-xx.com [XX.XX.XX.XX] ^M^M$
Login@XX.XX.XX.XX's password: ^M$
sftp> cd Output^M$
sftp> ls -1 *160914*^M$
MF_BAT_BB160914192302.csv.PGP^M$
sftp> bye^M$
just realised I changed the ls command.. it's sending ls -1 *$D*, $D is just $(date -d 'yesterday' +%d%m%y) so it's listing files with yesterdays date in the name.
those ^M's usually indicate windows newlines... it is the equivelent of hitting the return key after the filename (MF_BAT_BB160914192302.csv.PGP-enter-key).

seems like you are sending \r in the previous iteration.

Last edited by schneidz; 09-17-2014 at 10:16 AM.
 
1 members found this post helpful.
Old 09-17-2014, 10:14 AM   #15
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 132

Original Poster
Rep: Reputation: Disabled
That would make sense as the above is the output of this:

Code:
/usr/bin/expect <<! > $FTPLIST
        set timeout -1
        spawn sftp -o$PORT $USER@$HOST
        expect "password:"
        send "$PASS\r"
        expect "sftp>"
        send "cd Output\r"
        expect "sftp>"
        send "ls -1 *$D*\r"
        send "bye\r"
        expect eof
!
I tried omitting the \r on the rename line, but no difference.

Last edited by jonnybinthemix; 09-17-2014 at 10:15 AM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Issues with expect sftp manjuhp Linux - General 4 01-02-2014 03:09 PM
sftp group file rename solar05 Linux - Newbie 2 07-17-2009 01:10 AM
Running sftp under Expect and transfers fail jimbo1954 Linux - Software 2 11-27-2008 04:07 AM
Check for Success of SFTP transfer using expect kasthana Programming 4 06-03-2008 11:55 AM
transfer file using expect and sftp tanveer Linux - General 7 05-16-2008 11:29 AM


All times are GMT -5. The time now is 08:30 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration