LinuxQuestions.org
Latest LQ Deal: Linux Power User Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 09-17-2014, 09:16 AM   #16
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,076

Rep: Reputation: 861Reputation: 861Reputation: 861Reputation: 861Reputation: 861Reputation: 861Reputation: 861

(i think they should be new-lines \n).

i think it would be easier to contact the server admin about ssh-keys than to try to re-invent an sftp client ?

Last edited by schneidz; 09-17-2014 at 09:21 AM.
 
1 members found this post helpful.
Old 09-17-2014, 09:20 AM   #17
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 169

Original Poster
Rep: Reputation: Disabled
Ahh it's starting to make sense now... so somewhere on the end of the filename inside $FTPLIST there is a new line...

Code:
/usr/bin/expect <<! > $FTPLIST
        set timeout -1
        spawn sftp -o$PORT $USER@$HOST
        expect "password:"
        send "$PASS\n"
        expect "sftp>"
        send "cd Output\n"
        expect "sftp>"
        send "ls -1 *$D*\n"
        send "bye\n"
        expect eof
!

awk '/MF_BAT/' $FTPLIST | while read i; do

/usr/bin/expect <<!
        set timeout -1
        spawn sftp -o$PORT $USER@$HOST
        expect "password:"
        send "$PASS\n"
        expect "sftp>"
        send "cd Output\n"
        expect "sftp>"
        send "rename $i PROCESSED_$i \n"
        expect "sftp>"
        send "bye\n"
        expect eof
!
done
Result:

spawn sftp -oPort=22 Login@XX.XX.XX.XX
Connecting to XX.XX.XX.XX...
reverse mapping checking getaddrinfo for xxxxxx.com [XX.XX.XX.XX]
Login@XX.XX.XX.XX's password:
sftp> cd Output
sftp> rename MF_BAT_BB160914192302.csv.PGP
You must specify two paths after a rename command.
sftp> PROCESSED_MF_BAT_BB160914192302.csv.PGP
Invalid command.
sftp>
sftp> bye

Last edited by jonnybinthemix; 09-17-2014 at 09:22 AM.
 
Old 09-17-2014, 09:22 AM   #18
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,076

Rep: Reputation: 861Reputation: 861Reputation: 861Reputation: 861Reputation: 861Reputation: 861Reputation: 861
technically its called a carriage-return (what you want is a new-line).
 
1 members found this post helpful.
Old 09-17-2014, 09:22 AM   #19
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 169

Original Poster
Rep: Reputation: Disabled
Sorry, edited above post as realised I missed out an important bit
 
Old 09-17-2014, 09:23 AM   #20
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,076

Rep: Reputation: 861Reputation: 861Reputation: 861Reputation: 861Reputation: 861Reputation: 861Reputation: 861
can you try running:
Code:
cat -A $FTPLIST
again.
 
1 members found this post helpful.
Old 09-17-2014, 09:23 AM   #21
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 169

Original Poster
Rep: Reputation: Disabled
My understanding of Expect (extremely limited) was that you have to send a \r (Carriage Return) as it's just as if the user would be pressing ENTER. To process that command?
 
Old 09-17-2014, 09:26 AM   #22
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 169

Original Poster
Rep: Reputation: Disabled
Commented out the second SFTP Session and ran the following:

Code:
/usr/bin/expect <<! > $FTPLIST
        set timeout -1
        spawn sftp -o$PORT $USER@$HOST
        expect "password:"
        send "$PASS\n"
        expect "sftp>"
        send "cd Output\n"
        expect "sftp>"
        send "ls -1 *$D*\n"
        send "bye\n"
        expect eof
!

cat -A $FTPLIST

#awk '/MF_BAT/' $FTPLIST | while read i; do

#/usr/bin/expect <<!
#       set timeout -1
#        spawn sftp -o$PORT $USER@$HOST
#        expect "password:"
#        send "$PASS\n"
#        expect "sftp>"
#        send "cd Output\n"
#        expect "sftp>"
#        send "rename $i PROCESSED_$i \n"
#       expect "sftp>"
#        send "bye\n"
#        expect eof
#!
#done
Result is:

[root@server scripts]# ./ftpget1.sh
spawn sftp -oPort=22 Login@XX.XX.XX.XX^M$
Connecting to XX.XX.XX.XX...^M$
reverse mapping checking getaddrinfo for xxxx.com [XX.XX.XX.XX]^M^M$
BATUniversityLogin@XX.XX.XX.XX's password: ^M$
sftp> cd Output^M$
sftp> ls -1 *160914*^M$
MF_BAT_BB160914192302.csv.PGP^M$
sftp> bye^M$
 
Old 09-17-2014, 09:29 AM   #23
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,076

Rep: Reputation: 861Reputation: 861Reputation: 861Reputation: 861Reputation: 861Reputation: 861Reputation: 861
i purposely avoid expect since it is a hax. maybe try using \n\r ?

at this point you will have to unbug the output of expect for any functionality you will be adding to this process. keep playing around with it and maybe it will work; else, my recommendation of contacting the server admin still stands.
 
1 members found this post helpful.
Old 09-17-2014, 09:31 AM   #24
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,076

Rep: Reputation: 861Reputation: 861Reputation: 861Reputation: 861Reputation: 861Reputation: 861Reputation: 861
Quote:
Originally Posted by jonnybinthemix View Post
Commented out the second SFTP Session and ran the following:

Code:
/usr/bin/expect <<! > $FTPLIST
        set timeout -1
        spawn sftp -o$PORT $USER@$HOST
        expect "password:"
        send "$PASS\n"
        expect "sftp>"
        send "cd Output\n"
        expect "sftp>"
        send "ls -1 *$D*\n"
        send "bye\n"
        expect eof
!

cat -A $FTPLIST

#awk '/MF_BAT/' $FTPLIST | while read i; do

#/usr/bin/expect <<!
#       set timeout -1
#        spawn sftp -o$PORT $USER@$HOST
#        expect "password:"
#        send "$PASS\n"
#        expect "sftp>"
#        send "cd Output\n"
#        expect "sftp>"
#        send "rename $i PROCESSED_$i \n"
#       expect "sftp>"
#        send "bye\n"
#        expect eof
#!
#done
Result is:

[root@server scripts]# ./ftpget1.sh
spawn sftp -oPort=22 Login@XX.XX.XX.XX^M$
Connecting to XX.XX.XX.XX...^M$
reverse mapping checking getaddrinfo for xxxx.com [XX.XX.XX.XX]^M^M$
BATUniversityLogin@XX.XX.XX.XX's password: ^M$
sftp> cd Output^M$
sftp> ls -1 *160914*^M$
MF_BAT_BB160914192302.csv.PGP^M$
sftp> bye^M$
i still havent seen the output of
Code:
cat -A $FTPLIST
so not sure if this is your problem but maybe you can add a function to sanitize $FTPLIST to remove the carriage-returns.
 
1 members found this post helpful.
Old 09-17-2014, 09:34 AM   #25
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 169

Original Poster
Rep: Reputation: Disabled
The output of cat -A $FTPLIST is above...

spawn sftp -oPort=22 Login@XX.XX.XX.XX^M$
Connecting to XX.XX.XX.XX...^M$
reverse mapping checking getaddrinfo for xxxx.com [XX.XX.XX.XX]^M^M$
BATUniversityLogin@XX.XX.XX.XX's password: ^M$
sftp> cd Output^M$
sftp> ls -1 *160914*^M$
MF_BAT_BB160914192302.csv.PGP^M$
sftp> bye^M$

That (im assuming) is the result of the command, all output of the Expect session is redirected to $FTPLIST, so in theory.. the result of cat -A $FTPLIST would be the same as removing the > $FTLIST from the top of the expect session.
 
Old 09-17-2014, 09:39 AM   #26
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,076

Rep: Reputation: 861Reputation: 861Reputation: 861Reputation: 861Reputation: 861Reputation: 861Reputation: 861
oh yeah... thanks for explaining i wasnt seeing it clearly (i was expecting $FTPLIST to be a single column list of filenames).

try removing the carriage-returns before running the 2nd sftp ?
 
1 members found this post helpful.
Old 09-17-2014, 09:43 AM   #27
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 169

Original Poster
Rep: Reputation: Disabled
Added a line to clear carriage returns:

Code:
/usr/bin/expect <<! > $FTPLIST
        set timeout -1
        spawn sftp -o$PORT $USER@$HOST
        expect "password:"
        send "$PASS\n\r"
        expect "sftp>"
        send "cd Output\n\r"
        expect "sftp>"
        send "ls -1 *$D*\n\r"
        send "bye\n\r"
        expect eof
!
tr -d '\r' < $FTPLIST > $FTPLIST2
cat -A $FTPLIST2
The result of cat -A $FTPLIST2 is:

spawn sftp -oPort=22 Login@XX.XX.XX.XX$
Connecting to XX.XX.XX.XX...$
reverse mapping checking getaddrinfo for xxxx.com [XX.XX.XX.XX]$
Login@XX.XX.XX.XX's password: $
sftp> cd Output$
sftp> $
sftp> ls -1 *160914*$
MF_BAT_BB160914192302.csv.PGP$
sftp> $
sftp> bye$

So, the M has gone but the $ is still there... what does the $ mean?
 
Old 09-17-2014, 09:46 AM   #28
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 169

Original Poster
Rep: Reputation: Disabled
Running the whole script shows it works perfectly now:

RESULT:

[root@server scripts]# ./ftpget1.sh
spawn sftp -oPort=22 Login@XX.XX.XX.XX
Connecting to XX.XX.XX.XX...
reverse mapping checking getaddrinfo for xxxx.com [195.12.14.233]
Login@XX.XX.XX.XX's password:
sftp> cd Output
sftp> rename MF_BAT_BB160914192302.csv.PGP PROCESSED_MF_BAT_BB160914192302.csv.PGP
sftp> bye

Awesome, you nailed it Thanks so much for your help.
 
Old 09-17-2014, 09:47 AM   #29
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,076

Rep: Reputation: 861Reputation: 861Reputation: 861Reputation: 861Reputation: 861Reputation: 861Reputation: 861
$ = end of line anchor.
 
1 members found this post helpful.
Old 09-17-2014, 09:53 AM   #30
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 169

Original Poster
Rep: Reputation: Disabled
That works perfectly.. so $ is exactly what we want at the end of the line

Thank you so much for your help.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Issues with expect sftp manjuhp Linux - General 4 01-02-2014 02:09 PM
sftp group file rename solar05 Linux - Newbie 2 07-17-2009 12:10 AM
Running sftp under Expect and transfers fail jimbo1954 Linux - Software 2 11-27-2008 03:07 AM
Check for Success of SFTP transfer using expect kasthana Programming 4 06-03-2008 10:55 AM
transfer file using expect and sftp tanveer Linux - General 7 05-16-2008 10:29 AM


All times are GMT -5. The time now is 12:03 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration