LinuxQuestions.org
Did you know LQ has a Linux Hardware Compatibility List?
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 01-23-2013, 11:22 AM   #1
zimbot
Member
 
Registered: Nov 2005
Location: cincinnati , ohio . USA
Distribution: ubuntu , Opensuse , CentOS
Posts: 122

Rep: Reputation: 17
bash script to lftp mirror and log


<code>
#!/bin/bash
#########_______ test of mirror
log=~/log/f_h_logfile.txt
batch=00
dest=/media/facing_history_1/$batch
cd $dest
echo start $(date) >> $log

DATE='date +%Y%m%d'
#mkdir $DATE
echo "Get files"

lftp -c 'open -e "set ftp:list-options -a; mirror -a -v array1/f_t/ ./media/facing_history_1/00" -u uname,psswd 172.16.100.101 >> $log'
echo end $(date) >> $log
echo ............ >> $log
</code>

but it does not log
lftp -c 'open -e "set ftp:list-options -a; mirror -a -v array1/f_t/ ./media/facing_history_1/00" -u uname,psswd 172.16.100.101 >> $log'
isn the same as
lftp -c 'open -e "set ftp:list-options -a; mirror -a -v array1/f_t/ ./media/facing_history_1/00" -u uname,psswd 172.16.100.101'

and it does not go to
echo end $(date) >> $log
echo ............ >> $log

how might one do that?
What I would really like to do is capture into a log the file names of the files in the session.
I am also trying to have a better understanding of what happens w the
lftp -c 'open -e "set ftp:list-options -a; mirror -a -v

I have looked at the lftp man page ( that is how i got this far ) but am a bit uncertain of my switch use.
example - I wish to do binary ( the default )
is my use of -a correct?

any wisdom would be appreciated

thanks , thanks much
 
Old 01-24-2013, 05:45 AM   #2
H_TeXMeX_H
Guru
 
Registered: Oct 2005
Location: $RANDOM
Distribution: slackware64
Posts: 12,928
Blog Entries: 2

Rep: Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269
lftp has a log option that you may want to look at:

Code:
                   --log=FILE                write lftp  commands  being  exe-
                                             cuted to FILE
 
1 members found this post helpful.
Old 01-24-2013, 05:59 AM   #3
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,488

Rep: Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956
Quote:
Originally Posted by zimbot View Post
lftp -c 'open -e "set ftp:list-options -a; mirror -a -v array1/f_t/ ./media/facing_history_1/00" -u uname,psswd 172.16.100.101 >> $log'
This doesn't log anything, since you have put redirection inside the lftp command. It should be outside the single quotes instead.

Also look at the ~/.lftp directory and see if it contains a file named transfer_log. If enabled it should have been populated by details about file transfers. If not, you can enable it explicitly by means of
Code:
set xfer:log 1
set xfer:log-file /path/to/transfer_log
the second option is not available in older versions of lftp, so that the path of the log file is stuck to its default value (~/.lftp/transfer_log).

Quote:
Originally Posted by zimbot View Post
I have looked at the lftp man page ( that is how i got this far ) but am a bit uncertain of my switch use.
example - I wish to do binary ( the default )
is my use of -a correct?
The -a option in the mirror command has a completely different meaning:
Code:
-a             same as --allow-chown --allow-suid --no-umask
whereas in other commands like get or mget it makes lftp switch to ascii transfer.
 
1 members found this post helpful.
Old 01-30-2013, 01:27 PM   #4
zimbot
Member
 
Registered: Nov 2005
Location: cincinnati , ohio . USA
Distribution: ubuntu , Opensuse , CentOS
Posts: 122

Original Poster
Rep: Reputation: 17
1st thanks everyone

Yes I have a log happening with ~/.lftp/transfer_log
and that is good but what I actually wish to do is have a list of or know the FileNames of each file lftped down so that I can do a md5sum and compare that hash to the hash that was created on the source.

I did move my single quote so that the >> $log is outside
as in
lftp -c 'open -e "set ftp:list-options -a; mirror -a -v array1/f_t/ ./media/facing_history_1/00" -u uname,psswd 172.16.100.101' >> $log

and that is closer
Now I can log ( thanks colucix ) and the log looks like
Transferring file 'FileName1.ext'
Transferring file 'FileName2.ext'
Transferring file 'FileName3.ext'

so that is way way closer . maybe I will do some sed awk to replace
the Transferring file with " " and the single quote for a " "
then I should have a list of

FileName1.ext
FileName2.ext
FileName3.ext

so -- thanks all
This is proving to be an interesting challenge.
 
Old 01-30-2013, 01:44 PM   #5
zimbot
Member
 
Registered: Nov 2005
Location: cincinnati , ohio . USA
Distribution: ubuntu , Opensuse , CentOS
Posts: 122

Original Poster
Rep: Reputation: 17
and a special thanks for the statement that the
-a means
-a same as --allow-chown --allow-suid --no-umask
not ascii . For me I Knew it was not a ascii xfer because the files were video ( file.mov) and they were playable

and for others who might read thsi -- in lftp binary is the default.
all my stuff above is using that default.
 
Old 01-30-2013, 02:10 PM   #6
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,488

Rep: Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956
Quote:
Originally Posted by zimbot View Post
Transferring file 'FileName1.ext'
Transferring file 'FileName2.ext'
Transferring file 'FileName3.ext'
Maybe you can just extract the text inside the single quotes using sed:
Code:
sed -i -r "s/.*'(.*)'/\1/" $log
If by chance there are other unwanted lines in the lftp log, you can selectively extract those ones with the string "Transferring file" and leave the rest:
Code:
sed -i -nr "/Transferring file/s/.*'(.*)'/\1/p" $log
 
Old 09-23-2013, 07:55 AM   #7
anarchist18
LQ Newbie
 
Registered: Oct 2011
Posts: 5

Rep: Reputation: Disabled
Hi,

Does anyone know how to monitor lftp transfers for failures? How to redirect messages from STDERR to a log file and that send that file as an attachment in a message to an administrator?

Or if you have other suggestions on how to monitor failed transfers or connectivity problems

Regards
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
how to lftp mirror in a bash zimbot Linux - Server 1 01-22-2013 01:09 PM
calling lftp from a bash script; howto pass variables into the command syntax stvy Programming 2 01-09-2013 01:43 PM
lftp mirror problem ycxbiao Linux - Software 1 09-27-2009 09:42 PM
lftp mirror problem crusher88 Linux - Software 2 08-14-2009 01:03 AM
filezilla setting and lftp:mirror help samengr Linux - General 0 01-25-2008 05:51 AM


All times are GMT -5. The time now is 05:11 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration