LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - General (https://www.linuxquestions.org/questions/linux-general-1/)
-   -   bash script to lftp mirror and log (https://www.linuxquestions.org/questions/linux-general-1/bash-script-to-lftp-mirror-and-log-4175446913/)

zimbot 01-23-2013 11:22 AM

bash script to lftp mirror and log
 
<code>
#!/bin/bash
#########_______ test of mirror
log=~/log/f_h_logfile.txt
batch=00
dest=/media/facing_history_1/$batch
cd $dest
echo start $(date) >> $log

DATE='date +%Y%m%d'
#mkdir $DATE
echo "Get files"

lftp -c 'open -e "set ftp:list-options -a; mirror -a -v array1/f_t/ ./media/facing_history_1/00" -u uname,psswd 172.16.100.101 >> $log'
echo end $(date) >> $log
echo ............ >> $log
</code>

but it does not log
lftp -c 'open -e "set ftp:list-options -a; mirror -a -v array1/f_t/ ./media/facing_history_1/00" -u uname,psswd 172.16.100.101 >> $log'
isn the same as
lftp -c 'open -e "set ftp:list-options -a; mirror -a -v array1/f_t/ ./media/facing_history_1/00" -u uname,psswd 172.16.100.101'

and it does not go to
echo end $(date) >> $log
echo ............ >> $log

how might one do that?
What I would really like to do is capture into a log the file names of the files in the session.
I am also trying to have a better understanding of what happens w the
lftp -c 'open -e "set ftp:list-options -a; mirror -a -v

I have looked at the lftp man page ( that is how i got this far ) but am a bit uncertain of my switch use.
example - I wish to do binary ( the default )
is my use of -a correct?

any wisdom would be appreciated

thanks , thanks much

H_TeXMeX_H 01-24-2013 05:45 AM

lftp has a log option that you may want to look at:

Code:

                  --log=FILE                write lftp  commands  being  exe-
                                            cuted to FILE


colucix 01-24-2013 05:59 AM

Quote:

Originally Posted by zimbot (Post 4876224)
lftp -c 'open -e "set ftp:list-options -a; mirror -a -v array1/f_t/ ./media/facing_history_1/00" -u uname,psswd 172.16.100.101 >> $log'

This doesn't log anything, since you have put redirection inside the lftp command. It should be outside the single quotes instead.

Also look at the ~/.lftp directory and see if it contains a file named transfer_log. If enabled it should have been populated by details about file transfers. If not, you can enable it explicitly by means of
Code:

set xfer:log 1
set xfer:log-file /path/to/transfer_log

the second option is not available in older versions of lftp, so that the path of the log file is stuck to its default value (~/.lftp/transfer_log).

Quote:

Originally Posted by zimbot (Post 4876224)
I have looked at the lftp man page ( that is how i got this far ) but am a bit uncertain of my switch use.
example - I wish to do binary ( the default )
is my use of -a correct?

The -a option in the mirror command has a completely different meaning:
Code:

-a            same as --allow-chown --allow-suid --no-umask
whereas in other commands like get or mget it makes lftp switch to ascii transfer.

zimbot 01-30-2013 01:27 PM

1st thanks everyone

Yes I have a log happening with ~/.lftp/transfer_log
and that is good but what I actually wish to do is have a list of or know the FileNames of each file lftped down so that I can do a md5sum and compare that hash to the hash that was created on the source.

I did move my single quote so that the >> $log is outside
as in
lftp -c 'open -e "set ftp:list-options -a; mirror -a -v array1/f_t/ ./media/facing_history_1/00" -u uname,psswd 172.16.100.101' >> $log

and that is closer
Now I can log ( thanks colucix ) and the log looks like
Transferring file 'FileName1.ext'
Transferring file 'FileName2.ext'
Transferring file 'FileName3.ext'

so that is way way closer . maybe I will do some sed awk to replace
the Transferring file with " " and the single quote for a " "
then I should have a list of

FileName1.ext
FileName2.ext
FileName3.ext

so -- thanks all
This is proving to be an interesting challenge.

zimbot 01-30-2013 01:44 PM

and a special thanks for the statement that the
-a means
-a same as --allow-chown --allow-suid --no-umask
not ascii . For me I Knew it was not a ascii xfer because the files were video ( file.mov) and they were playable

and for others who might read thsi -- in lftp binary is the default.
all my stuff above is using that default.

colucix 01-30-2013 02:10 PM

Quote:

Originally Posted by zimbot (Post 4880867)
Transferring file 'FileName1.ext'
Transferring file 'FileName2.ext'
Transferring file 'FileName3.ext'

Maybe you can just extract the text inside the single quotes using sed:
Code:

sed -i -r "s/.*'(.*)'/\1/" $log
If by chance there are other unwanted lines in the lftp log, you can selectively extract those ones with the string "Transferring file" and leave the rest:
Code:

sed -i -nr "/Transferring file/s/.*'(.*)'/\1/p" $log

anarchist18 09-23-2013 07:55 AM

Hi,

Does anyone know how to monitor lftp transfers for failures? How to redirect messages from STDERR to a log file and that send that file as an attachment in a message to an administrator?

Or if you have other suggestions on how to monitor failed transfers or connectivity problems

Regards


All times are GMT -5. The time now is 01:13 PM.