LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 04-29-2014, 08:10 AM   #1
bhushan.patil
LQ Newbie
 
Registered: Apr 2014
Posts: 3

Rep: Reputation: Disabled
Unhappy Is lsof command list out completely downloaded files?


I am trying to write unix script which will help to move files from dir A to dir B.
dir A will get files from ftp connection.

ftp >> dir A >> dir B

I have no issue to second part i.e. move files from dir A >> dir B

My doubt is if I can use lsof command to list out all files which are completely ftp'd to dir A?

and do lsof command can check on completely downloaded files and ommit partial downloaded at dir A?
 
Old 04-29-2014, 08:16 AM   #2
cin_
Member
 
Registered: Dec 2010
Posts: 277

Rep: Reputation: 24
man pages

linux commands come packaged with man pages that are manuals for each command's use

they also describe what the command does

just type man in front of the command name..

Code:
# man lsof

Code:
lsof - list open files
..
 An  open file may be a regular file, a directory, a block special file,
       a character special file, an executing text  reference,  a  library,  a
       stream  or  a  network  file  (Internet socket, NFS file or UNIX domain
       socket.)  A specific file or all the files in  a  file  system  may  be
       selected by path.

       Instead  of  a  formatted display, lsof will produce output that can be
       parsed by other programs.  See the -F, option description, and the OUT‐
       PUT FOR OTHER PROGRAMS section for more information.
so standalone lsof will show all open files on the system..
you could grep out a specific directory

Code:
# lsof | grep /dirA
..
basically while the file is downloading you will see a lot of results in there from all of the different services using the file causing it to be in an open state

once the file is completely downloaded, or the download fails and ceases,
the output of lsof on that directory will be empty

theoretically you could write a bash script that watches a dir and determines if the output of lsof is populated or empty and assign actions for each outcome:
populated : check again in 17 seconds
empty : alert the file has completed downloading

this is useless in determining if the file completely downloaded,
for that you would have to write another part of the script that checks the size of the file on your system to see if it matches the size of the file on the ftp server

also it would be unnecessarily slow,
it would be better to write a script that performs the get and then when the get finishes alerts you with each file

Last edited by cin_; 04-29-2014 at 08:41 AM. Reason: gramm`err
 
1 members found this post helpful.
Old 04-29-2014, 08:24 AM   #3
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,711

Rep: Reputation: 1279Reputation: 1279Reputation: 1279Reputation: 1279Reputation: 1279Reputation: 1279Reputation: 1279Reputation: 1279Reputation: 1279
Quote:
Originally Posted by bhushan.patil View Post
I am trying to write unix script which will help to move files from dir A to dir B.
dir A will get files from ftp connection.

ftp >> dir A >> dir B

I have no issue to second part i.e. move files from dir A >> dir B

My doubt is if I can use lsof command to list out all files which are completely ftp'd to dir A?

and do lsof command can check on completely downloaded files and ommit partial downloaded at dir A?
Check out the "wget" utility which handles FTP. IT has some methods of handling incomplete transfer, and suitable exit codes to indicate a failure.
 
Old 04-29-2014, 08:29 AM   #4
tronayne
Senior Member
 
Registered: Oct 2003
Location: Northeastern Michigan, where Carhartt is a Designer Label
Distribution: Slackware 32- & 64-bit Stable
Posts: 3,541

Rep: Reputation: 1060Reputation: 1060Reputation: 1060Reputation: 1060Reputation: 1060Reputation: 1060Reputation: 1060Reputation: 1060
Well, no, that's not what lsof does:
Quote:
Lsof revision 4.83 lists on its standard output file information about files opened by processes...
More typically you would simply use ls to see the list of files. Command line options for ls that might be useful would be
Quote:
-a do not ignore entries starting with .
-C list entries by colums
-l use a long listing format
See the manual pages for both utilities for more.

I do wonder why you are using ftp to copy or move files from one directory to another; the cp utility (or the mv utility) can do the same thing, possibly more quickly.

You would use cp -pr dirA dirB to recursively copy the content of dirA to dirB. You can also simply rename (with the mv utility) dirA to dirB and not copy anything if both directories are in the same disk partition (if they are in separate disk partitions mv will do the recursive copy for you); across a network interface it's a little more complicated, however.

Hope this helps some.
 
Old 04-29-2014, 09:45 AM   #5
bhushan.patil
LQ Newbie
 
Registered: Apr 2014
Posts: 3

Original Poster
Rep: Reputation: Disabled
No actually I am getting files from other server through ftp connection @ dirA
So far now I have write script like this:

CHECK=`ls -al dirA |grep flagfile |wc -l`
cd dirA
touch dirA/flagfile
echo `ls -ltr`> HOMEDIR/result1.txt
sleep 5
echo `ls -ltr`> HOMEDIR/result2.txt
diff HOMEDIR/result1.txt HOMEDIR/result2.txt > HOMEDIR/compare.txt
COMPARE=`cat HOMEDIR/compare.txt|wc -l`
if this is compare is "0" it means my file is ready to move at dirB

but what if I don't want to spend time on "sleep 5". In fact I can afford to lose this time..
as there would be thousands of files floating.

Is there any alternate way I can do that speedily.
 
Old 04-29-2014, 12:14 PM   #6
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,711

Rep: Reputation: 1279Reputation: 1279Reputation: 1279Reputation: 1279Reputation: 1279Reputation: 1279Reputation: 1279Reputation: 1279Reputation: 1279
Why are you doing that?

Like I indicated earlier - use wget. When it finishes with a successful transfer, move the file.
 
Old 04-30-2014, 12:48 AM   #7
bhushan.patil
LQ Newbie
 
Registered: Apr 2014
Posts: 3

Original Poster
Rep: Reputation: Disabled
Not sure why you suggesting me to use WGET whereas I am not downloading any files WEB.
file processing happening like this.

Other server >> ftp'd files >> dirA >> dirB

dirA and dirB are same unix server.

since we have to use ftp I am wondering how can save my time to check if files are completely doenloaded at dir A from other server. (through ftp)
 
Old 04-30-2014, 03:19 AM   #8
ndc85430
Member
 
Registered: Apr 2014
Distribution: Slackware
Posts: 92

Rep: Reputation: Disabled
Sigh. Wget is just a file retrieval utility that supports FTP and other protocols. The point is that you can use it to obtain the file you need and check that the transfer was successful, presumably by checking the return code. If you're using Bash, this will be stored in the $? variable.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
md5sum error of a completely downloaded debian ISO rajuchal Linux - Newbie 5 04-28-2011 12:08 AM
LXer: Linux Process Management: Using lsof to List Open Files LXer Syndicated Linux News 0 04-10-2009 08:20 AM
How to ensure that the zip file is completely downloaded. indiancosmonaut Programming 10 05-01-2008 12:48 PM
Command to run another command against a list of files psweetma Linux - General 3 11-09-2005 06:29 PM
SuperKaramba remove downloaded themes completely jimdaworm Linux - Software 8 09-21-2005 12:45 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 06:52 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration