LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Software (https://www.linuxquestions.org/questions/linux-software-2/)
-   -   How to obtain info that file is still open for ftp (https://www.linuxquestions.org/questions/linux-software-2/how-to-obtain-info-that-file-is-still-open-for-ftp-4175590383/)

mackowiakp 09-29-2016 01:20 AM

How to obtain info that file is still open for ftp
 
I use the server (Synology NAS) on which are copied by ftp, video clips from recorders of CCTV system. When file is copied, small script convert it from DAV format to AVI, using the statement below:

Code:

find ./ -type f -name "*.dav" -exec ffmpeg -y -i {} -vcodec copy -movflags +faststart {}.avi \;
The problem is that there is a lot of clips transmitted constantly from CCTV recorders to Synology. That is many clips is transmitted from recorders to Synology at the same time. So the statement above will try to convert even clips where transmission is not over yet. How can I exclude such non complete clips from conversion? I can not use "inotify" feature because there is not such facility in Synology Linux. I think that it is necessary to know that particular file is still open for writing and exclude it for conwersion. Any idea What to do?

Turbocapitalist 09-29-2016 04:03 AM

The way I can think of would be to make a very short script to a) take a filename from "find" as a parameter, b) find the full path of the file, and then c) check the output of "lsof" for that file including the full path using "grep" The script would return 0 or 1 depending on what is found, but inverted so that if the file is found it fails. It would be called by -exec just before the existing -exec there in "find".

chrism01 09-29-2016 04:08 AM

I'd write a script based on a loop like eg
Code:

while true
do
    find ./ -type f -name "*.dav" -exec ls -l {} \; >vfile.lis

    sleep 30  # or whatever

    # Now loop process vfile.lis, checking current file size.
    # if a given filesize is the same as 30 secs ago, its finished d/loading, so process it

# After processing all 'completed' files in vfile.lis, rinse & repeat
done

(Overlapped by TC; lsof would work as well ;) )

mackowiakp 09-29-2016 04:43 AM

Quote:

Originally Posted by chrism01 (Post 5611339)
I'd write a script based on a loop like eg
.............

(Overlapped by TC; lsof would work as well ;) )

I try to do as You advice several days ago. But amount of video clips from 12 CCTV recorders (each 4-8 cameras connected) is so high that practically all the time a video clips are transmitted. Actually, a few at the same time. Thus, the conversion process does not have a chance (using the script) to start.
Unfortunately there is no lsof command in Synology original software (very strange). The only possibility is to use flock command. If it resolve a problem of course.

Turbocapitalist 09-29-2016 05:05 AM

Quote:

Originally Posted by mackowiakp (Post 5611347)
The only possibility is to use flock command. If it resolve a problem of course.

Can you use a separate FTP session to grab each video clip? If so then "flock" might work because you could wrap the session in "flock" and check on it via "find". Again, you'd need a very short wrapper script to call from an extra -exec

mackowiakp 09-29-2016 05:17 AM

No I cant use separate FTP session to grab each video clip

Turbocapitalist 09-29-2016 05:48 AM

If it's not too nosy to ask, can you go into more detail about how the videos are getting to your machine? I'd like to think there is still a way to identify which ones are new and when they have finished downloading.

mackowiakp 09-29-2016 06:02 AM

Each CCTV recorder, transmit via FTP video clips to dedicated subdir on Synology. Recorders during transmission procedure, creates separate directory for each camera connected to one. So I have separeted dirs for each recorder and inside such dir, FTP creates separated subdir for each camera. Thats all

pan64 09-29-2016 06:17 AM

find .... -exec <Script>
the script first should rename the file, therefore the next find will not catch that.

Turbocapitalist 09-29-2016 06:21 AM

Maybe "fuser" is available? it will return 0 if the file is open and 1 if it is not. I don't have Synology to look at.

Looking around, there are add-ons via "ipkg" where you should be able to get "lsof"

mackowiakp 09-29-2016 06:28 AM

Hmmm. I dont understand. What about if the file will be in transmission state and the script renamed it?

pan64 09-29-2016 06:34 AM

if it was already opened (still writing it) - (you can check it with fuser for example) you can leave it as is.
It it was not opened at all rename it and process.

jpollard 09-29-2016 06:35 AM

Does the synology file server support modification time records? If it does, you MIGHT be able to determine the files that have not been updated in xxx time (where xxx is larger than the longest update interval). Thus files whose update time older than the current time - xxx would be subject to renaming/conversion.

mackowiakp 09-29-2016 06:37 AM

Quote:

Originally Posted by Turbocapitalist (Post 5611379)
Maybe "fuser" is available? it will return 0 if the file is open and 1 if it is not. I don't have Synology to look at.

Looking around, there are add-ons via "ipkg" where you should be able to get "lsof"

ipkg is a part of optware that means part of "crack" of the system by adding more features. I can not install it because according to directives, the system must be original because only in this case, Synology paid support wi help me in any problems.

fuser is absent too. The only potentially useful command existing in system is lslocks.

mackowiakp 09-29-2016 06:40 AM

Quote:

Originally Posted by pan64 (Post 5611383)
if it was already opened (still writing it) - (you can check it with fuser for example) you can leave it as is.
It it was not opened at all rename it and process.

There is no fuser command


All times are GMT -5. The time now is 09:46 AM.