[SOLVED] Deleting not fully empty direcrory

mackowiakp · 05-09-2017, 08:18 AM

I use surveillance software witch collects clips from IP cameras. This software creates directory's on HDD named as date of creation video clip it contains.
Additionally each such directory contains "special" subdir, named

Code:

.@__thumb

containing thumbs of all video clips this directory contains.
The content of this "special subdir" is refreshed each day automatically by surveillance software.
So, time of creation of video clip is equal to modification time. But time of modification of "special subdir" is always set to today. So time of modification of all directory with all clips is set to today too.
Linux does not keep informations about creation time. Only access and modification time is available.
I want to remove all files and directory's older then 30 days. It is easy with video clips in the way like this

Code:

find /path_to_clips/dirs -type f -mtime +30 -exec rm -f {} \;

But after such operation, the directory is empty except subdirectory .@__thumb and its content (thumbs). So it looks like this (example)

Code:

/record/2017-05-07/.@__thumb

How can I modify the code line above, to delete all directory's containing only one hidden subdir?

PS
I can not use command

Code:

find /path_to_clips/dirs -type d -mtime +30 -exec rm -rf {} \;

because time of directory's above are always set to today. That is my problem. Any help?

pan64 · 05-09-2017, 09:18 AM

as far as I see you need python/perl or similar to implement that. You cannot solve it by modifying that code.

BW-userx · 05-09-2017, 09:22 AM

is that a dot file or a speck on my monitor screen?

Code:

userx%slackwhere ⚡ ~ ⚡> find ~/testdir -type f -name "*" -exec rm {} \;

filled with two types of files and directories. That command leaves me with this.

Code:

userx%slackwhere ⚡ ~ ⚡> ls -la testdir
total 28
drwxr-xr-x  7 userx users 4096 May  9 08:35 .
drwx--x--x 53 userx userx 4096 May  9 08:24 ..
drwxr-xr-x  2 userx users 4096 May  9 08:25 .hiddenone
drwxr-xr-x  2 userx users 4096 May  9 08:33 .my1
drwxr-xr-x  2 userx users 4096 May  9 08:33 my2
drwxr-xr-x  2 userx users 4096 May  9 08:33 my3
drwxr-xr-x  2 userx users 4096 May  9 08:33 my4

what are dot files again?

chrism01 · 05-10-2017, 02:02 AM

If this means what it says literally

Quote:

How can I modify the code line above, to delete all directory's containing only one hidden subdir?

then, as per pan64, try Perl, as you'll need to count how many hidden subdirs there are to get just the upper dir that contains only one hidden subdir (iiuc)
Try starting here https://docstore.mik.ua/orelly/perl4/cook/ch09_06.htm.

Quote:

what are dot files again?

Well, '.' is curr dir (aka pwd) and '..' is parent dir.
.xxx can be a dir or file and just contains stuff eg .mozilla in $HOME contains firefox cfgs etc.
dot files are often ignored by 'normal' cmds (wildcards) and need more special coding to catch them.

Turbocapitalist · 05-10-2017, 02:16 AM

You might be able to build a suitable query by adding parenthesis to take advantage of logical OR in find.

Code:

find /path_to_clips/dirs -type f -mtime +30 \
 -o \( -type d  -mtime +30 -name '.@__thumb' \) \
 -exec echo rm -rf {} \;

Or something like that.

mackowiakp · 05-10-2017, 02:56 AM

Quote:

Originally Posted by Turbocapitalist

You might be able to build a suitable query by adding parenthesis to take advantage of logical OR in find.

Code:

find /path_to_clips/dirs -type f -mtime +30 \
 -o \( -type d  -mtime +30 -name '.@__thumb' \) \
 -exec echo rm -rf {} \;

Or something like that.

It looks like right way. But how can I "find" directory containing only one subdir named ".@__thumb" and no other files?

pan64 · 05-10-2017, 03:15 AM

Quote:

Originally Posted by mackowiakp

It looks like right way. But how can I "find" directory containing only one subdir named ".@__thumb" and no other files?

again, find is unable to do that, this is not the right way.
You need to implement a script which can check if a directory contains only one subdir (named ".@__thumb") and nothing else.

Turbocapitalist · 05-10-2017, 05:07 AM

Quote:

Originally Posted by pan64

You need to implement a script which can check if a directory contains only one subdir (named ".@__thumb") and nothing else.

Correct, but that script, if short enough, can be part of the find formula as the expression option's return value will be used.

Code:

find /dir/subdir -type d -exec sh -c "true" \; -print

find /dir/subdir -type d -exec sh -c "false" \; -print

ondoho · 05-11-2017, 02:12 AM

i'm a little confused about the examples in post #1, and how you used placeholders like "dir" and "subdir".
would it be possible to show us the exact names and what you want to delete? maybe coupled with the output of 'tree'?

furthermore, i am thinking that the thumbnail's name should reflect on the movie clip's name.

MadeInGermany · 05-12-2017, 02:30 PM

I think you must postprocess the find output, e.g. with awk

Code:

find /record -type d -name ".?*" -prune -print -o -print |
awk -F/ '{ x=$0; sub("/[^/]*$","",x); c[x]++ } ($NF==".@__thumb") { d[x] } END { for (i in d) if (c[i]==1) print "rm -rf", i }'

Add a pipe to sh to run the rm commands.
Explanation: in order to run faster, find prunes all .xxx directories.
In the embedded awk code, x is the dirname of each pathname, c[] counts the directory entries, d[] stores the directories that have .@__thumb. At the END i loops through the d[] keys, and prints if the corresponding c[] is 1.

scasey · 05-12-2017, 10:27 PM

Hmmm.
When I execute

Code:

find /path_to_clips/dirs -type f -mtime +30

it can see dot files in dot directories.

Perhaps the rm needs to be recursive?:

Code:

-exec rm -rfR {} \;

Sean

[never mind...you already have -r in there...I shouldn't do this when I'm tired...sorry...]
Sean

Turbocapitalist · 05-13-2017, 12:17 AM

Here seems to be one way of finding them. The following recursively finds all directories that do contain a subdirectory named 'foo' but at the same time don't contain any regular files:

Code:

for f in $(find /some/directory -type d -exec test -d {}/foo \; -print); do 
        find $f -mindepth 1 -maxdepth 1 -type f -print | grep -m 1 -q . || echo $f; 
done

Note: it will choke on names with spaces.

pan64 · 05-13-2017, 01:35 AM

I'm really sorry, I have to say probably that works, but you made four loops which is very-very inefficient (a for loop, 2 find loops and a grep). And hard to understand.
That's why I suggested to use for example python, the os.walk module will deliver all the information you need to be able to decide.
You can also try to process the output of ls -lR too (which eventually runs faster than find) with awk.

Turbocapitalist · 05-13-2017, 01:50 AM

pan64, yeah, you're right. I see the loops and python (or perl) is the easy way to go here. For perl there is the File::Find module from CPAN.

I also now see that my suggestion will falsely identify directories that have no files but multiple subdirectories.

Turbocapitalist · 05-17-2017, 04:06 AM

Just to revisit this one since it is marked "solved" here's one quick way to do it in perl:

Code:

#!/usr/bin/perl -T

use File::Find;
use warnings;
use strict;

our @dirs = ();

my $directory = shift || exit ( 1 );

File::Find::find( 
    { 
        wanted=>\&wanted, 
        untaint=>1, 
        untaint_pattern => qr|^([-+@\w./]+)$| 
    }, $directory );

print qq(directories=\n),join("\n", @dirs),qq(\n);

exit ( 0 );

sub wanted {
    return ( 0 ) if ( $File::Find::dir =~ /\/foo$/ ||
                      ! -d $File::Find::name  || 
                      -d $File::Find::name && $File::Find::name =~ /\/foo$/ );
    
    my $d = $File::Find::name;

    opendir( DIR, $d )
        or die( "Could not open '$d': $!\n" );

    my @files = readdir( DIR );
    
    closedir( DIR );
    
    foreach my $f ( @files ) { 
        next if ( $f eq '.' || $f eq '..' );
        return ( 0 ) if ( -f "$d/$f" || -d "$d/$f" && $f ne 'foo' );
    }

    push( @dirs, $d );
    # print qq(deleting\trm -rf), $d, qq(\n);
    
    return ( 1 );
}

There's also a utility, find2perl, which can rewrite find formulas as perl for the File::Find module if a quick start is needed.