LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 05-09-2017, 07:18 AM   #1
mackowiakp
Member
 
Registered: Jun 2014
Location: Poland/Gdynia
Distribution: Mageia 7, SH4, Debian
Posts: 316

Rep: Reputation: 8
Deleting not fully empty direcrory


I use surveillance software witch collects clips from IP cameras. This software creates directory's on HDD named as date of creation video clip it contains.
Additionally each such directory contains "special" subdir, named
Code:
.@__thumb
containing thumbs of all video clips this directory contains.
The content of this "special subdir" is refreshed each day automatically by surveillance software.
So, time of creation of video clip is equal to modification time. But time of modification of "special subdir" is always set to today. So time of modification of all directory with all clips is set to today too.
Linux does not keep informations about creation time. Only access and modification time is available.
I want to remove all files and directory's older then 30 days. It is easy with video clips in the way like this
Code:
find /path_to_clips/dirs -type f -mtime +30 -exec rm -f {} \;
But after such operation, the directory is empty except subdirectory .@__thumb and its content (thumbs). So it looks like this (example)
Code:
/record/2017-05-07/.@__thumb
How can I modify the code line above, to delete all directory's containing only one hidden subdir?

PS
I can not use command
Code:
find /path_to_clips/dirs -type d -mtime +30 -exec rm -rf {} \;
because time of directory's above are always set to today. That is my problem. Any help?
 
Old 05-09-2017, 08:18 AM   #2
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 16,604

Rep: Reputation: 5604Reputation: 5604Reputation: 5604Reputation: 5604Reputation: 5604Reputation: 5604Reputation: 5604Reputation: 5604Reputation: 5604Reputation: 5604Reputation: 5604
as far as I see you need python/perl or similar to implement that. You cannot solve it by modifying that code.
 
Old 05-09-2017, 08:22 AM   #3
BW-userx
LQ Guru
 
Registered: Sep 2013
Location: Somewhere in my head.
Distribution: Slackware (current), FreeBSD, Win10, It varies
Posts: 9,952

Rep: Reputation: 2148Reputation: 2148Reputation: 2148Reputation: 2148Reputation: 2148Reputation: 2148Reputation: 2148Reputation: 2148Reputation: 2148Reputation: 2148Reputation: 2148
is that a dot file or a speck on my monitor screen?

Code:
userx%slackwhere ⚡ ~ ⚡> find ~/testdir -type f -name "*" -exec rm {} \;
filled with two types of files and directories. That command leaves me with this.
Code:
userx%slackwhere ⚡ ~ ⚡> ls -la testdir
total 28
drwxr-xr-x  7 userx users 4096 May  9 08:35 .
drwx--x--x 53 userx userx 4096 May  9 08:24 ..
drwxr-xr-x  2 userx users 4096 May  9 08:25 .hiddenone
drwxr-xr-x  2 userx users 4096 May  9 08:33 .my1
drwxr-xr-x  2 userx users 4096 May  9 08:33 my2
drwxr-xr-x  2 userx users 4096 May  9 08:33 my3
drwxr-xr-x  2 userx users 4096 May  9 08:33 my4
what are dot files again?

Last edited by BW-userx; 05-09-2017 at 08:58 AM.
 
Old 05-10-2017, 01:02 AM   #4
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 7.7 (?), Centos 8.1
Posts: 17,873

Rep: Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600
If this means what it says literally
Quote:
How can I modify the code line above, to delete all directory's containing only one hidden subdir?
then, as per pan64, try Perl, as you'll need to count how many hidden subdirs there are to get just the upper dir that contains only one hidden subdir (iiuc)
Try starting here https://docstore.mik.ua/orelly/perl4/cook/ch09_06.htm.


Quote:
what are dot files again?
Well, '.' is curr dir (aka pwd) and '..' is parent dir.
.xxx can be a dir or file and just contains stuff eg .mozilla in $HOME contains firefox cfgs etc.
dot files are often ignored by 'normal' cmds (wildcards) and need more special coding to catch them.
 
Old 05-10-2017, 01:16 AM   #5
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 5,655
Blog Entries: 3

Rep: Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901
You might be able to build a suitable query by adding parenthesis to take advantage of logical OR in find.

Code:
find /path_to_clips/dirs -type f -mtime +30 \
 -o \( -type d  -mtime +30 -name '.@__thumb' \) \
 -exec echo rm -rf {} \;
Or something like that.
 
Old 05-10-2017, 01:56 AM   #6
mackowiakp
Member
 
Registered: Jun 2014
Location: Poland/Gdynia
Distribution: Mageia 7, SH4, Debian
Posts: 316

Original Poster
Rep: Reputation: 8
Quote:
Originally Posted by Turbocapitalist View Post
You might be able to build a suitable query by adding parenthesis to take advantage of logical OR in find.

Code:
find /path_to_clips/dirs -type f -mtime +30 \
 -o \( -type d  -mtime +30 -name '.@__thumb' \) \
 -exec echo rm -rf {} \;
Or something like that.
It looks like right way. But how can I "find" directory containing only one subdir named ".@__thumb" and no other files?
 
Old 05-10-2017, 02:15 AM   #7
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 16,604

Rep: Reputation: 5604Reputation: 5604Reputation: 5604Reputation: 5604Reputation: 5604Reputation: 5604Reputation: 5604Reputation: 5604Reputation: 5604Reputation: 5604Reputation: 5604
Quote:
Originally Posted by mackowiakp View Post
It looks like right way. But how can I "find" directory containing only one subdir named ".@__thumb" and no other files?
again, find is unable to do that, this is not the right way.
You need to implement a script which can check if a directory contains only one subdir (named ".@__thumb") and nothing else.
 
Old 05-10-2017, 04:07 AM   #8
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 5,655
Blog Entries: 3

Rep: Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901
Quote:
Originally Posted by pan64 View Post
You need to implement a script which can check if a directory contains only one subdir (named ".@__thumb") and nothing else.
Correct, but that script, if short enough, can be part of the find formula as the expression option's return value will be used.

Code:
find /dir/subdir -type d -exec sh -c "true" \; -print

find /dir/subdir -type d -exec sh -c "false" \; -print
 
Old 05-11-2017, 01:12 AM   #9
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 17,393
Blog Entries: 10

Rep: Reputation: 5212Reputation: 5212Reputation: 5212Reputation: 5212Reputation: 5212Reputation: 5212Reputation: 5212Reputation: 5212Reputation: 5212Reputation: 5212Reputation: 5212
i'm a little confused about the examples in post #1, and how you used placeholders like "dir" and "subdir".
would it be possible to show us the exact names and what you want to delete? maybe coupled with the output of 'tree'?

furthermore, i am thinking that the thumbnail's name should reflect on the movie clip's name.
 
Old 05-12-2017, 01:30 PM   #10
MadeInGermany
Senior Member
 
Registered: Dec 2011
Location: Simplicity
Posts: 1,765

Rep: Reputation: 796Reputation: 796Reputation: 796Reputation: 796Reputation: 796Reputation: 796Reputation: 796
I think you must postprocess the find output, e.g. with awk
Code:
find /record -type d -name ".?*" -prune -print -o -print |
awk -F/ '{ x=$0; sub("/[^/]*$","",x); c[x]++ } ($NF==".@__thumb") { d[x] } END { for (i in d) if (c[i]==1) print "rm -rf", i }'
Add a pipe to sh to run the rm commands.
Explanation: in order to run faster, find prunes all .xxx directories.
In the embedded awk code, x is the dirname of each pathname, c[] counts the directory entries, d[] stores the directories that have .@__thumb. At the END i loops through the d[] keys, and prints if the corresponding c[] is 1.

Last edited by MadeInGermany; 05-12-2017 at 01:33 PM.
 
Old 05-12-2017, 09:27 PM   #11
scasey
LQ Veteran
 
Registered: Feb 2013
Location: Tucson, AZ, USA
Distribution: CentOS 7.8.2003
Posts: 5,432

Rep: Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055Reputation: 2055
Hmmm.
When I execute
Code:
find /path_to_clips/dirs -type f -mtime +30
it can see dot files in dot directories.

Perhaps the rm needs to be recursive?:
Code:
-exec rm -rfR {} \;
Sean

[never mind...you already have -r in there...I shouldn't do this when I'm tired...sorry...]
Sean

Last edited by scasey; 05-12-2017 at 09:34 PM.
 
Old 05-12-2017, 11:17 PM   #12
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 5,655
Blog Entries: 3

Rep: Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901
Here seems to be one way of finding them. The following recursively finds all directories that do contain a subdirectory named 'foo' but at the same time don't contain any regular files:

Code:
for f in $(find /some/directory -type d -exec test -d {}/foo \; -print); do 
        find $f -mindepth 1 -maxdepth 1 -type f -print | grep -m 1 -q . || echo $f; 
done
Note: it will choke on names with spaces.

Last edited by Turbocapitalist; 05-12-2017 at 11:25 PM. Reason: -m 1
 
1 members found this post helpful.
Old 05-13-2017, 12:35 AM   #13
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 16,604

Rep: Reputation: 5604Reputation: 5604Reputation: 5604Reputation: 5604Reputation: 5604Reputation: 5604Reputation: 5604Reputation: 5604Reputation: 5604Reputation: 5604Reputation: 5604
I'm really sorry, I have to say probably that works, but you made four loops which is very-very inefficient (a for loop, 2 find loops and a grep). And hard to understand.
That's why I suggested to use for example python, the os.walk module will deliver all the information you need to be able to decide.
You can also try to process the output of ls -lR too (which eventually runs faster than find) with awk.
 
Old 05-13-2017, 12:50 AM   #14
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 5,655
Blog Entries: 3

Rep: Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901
pan64, yeah, you're right. I see the loops and python (or perl) is the easy way to go here. For perl there is the File::Find module from CPAN.

I also now see that my suggestion will falsely identify directories that have no files but multiple subdirectories.
 
Old 05-17-2017, 03:06 AM   #15
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 5,655
Blog Entries: 3

Rep: Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901
code golf

Just to revisit this one since it is marked "solved" here's one quick way to do it in perl:

Code:
#!/usr/bin/perl -T

use File::Find;
use warnings;
use strict;

our @dirs = ();

my $directory = shift || exit ( 1 );

File::Find::find( 
    { 
        wanted=>\&wanted, 
        untaint=>1, 
        untaint_pattern => qr|^([-+@\w./]+)$| 
    }, $directory );

print qq(directories=\n),join("\n", @dirs),qq(\n);

exit ( 0 );

sub wanted {
    return ( 0 ) if ( $File::Find::dir =~ /\/foo$/ ||
                      ! -d $File::Find::name  || 
                      -d $File::Find::name && $File::Find::name =~ /\/foo$/ );
    
    my $d = $File::Find::name;

    opendir( DIR, $d )
        or die( "Could not open '$d': $!\n" );

    my @files = readdir( DIR );
    
    closedir( DIR );
    
    foreach my $f ( @files ) { 
        next if ( $f eq '.' || $f eq '..' );
        return ( 0 ) if ( -f "$d/$f" || -d "$d/$f" && $f ne 'foo' );
    }

    push( @dirs, $d );
    # print qq(deleting\trm -rf), $d, qq(\n);
    
    return ( 1 );
}
There's also a utility, find2perl, which can rewrite find formulas as perl for the File::Find module if a quick start is needed.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Deleting a non empty folder without having to change directories poadeosun1 Linux - Newbie 2 02-04-2015 08:32 PM
Deleting empty line at end of text file in BASH human2.0 Linux - General 8 04-01-2009 02:44 AM
Deleting empty/blank line with per script knockout_artist Programming 2 10-31-2008 01:25 PM
ProFTPD - deleting non-empty directories? orange400 Linux - Software 1 05-28-2004 03:36 AM
deleting non-empty directory sadiboyz Linux - Newbie 2 04-21-2003 09:29 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 10:50 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration