[SOLVED] How to Write Python Script to Move Empty Folders to Target with Incrementing Name

ericlindellnyc · 08-16-2022, 05:08 PM

I have an enormous number of empty folders interspersed through my 6TB HDD.

I would like to keep the folders as a precaution against accidentally deleting something I need.

So I've tried both python and bash. Haven't gotten too far with either.

I have this python script to remove empty directories

Code:

for p in Path(target_path).glob('**/*'):
    if p.is_dir() and len(list(p.iterdir())) == 0:
        os.removedirs(p)

I've tried this python script to append incrementing number

Code:

def uniquify(path, sep = ''):
    def name_sequence():
        count = IT.count()
        yield ''
        while True:
            yield '{s}{n:d}'.format(s = sep, n = next(count))
    orig = tempfile._name_sequence 
    with tempfile._once_lock:
        tempfile._name_sequence = name_sequence()
        path = os.path.normpath(path)
        dirname, basename = os.path.split(path)
        filename, ext = os.path.splitext(basename)
        fd, filename = tempfile.mkstemp(dir = dirname, prefix = filename, suffix = ext)
        tempfile._name_sequence = orig
    return filename

Can't seem to get them to work together.

In bash I've gotten this far

Code:

find . -type d -empty 
	-exec
    		filename=$(basename "$file")
    		extension=${filename##*.}
    		filename=${filename%.*}
    		mv "$file" "$(($filename + 1))".$extension

but I don't know how to combine all the necessary statements including moving file to target folder into an exec.

Also, the total number of "dots" in a folder name may be 0, 1, 2 or more. So the code that assumes an extension exists doesn't always apply.

Any assistance much appreciated !!
BTW -- if I'm on the wrong forum for python, I'll try to transfer this to another forum.

michaelk · 08-16-2022, 10:20 PM

Code:

#!/bin/bash
while IFS= read -r -d '' dirname; do
    echo "$dirname"
    # your code here

done < <(find /path-of-directories -type d -empty -print0)

It would be much easier using bash. The number of "." does not necessarily matter since string manipulation will find the "last" match.

boughtonp · 08-17-2022, 08:14 AM

Why Python?

mv has a "--backup=numbered" option.

But if you're only dealing with empty directories, do you even need that?

Surely all you need is a text file record of the dirs that existed before you deleted them, then - if there are any issues - you mkdir -p the relevant path(s)?

pan64 · 08-17-2022, 09:23 AM

something like this (?):

Code:

find <where> -type d -empty -print -delete

(probably)

GazL · 08-18-2022, 06:28 AM

If it's important to keep a record of owner, group and timestamp metadata for the dir then using cpio might be the best bet.

Something like this:

Code:

find /dir -type d -empty -print0 >/tmp/empty.list0 \
  && cpio -0ov </tmp/empty.list0 >/bkp/emptydirs.cpio \
  && xargs -0r rmdir -- </tmp/empty.list0

Then you can use cpio -tv to list them, and cpio -i to restore them (if need be).

pan64 · 08-18-2022, 07:48 AM

find also has a -ls option (instead of -print) which will show the permission/ownership settings. But cannot be used directly to restore those dirs.

ericlindellnyc · 08-18-2022, 10:39 PM

Thank you all so much for your helpful suggestions. I will be trying them out in the near term and hope to have results to post here.

ericlindellnyc · 08-18-2022, 11:04 PM

Quote:

Originally Posted by michaelk

Code:

#!/bin/bash
while IFS= read -r -d '' dirname; do
    echo "$dirname"
    # your code here

done < <(find /path-of-directories -type d -empty -print0)

It would be much easier using bash. The number of "." does not necessarily matter since string manipulation will find the "last" match.

Pursuant to the foregoing, I have tested the following code

Code:

while IFS= read -r -d ‘’ dirname; do 
	echo "$dirname" 
	mv --backup=numbered $dirname /Volumes/TM3gbBu/testEmptyFolderTrashScript/ 
done

upon execution of which it just hangs, with a greater-than symbol where the prompt goes in the terminal.

I'm not versed in the subtleties of IFS, and the echo in the first line of the while-loop doesn't execute to show me how far it's gotten.

I'm using dirname three times . . once with a dollar sign, once with quotes and a dollar sign, and once with neither. I'm not sure if this is consistent usage.

I've implemented what appears to be two single-quotes in the while-loop condition as a pair of single directional quotes, with nothing between them.

I also copied and pasted everything into a text editor and then back into my editor, to lose any extraneous characters.

I'm using the mv --backup=numbered option to avoid name collisions.

I'm running the script from the same directory where the empty directories are -- and then moving them to a directory at the top level on the same drive.

Not sure why it's hanging. Help much appreciated !!

pan64 · 08-19-2022, 12:19 AM

Quote:

Originally Posted by ericlindellnyc

Not sure why it's hanging. Help much appreciated !!

because that read command is waiting for input.
You missed the redirection, which was the input for that while cycle:

Code:

done < <(find /path-of-directories -type d -empty -print0)

boughtonp · 08-19-2022, 06:48 AM

Quote:

Originally Posted by ericlindellnyc

I'm using dirname three times . . once with a dollar sign, once with quotes and a dollar sign, and once with neither. I'm not sure if this is consistent usage.

A dollar sign is used to read from a variable, and should almost always be quoted. When the variable is a path (file and/or directory), it must always be quoted when reading that variable.

ShellCheck is a tool which can highlight this and other issues (such as you incorrectly changing the single quotes to unicode characters).

You can consult the Bash manual (e.g. built-in read command) to understand what a command/option does).
Also, Wooledge's BashFAQ has plenty of common tasks (and mistakes to avoid), and can also give a clue how a command might look like.

pan64 · 08-19-2022, 07:52 AM

oh yes, dirname and basename are executables in linux, therefore you need to avoid them in a shell script (as variables). But you can use them in a python script.

michaelk · 08-19-2022, 08:53 AM

Quote:

Originally Posted by pan64

oh yes, dirname and basename are executables in linux, therefore you need to avoid them in a shell script (as variables). But you can use them in a python script.

Oops, my screw up...

MadeInGermany · 08-19-2022, 12:24 PM

Your find must have -print0
(print null-terminated) in order to work with the
read -r -d ‘’
that reads a null-terminated line.
And the while loops until all lines are read.

You feed the find output to the loop,
either with a pipe

Code:

find . -type d -empty -print0 |
while IFS= read -r -d ‘’ dirname
do
  echo mv --backup=numbered "$dirname" /Volumes/TM3gbBu/testEmptyFolderTrashScript/ 
done

Or with a redirection from a special file that in turn is fed by the find command.

Code:

while IFS= read -r -d ‘’ dirname
do
  echo mv --backup=numbered "$dirname" /Volumes/TM3gbBu/testEmptyFolderTrashScript/ 
done < <( find . -type d -empty -print0 )

Remove the echo to really run the mv

ericlindellnyc · 08-20-2022, 11:56 AM

Quote:

Originally Posted by MadeInGermany

Your find must have -print0
(print null-terminated) in order to work with the
read -r -d ‘’
that reads a null-terminated line.
And the while loops until all lines are read.

You feed the find output to the loop,
either with a pipe

Code:

find . -type d -empty -print0 |
while IFS= read -r -d ‘’ dirname
do
  echo mv --backup=numbered "$dirname" /Volumes/TM3gbBu/testEmptyFolderTrashScript/ 
done

Or with a redirection from a special file that in turn is fed by the find command.

Code:

while IFS= read -r -d ‘’ dirname
do
  echo mv --backup=numbered "$dirname" /Volumes/TM3gbBu/testEmptyFolderTrashScript/ 
done < <( find . -type d -empty -print0 )

Remove the echo to really run the mv

Thank you all so much for your incredibly helpful insights and suggestions.
What finally worked was

Code:

find . -type d -empty -print0 | while IFS= read -r -d '' dirname; do mv -nv "$dirname" /Volumes/TM3gbBu/testEmptyFolderTrashScript/; done

I used spellcheck, which gives actionable error messages -- like replacing smart single quotes with unicode. For this, I copied single quotes from this page to a text editor.

Spellcheck said "do" had no matching done, so I put ; before done.

Strangely, this script moves some 0-byte FILES -- despite -type d option. This happens only for files with RTFD extension. I tried other EXTs, including 4 letter ones.

I can live with this, though ideally I'd like to remove 0-byte files as a separate process -- and modify the script to work variously with FN's starting with tilde or other cruft.

MadeInGermany · 08-20-2022, 12:40 PM

All in one line??
Yes then you must put a semicolon before the done.

You mean shellcheck not spellcheck.

What is a 0-byte file?
Is it named *.rtfd?
Then you can augment your find command with
! -name "*.rtfd"
or
-not -name "*.rtfd"