LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 08-16-2022, 05:08 PM   #1
ericlindellnyc
Member
 
Registered: Jun 2017
Posts: 181

Rep: Reputation: Disabled
How to Write Python Script to Move Empty Folders to Target with Incrementing Name


I have an enormous number of empty folders interspersed through my 6TB HDD.

I would like to keep the folders as a precaution against accidentally deleting something I need.

So I've tried both python and bash. Haven't gotten too far with either.

I have this python script to remove empty directories
Code:
for p in Path(target_path).glob('**/*'):
    if p.is_dir() and len(list(p.iterdir())) == 0:
        os.removedirs(p)
I've tried this python script to append incrementing number
Code:
def uniquify(path, sep = ''):
    def name_sequence():
        count = IT.count()
        yield ''
        while True:
            yield '{s}{n:d}'.format(s = sep, n = next(count))
    orig = tempfile._name_sequence 
    with tempfile._once_lock:
        tempfile._name_sequence = name_sequence()
        path = os.path.normpath(path)
        dirname, basename = os.path.split(path)
        filename, ext = os.path.splitext(basename)
        fd, filename = tempfile.mkstemp(dir = dirname, prefix = filename, suffix = ext)
        tempfile._name_sequence = orig
    return filename
Can't seem to get them to work together.

In bash I've gotten this far
Code:
find . -type d -empty 
	-exec
    		filename=$(basename "$file")
    		extension=${filename##*.}
    		filename=${filename%.*}
    		mv "$file" "$(($filename + 1))".$extension
but I don't know how to combine all the necessary statements including moving file to target folder into an exec.

Also, the total number of "dots" in a folder name may be 0, 1, 2 or more. So the code that assumes an extension exists doesn't always apply.

Any assistance much appreciated !!
BTW -- if I'm on the wrong forum for python, I'll try to transfer this to another forum.
 
Old 08-16-2022, 10:20 PM   #2
michaelk
Moderator
 
Registered: Aug 2002
Posts: 25,668

Rep: Reputation: 5891Reputation: 5891Reputation: 5891Reputation: 5891Reputation: 5891Reputation: 5891Reputation: 5891Reputation: 5891Reputation: 5891Reputation: 5891Reputation: 5891
Code:
#!/bin/bash
while IFS= read -r -d '' dirname; do
    echo "$dirname"
    # your code here

done < <(find /path-of-directories -type d -empty -print0)
It would be much easier using bash. The number of "." does not necessarily matter since string manipulation will find the "last" match.

Last edited by michaelk; 08-17-2022 at 05:20 AM.
 
1 members found this post helpful.
Old 08-17-2022, 08:14 AM   #3
boughtonp
Senior Member
 
Registered: Feb 2007
Location: UK
Distribution: Debian
Posts: 3,588

Rep: Reputation: 2542Reputation: 2542Reputation: 2542Reputation: 2542Reputation: 2542Reputation: 2542Reputation: 2542Reputation: 2542Reputation: 2542Reputation: 2542Reputation: 2542

Why Python?

mv has a "--backup=numbered" option.

But if you're only dealing with empty directories, do you even need that?

Surely all you need is a text file record of the dirs that existed before you deleted them, then - if there are any issues - you mkdir -p the relevant path(s)?

 
1 members found this post helpful.
Old 08-17-2022, 09:23 AM   #4
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,772

Rep: Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302
something like this (?):
Code:
find <where> -type d -empty -print -delete
(probably)
 
Old 08-18-2022, 06:28 AM   #5
GazL
LQ Veteran
 
Registered: May 2008
Posts: 6,894

Rep: Reputation: 5013Reputation: 5013Reputation: 5013Reputation: 5013Reputation: 5013Reputation: 5013Reputation: 5013Reputation: 5013Reputation: 5013Reputation: 5013Reputation: 5013
If it's important to keep a record of owner, group and timestamp metadata for the dir then using cpio might be the best bet.

Something like this:
Code:
find /dir -type d -empty -print0 >/tmp/empty.list0 \
  && cpio -0ov </tmp/empty.list0 >/bkp/emptydirs.cpio \
  && xargs -0r rmdir -- </tmp/empty.list0
Then you can use cpio -tv to list them, and cpio -i to restore them (if need be).

Last edited by GazL; 08-18-2022 at 06:29 AM.
 
Old 08-18-2022, 07:48 AM   #6
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,772

Rep: Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302
find also has a -ls option (instead of -print) which will show the permission/ownership settings. But cannot be used directly to restore those dirs.
 
Old 08-18-2022, 10:39 PM   #7
ericlindellnyc
Member
 
Registered: Jun 2017
Posts: 181

Original Poster
Rep: Reputation: Disabled
Thank you all so much for your helpful suggestions. I will be trying them out in the near term and hope to have results to post here.
 
Old 08-18-2022, 11:04 PM   #8
ericlindellnyc
Member
 
Registered: Jun 2017
Posts: 181

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by michaelk View Post
Code:
#!/bin/bash
while IFS= read -r -d '' dirname; do
    echo "$dirname"
    # your code here

done < <(find /path-of-directories -type d -empty -print0)
It would be much easier using bash. The number of "." does not necessarily matter since string manipulation will find the "last" match.
Pursuant to the foregoing, I have tested the following code
Code:
while IFS= read -r -d  dirname; do 
	echo "$dirname" 
	mv --backup=numbered $dirname /Volumes/TM3gbBu/testEmptyFolderTrashScript/ 
done
upon execution of which it just hangs, with a greater-than symbol where the prompt goes in the terminal.

I'm not versed in the subtleties of IFS, and the echo in the first line of the while-loop doesn't execute to show me how far it's gotten.

I'm using dirname three times . . once with a dollar sign, once with quotes and a dollar sign, and once with neither. I'm not sure if this is consistent usage.

I've implemented what appears to be two single-quotes in the while-loop condition as a pair of single directional quotes, with nothing between them.

I also copied and pasted everything into a text editor and then back into my editor, to lose any extraneous characters.

I'm using the mv --backup=numbered option to avoid name collisions.

I'm running the script from the same directory where the empty directories are -- and then moving them to a directory at the top level on the same drive.

Not sure why it's hanging. Help much appreciated !!
 
Old 08-19-2022, 12:19 AM   #9
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,772

Rep: Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302
Quote:
Originally Posted by ericlindellnyc View Post
Not sure why it's hanging. Help much appreciated !!
because that read command is waiting for input.
You missed the redirection, which was the input for that while cycle:
Code:
done < <(find /path-of-directories -type d -empty -print0)
 
Old 08-19-2022, 06:48 AM   #10
boughtonp
Senior Member
 
Registered: Feb 2007
Location: UK
Distribution: Debian
Posts: 3,588

Rep: Reputation: 2542Reputation: 2542Reputation: 2542Reputation: 2542Reputation: 2542Reputation: 2542Reputation: 2542Reputation: 2542Reputation: 2542Reputation: 2542Reputation: 2542
Quote:
Originally Posted by ericlindellnyc View Post
I'm using dirname three times . . once with a dollar sign, once with quotes and a dollar sign, and once with neither. I'm not sure if this is consistent usage.
A dollar sign is used to read from a variable, and should almost always be quoted. When the variable is a path (file and/or directory), it must always be quoted when reading that variable.

ShellCheck is a tool which can highlight this and other issues (such as you incorrectly changing the single quotes to unicode characters).

You can consult the Bash manual (e.g. built-in read command) to understand what a command/option does).
Also, Wooledge's BashFAQ has plenty of common tasks (and mistakes to avoid), and can also give a clue how a command might look like.


Last edited by boughtonp; 08-19-2022 at 06:49 AM.
 
Old 08-19-2022, 07:52 AM   #11
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,772

Rep: Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302Reputation: 7302
oh yes, dirname and basename are executables in linux, therefore you need to avoid them in a shell script (as variables). But you can use them in a python script.
 
Old 08-19-2022, 08:53 AM   #12
michaelk
Moderator
 
Registered: Aug 2002
Posts: 25,668

Rep: Reputation: 5891Reputation: 5891Reputation: 5891Reputation: 5891Reputation: 5891Reputation: 5891Reputation: 5891Reputation: 5891Reputation: 5891Reputation: 5891Reputation: 5891
Quote:
Originally Posted by pan64 View Post
oh yes, dirname and basename are executables in linux, therefore you need to avoid them in a shell script (as variables). But you can use them in a python script.
Oops, my screw up...
 
Old 08-19-2022, 12:24 PM   #13
MadeInGermany
Senior Member
 
Registered: Dec 2011
Location: Simplicity
Posts: 2,779

Rep: Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197
Your find must have -print0
(print null-terminated) in order to work with the
read -r -d
that reads a null-terminated line.
And the while loops until all lines are read.

You feed the find output to the loop,
either with a pipe
Code:
find . -type d -empty -print0 |
while IFS= read -r -d  dirname
do
  echo mv --backup=numbered "$dirname" /Volumes/TM3gbBu/testEmptyFolderTrashScript/ 
done
Or with a redirection from a special file that in turn is fed by the find command.
Code:
while IFS= read -r -d  dirname
do
  echo mv --backup=numbered "$dirname" /Volumes/TM3gbBu/testEmptyFolderTrashScript/ 
done < <( find . -type d -empty -print0 )
Remove the echo to really run the mv
 
Old 08-20-2022, 11:56 AM   #14
ericlindellnyc
Member
 
Registered: Jun 2017
Posts: 181

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by MadeInGermany View Post
Your find must have -print0
(print null-terminated) in order to work with the
read -r -d
that reads a null-terminated line.
And the while loops until all lines are read.

You feed the find output to the loop,
either with a pipe
Code:
find . -type d -empty -print0 |
while IFS= read -r -d  dirname
do
  echo mv --backup=numbered "$dirname" /Volumes/TM3gbBu/testEmptyFolderTrashScript/ 
done
Or with a redirection from a special file that in turn is fed by the find command.
Code:
while IFS= read -r -d  dirname
do
  echo mv --backup=numbered "$dirname" /Volumes/TM3gbBu/testEmptyFolderTrashScript/ 
done < <( find . -type d -empty -print0 )
Remove the echo to really run the mv
Thank you all so much for your incredibly helpful insights and suggestions.
What finally worked was
Code:
find . -type d -empty -print0 | while IFS= read -r -d '' dirname; do mv -nv "$dirname" /Volumes/TM3gbBu/testEmptyFolderTrashScript/; done
I used spellcheck, which gives actionable error messages -- like replacing smart single quotes with unicode. For this, I copied single quotes from this page to a text editor.

Spellcheck said "do" had no matching done, so I put ; before done.

Strangely, this script moves some 0-byte FILES -- despite -type d option. This happens only for files with RTFD extension. I tried other EXTs, including 4 letter ones.

I can live with this, though ideally I'd like to remove 0-byte files as a separate process -- and modify the script to work variously with FN's starting with tilde or other cruft.

Last edited by ericlindellnyc; 08-20-2022 at 12:00 PM.
 
Old 08-20-2022, 12:40 PM   #15
MadeInGermany
Senior Member
 
Registered: Dec 2011
Location: Simplicity
Posts: 2,779

Rep: Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197
All in one line??
Yes then you must put a semicolon before the done.

You mean shellcheck not spellcheck.

What is a 0-byte file?
Is it named *.rtfd?
Then you can augment your find command with
! -name "*.rtfd"
or
-not -name "*.rtfd"
 
  


Reply

Tags
empty, folders, incrementing, python


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Shell Script to compare folders,Sub-Folders and Sub-Sub-Folders unix_72427 Programming 8 08-08-2012 02:51 PM
Write a script to move specific files in various folders to one folder linuxisthedevil Linux - General 13 11-18-2010 06:29 AM
LXer: Python Python Python (aka Python 3) LXer Syndicated Linux News 0 08-05-2009 08:30 PM
Incrementing 001 value in shell script RobertoBech Programming 6 08-20-2007 07:38 PM
bash script - incrementing a filename in a script tslinux Programming 10 08-05-2003 11:58 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 11:40 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration