LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 08-18-2017, 01:08 AM   #16
ericlindellnyc
Member
 
Registered: Jun 2017
Posts: 129

Original Poster
Rep: Reputation: Disabled
Appending incrementing number to directory name.


Quote:
Originally Posted by Laserbeak View Post
So if you have a directory /path/to/some/of/my/really/cool/computer/files you want to move of/my/really/cool/computer/files to /path/to/cool/computer/files? What if there's a directory with that name already there? Merge them, or rename it by adding 1, 2, 3 etc.?
Laserbeak, thanks for the reply, and please excuse my delay in responding.

Yes, this is exactly the idea. I figured out (I think) how to "transplant" all directories at level 10 to level 3 (or whatever) -- with a simple mv.

But the problemo, as you observe, is name duplicates. I'd prefer to rename by appending incrementing number, though merging would probably work, too.

I've been using this, which unfortunately does not address duplicates:

CODE
find -maxdepth 10 -mindepth 10 -type d in ./* -exec echo mv -i

BACK TO TEXT
If you can suggest how to modify directory names to avoid collisions, that would help a lot.

Thanks much for your help with this.

Eric
 
Old 08-18-2017, 01:22 AM   #17
ericlindellnyc
Member
 
Registered: Jun 2017
Posts: 129

Original Poster
Rep: Reputation: Disabled
THANK YOU ALL FOR YOUR HELPFUL REPLIES.

I actually wrote a more detailed reply for each of your various approaches, but unfortunately it got zapped by accident, and it's too late for me to redo.

I'm going with appending incrementing number to avoid directory name collisions.

I might also try (if that doesn't work) combining path and filename into one filename.

Thank you again for your help, and please excuse my absence for the last month.

Eric
 
Old 09-12-2022, 08:28 PM   #18
ericlindellnyc
Member
 
Registered: Jun 2017
Posts: 129

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by JeremyBoden View Post
Why not use a symbolic link?

e.g. create a directory at level 5 called 'some-links' which points to a some deeply nested "parent directory"
It will ease your navigation through loads of files.

For example:-
Code:
mkdir ~/Desktop/Links
ls -l ~/Desktop/Links/Links
lrwxrwxrwx 1 jeremy jeremy   18 Nov 26  2015 Link to avi -> /mnt/filestore/avi
lrwxrwxrwx 1 jeremy jeremy   24 Nov 26  2015 Link to documents -> /mnt/filestore/documents
lrwxrwxrwx 1 jeremy jeremy   37 Nov 26  2015 Link to Manuals -> /mnt/filestore/books/Computer Manuals
lrwxrwxrwx 1 jeremy jeremy   23 Nov 26  2015 Link to pictures -> /mnt/filestore/pictures
lrwxrwxrwx 1 jeremy jeremy   10 Nov 26  2015 Link to video -> /mnt/video
drwxr-xr-x 2 jeremy jeremy 4096 Feb  6 16:54 PC-problems
then,
Code:
ls -l ~/Desktop/Links/'Link to video/'
total 54980
drwxrwxr-x 28 nobody public     4096 Mar 25 23:35 iplayer
-r--r--r--  1 jeremy public 45196229 May 10  2013 MVI_0357.mp4
drwxrwxr-x 10 nobody public     4096 Jul  3 00:27 radio
-r--r--r--  1 jeremy public  1930185 May 10  2013 Vid 001.mp4
-r--r--r--  1 jeremy public  1680817 May 10  2013 Vid 002.mp4
-r--r--r--  1 jeremy public  2798262 May 10  2013 Vid 003.mp4
-r--r--r--  1 jeremy public  2097162 May 10  2013 Vid 004.mp4
-r--r--r--  1 jeremy public  2565052 May 10  2013 Vid 006.mp4
drwxrwxr-x 10 nobody public     4096 Jul 20 16:20 video
Hi Jeremy . . just getting around to seeing this old post.
I agree symbolic links present an effective solution. However, I've had links broken in the past -- not sure how it happened. I'm afraid it will happen again. I'd like a solution that doesn't require me to remember there are symbolic links that can be broken.

Not sure if it was your suggestion, but appending incrementing number to directory name before moving it to a higher level seems like a good idea.
 
Old 09-12-2022, 08:40 PM   #19
ericlindellnyc
Member
 
Registered: Jun 2017
Posts: 129

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by scasey View Post
Or -- lets talk about the "errors in certain operating systems doing certain procedures" -- perhaps flattening the directory structure is not the best solution to the problem.
Nothing in particular -- it just kinda scares me to have way deeply nested directories . . sometimes as deep as 40 levels.
I'm afraid at some point in the future it might become an issue -- perhaps for some other OS, or some other backup system.
 
Old 09-12-2022, 08:46 PM   #20
ericlindellnyc
Member
 
Registered: Jun 2017
Posts: 129

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by Habitual View Post
Let's talk about backups.
One of my favorite topics.
Right now I just do a manual backup every so often -- at risk of losing stuff since the previous backup.
I've tried rsync, but it doesn't copy all the files for some reason.

I've just recently learned how to file-share, and I'd like to set this up with some rsync-like backup solution.
Placing the backup on a separate system may be more secure.

Also, it slows the system file operations down -- sometimes it seems like it gets stuck.

Last edited by ericlindellnyc; 09-12-2022 at 08:57 PM. Reason: add
 
Old 09-12-2022, 08:46 PM   #21
ericlindellnyc
Member
 
Registered: Jun 2017
Posts: 129

Original Poster
Rep: Reputation: Disabled
blank post

...

Last edited by ericlindellnyc; 09-12-2022 at 09:01 PM. Reason: posted by mistake
 
Old 09-12-2022, 09:04 PM   #22
ericlindellnyc
Member
 
Registered: Jun 2017
Posts: 129

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by scasey View Post
Ahh yes. I have experienced those kinds of issues, but the problem has been the number of files the command tried to return, not the depth of the file structure. Flattening the structure won't reduce the number of files. Can you show us what happens?
Definitely searching operations are slowed down to the point of stopping, it seems.
Also, copying deeply nested directory structures results in incomplete copying -- I think.
It can be hard to tell when you don't know if the system has just slowed down or stopped altogether.
 
Old 09-13-2022, 02:26 PM   #23
ericlindellnyc
Member
 
Registered: Jun 2017
Posts: 129

Original Poster
Rep: Reputation: Disabled
to Astrogeek

I wasn't sure if I could continue a thread from 5 years ago.
I figured it would be most likely seen if posted anew.
Thanks for letting me know.
 
Old 09-14-2022, 03:51 PM   #24
ericlindellnyc
Member
 
Registered: Jun 2017
Posts: 129

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by ericlindellnyc View Post
I would like to shorten some long paths in a deeply nested directory structure. The directory structure itself starts several levels down from root.

So let's say i have a directory structure with path depths varying from 1 to 15 (counting the directory structure as level 1).

I want to find all paths that go to depth of at least 5.

& then snip each path at level 5 and move it (along with its subdirectories) to level 2.

The net result of all this will have been to reduce maxdepth by 3 (i.e., from level 15 down to 12).

I don't want to flatten the directory structure -- that's more flattening than I need. I just need to flatten it a little so that it doesn't create errors in certain operating systems doing certain procedures.

I'd also like to include echoing each step to the standard output to make sure I'm not doing this on the entire file system.

Is there a way to do this?
Thanks.
EVL
I believe the "solution" that I used at the time of original post, 2017, was to move any directory at
-mindepth 15 -maxdepth 15
or something similar. But this posed 2 problems . .
1) It didn't check for duplicate names before moving to target directory.
Recently (5 years after OP) I learned how to append incrementing numbers in python.

2) It didn't determine a file's nesting depth. I can use the awkward -maxdepth & -mindepth from bash, guessing at the depth and then seeing what happens. This flattened only part of a huge directory.
More recently, I've researched how to find path length in python. This guy on reddit/learnpython describes my experience . .

Quote:
I tried googling "Python path length" but all the results are about Python max path length or stuff about physics.
Then I tried "Python get path size" but again everything was about how to get the total size of a directory in bytes.
Suppose I have a directory structure that includes /a/b/c/d/ee/FN1.ext -- and it also includes several other files and subdirectories with a path like /a/b/c/FN2.ext -- but no longer than that. In that case, the first one, /a/b/c/d/e/FN1.ext would be the longest -- of path length 5. (Or 6 if you count the file).
I'd like this output . .
Code:
longest path in directory structure is 5
Knowing that, I can then "prune the tree" by moving all files with a path length greater than Maxdepth to a target directory, appending incrementing number.
This may make the directory structure easier to search, back up, remove duplicates, and transfer to other file systems.
There's also a variation where the number returned is not the integer number of components in the directory path -- but rather the number of characters (alphanumeric, punctuation, delimiters) in the entire path. So the character count for
Code:
/a/bb/FN1.ext
is 13.

Then I can meet NTFS limit of 4096 characters for a path.
I can use my script to move empty folders to a target folder. But instead of testing if folder is empty, I'd test for whether its length exceeds some limit.
Code:
import os 
import shutil
counter = 0
for root, dirs, files in os.walk("."):
    for dirName in dirs:
        dirPath = os.path.join(root, dirName)

        ### if len(os.listdir(dirPath)) == 0: [THIS CONDITIONAL TO BE REPLACED WITH TEST OF PATH LENGTH] ###

            counter += 1
            incrDN = dirName + '_' + str(counter)
            shutil.move(dirPath, "/Volumes/TM3gbBu/destDirsMTsPy/" + incrDN)
 
Old 09-14-2022, 05:27 PM   #25
dugan
LQ Guru
 
Registered: Nov 2003
Location: Canada
Distribution: distro hopper
Posts: 10,619

Rep: Reputation: 5018Reputation: 5018Reputation: 5018Reputation: 5018Reputation: 5018Reputation: 5018Reputation: 5018Reputation: 5018Reputation: 5018Reputation: 5018Reputation: 5018
Quote:
Originally Posted by ericlindellnyc View Post
I don't want to flatten the directory structure -- that's more flattening than I need. I just need to flatten it a little so that it doesn't create errors in certain operating systems doing certain procedures.
Darn those "certain" (MS) operating systems!

Anyway, this isn't difficult to do in Python, and based on your posting history, you're more than capable.

Use this:

https://docs.python.org/3/library/pathlib.html

Last edited by dugan; 09-14-2022 at 05:36 PM.
 
Old 09-23-2022, 12:00 AM   #26
ericlindellnyc
Member
 
Registered: Jun 2017
Posts: 129

Original Poster
Rep: Reputation: Disabled
THIS IS THE CODE THAT FINALLY WORKED. THANKS TO ALL
Code:
import os
import shutil
counter = 0
for root, dirs, files in os.walk("."):
    for dirName in dirs:
        dirPath = os.path.join(root, dirName)

if os.path.isdir(dirPath):
            if dirName begins DG:
                if dirDepth > 10 and < 20:
                    counter += 1
                    incrDN = dirName + '_' + str(counter)
                    print('incrDN is ', incrDN)
                    shutil.move(dirPath, "/Volumes/ . . /destDirs" + incrDN)
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: Refactoring Deeply Nested Code LXer Syndicated Linux News 0 08-17-2011 04:10 AM
creating a link to a deeply nested folder like ~ = /home/username/ AlphaLexman Linux - Newbie 9 09-17-2009 06:31 PM
Convert directory structure from long file names in Linux to DOS 8.3 structure? manorina Linux - Software 5 09-12-2009 09:18 AM
Home Jail Folder Structure like Gobolinux Directory Structure luispt Linux - General 3 07-26-2008 06:46 PM
LXer: Retain and recall long paths with rr utility LXer Syndicated Linux News 0 10-16-2007 06:42 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 06:28 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration