LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 06-26-2017, 02:17 PM   #1
ericlindellnyc
Member
 
Registered: Jun 2017
Posts: 88

Rep: Reputation: Disabled
shorten long paths in deeply nested directory structure


I would like to shorten some long paths in a deeply nested directory structure. The directory structure itself starts several levels down from root.

So let's say i have a directory structure with path depths varying from 1 to 15 (counting the directory structure as level 1).

I want to find all paths that go to depth of at least 5.

& then snip each path at level 5 and move it (along with its subdirectories) to level 2.

The net result of all this will have been to reduce maxdepth by 3 (i.e., from level 15 down to 12).

I don't want to flatten the directory structure -- that's more flattening than I need. I just need to flatten it a little so that it doesn't create errors in certain operating systems doing certain procedures.

I'd also like to include echoing each step to the standard output to make sure I'm not doing this on the entire file system.

Is there a way to do this?
Thanks.
EVL
 
Old 06-26-2017, 02:23 PM   #2
Laserbeak
Member
 
Registered: Jan 2017
Location: Manhattan, NYC NY
Distribution: Mac OS X, iOS, Solaris
Posts: 508

Rep: Reputation: 143Reputation: 143
Is a Perl program acceptable to you?
 
Old 06-27-2017, 10:49 AM   #3
scasey
LQ Veteran
 
Registered: Feb 2013
Location: Tucson, AZ, USA
Distribution: CentOS 7.8.2003
Posts: 5,442

Rep: Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070
Or -- lets talk about the "errors in certain operating systems doing certain procedures" -- perhaps flattening the directory structure is not the best solution to the problem.
 
Old 07-01-2017, 03:36 PM   #4
ericlindellnyc
Member
 
Registered: Jun 2017
Posts: 88

Original Poster
Rep: Reputation: Disabled
Perl Program to shorten directory paths

Quote:
Originally Posted by Laserbeak View Post
Is a Perl program acceptable to you?
Yes, absolutely .. As long as I know how to execute it .. not sure I've ever run a Perl program.
 
Old 07-01-2017, 03:40 PM   #5
ericlindellnyc
Member
 
Registered: Jun 2017
Posts: 88

Original Poster
Rep: Reputation: Disabled
flatten directory structure vs errors in OS

Quote:
Originally Posted by scasey View Post
Or -- lets talk about the "errors in certain operating systems doing certain procedures" -- perhaps flattening the directory structure is not the best solution to the problem.
Great idea .. how do I translate that into a solution?

BTW, my printout of the directory tree suggests nesting up to 20 levels deep.

I'd probly wanna flatten it to maybe 10 levels -- even if it's an OS problem I can circumvent.

Which procedures of which OS might be problematic with deeply nested directories?

Thanks much for your help.

Eric
 
Old 07-01-2017, 03:53 PM   #6
Laserbeak
Member
 
Registered: Jan 2017
Location: Manhattan, NYC NY
Distribution: Mac OS X, iOS, Solaris
Posts: 508

Rep: Reputation: 143Reputation: 143
So if you have a directory /path/to/some/of/my/really/cool/computer/files you want to move of/my/really/cool/computer/files to /path/to/cool/computer/files? What if there's a directory with that name already there? Merge them, or rename it by adding 1, 2, 3 etc.?
 
Old 07-01-2017, 03:54 PM   #7
scasey
LQ Veteran
 
Registered: Feb 2013
Location: Tucson, AZ, USA
Distribution: CentOS 7.8.2003
Posts: 5,442

Rep: Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070
Quote:
Originally Posted by ericlindellnyc View Post
Great idea .. how do I translate that into a solution?

BTW, my printout of the directory tree suggests nesting up to 20 levels deep.

I'd probly wanna flatten it to maybe 10 levels -- even if it's an OS problem I can circumvent.

Which procedures of which OS might be problematic with deeply nested directories?

Thanks much for your help.

Eric
I'm basically suggesting that the solution to your problems might not be to flatten the directory structure, but lie elsewhere. I could be wrong, of course, or you may not care to investigate another solution.

If you do, please explain what problems you are having that you're attributing to the depth of your directory structure. That is: Which procedures are causing errors because of the long path names? What are you trying to do that not working because of long path names?

If not, please excuse me...
 
Old 07-20-2017, 05:10 PM   #8
ericlindellnyc
Member
 
Registered: Jun 2017
Posts: 88

Original Poster
Rep: Reputation: Disabled
If there's a directory with that name already there, rename with incrementing number.

Quote:
Originally Posted by Laserbeak View Post
So if you have a directory /path/to/some/of/my/really/cool/computer/files you want to move of/my/really/cool/computer/files to /path/to/cool/computer/files? What if there's a directory with that name already there? Merge them, or rename it by adding 1, 2, 3 etc.?
Good point .. Probly rename with incrementing number.

Thanks.

Eric
 
Old 07-20-2017, 05:16 PM   #9
ericlindellnyc
Member
 
Registered: Jun 2017
Posts: 88

Original Poster
Rep: Reputation: Disabled
what problems you are having that you're attributing to the depth of your directory structure

[QUOTE=scasey;5729562]I'm basically suggesting that the solution to your problems might not be to flatten the directory structure, but lie elsewhere. I could be wrong, of course, or you may not care to investigate another solution.

If you do, please explain what problems you are having that you're attributing to the depth of your directory structure. That is: Which procedures are causing errors because of the long path names? What are you trying to do that not working because of long path names?


I SEE YOUR POINT .. One thing is copying for the sake of backup. Searching for files by content.

But even more, just I never know what I'm going to try to do with the files in future -- or on which system (I use mac/win/linux/android). So I just kinda wanna know that whatever I try to do in future will go without a hitch -- at least not cuz excessive nesting depth.

BTW -- the whole thing right now is on exfat file system hooked up to mac (easy to reconnect to linux, though)

Thank you.

Eric
 
Old 07-20-2017, 08:08 PM   #10
scasey
LQ Veteran
 
Registered: Feb 2013
Location: Tucson, AZ, USA
Distribution: CentOS 7.8.2003
Posts: 5,442

Rep: Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070
Quote:
Originally Posted by ericlindellnyc View Post
I SEE YOUR POINT .. One thing is copying for the sake of backup. Searching for files by content.

But even more, just I never know what I'm going to try to do with the files in future -- or on which system (I use mac/win/linux/android). So I just kinda wanna know that whatever I try to do in future will go without a hitch -- at least not cuz excessive nesting depth.

BTW -- the whole thing right now is on exfat file system hooked up to mac (easy to reconnect to linux, though)

Thank you.

Eric
Ahh yes. I have experienced those kinds of issues, but the problem has been the number of files the command tried to return, not the depth of the file structure. Flattening the structure won't reduce the number of files. Can you show us what happens?
 
Old 07-21-2017, 01:44 PM   #11
ericlindellnyc
Member
 
Registered: Jun 2017
Posts: 88

Original Poster
Rep: Reputation: Disabled
problem is number of files -- not depth of nesting

Quote:
Originally Posted by scasey View Post
Ahh yes. I have experienced those kinds of issues, but the problem has been the number of files the command tried to return, not the depth of the file structure. Flattening the structure won't reduce the number of files. Can you show us what happens?
I'm going to try to remember or duplicate what happened & post it then .... For now, I seem to recall copying all files from a failing drive, but all files didn't copy, & I believe nesting may have been the issue. I'd like to toss the old drive, but not till all files have copied.

For now, I just have a general memory of something going wrong when data was deeply nested -- but not when i reduced the nesting.
I don't recall how many files there were in either case.

I'd be curious if there's a max number of files one can use in one operation on mac/linux/win etc .... If there is such limit, it would be a good thing to know.

I've heard of a limit on path length, i.e., number of characters. But the previous 256 (+ or -) has been transcended on windows & doesn't exist on mac or linux, I believe.

Thanks for your continued feedback.
 
Old 07-21-2017, 05:25 PM   #12
JeremyBoden
Senior Member
 
Registered: Nov 2011
Location: London, UK
Distribution: Debian
Posts: 1,896

Rep: Reputation: 479Reputation: 479Reputation: 479Reputation: 479Reputation: 479
Why not use a symbolic link?

e.g. create a directory at level 5 called 'some-links' which points to a some deeply nested "parent directory"
It will ease your navigation through loads of files.

For example:-
Code:
mkdir ~/Desktop/Links
ls -l ~/Desktop/Links/Links
lrwxrwxrwx 1 jeremy jeremy   18 Nov 26  2015 Link to avi -> /mnt/filestore/avi
lrwxrwxrwx 1 jeremy jeremy   24 Nov 26  2015 Link to documents -> /mnt/filestore/documents
lrwxrwxrwx 1 jeremy jeremy   37 Nov 26  2015 Link to Manuals -> /mnt/filestore/books/Computer Manuals
lrwxrwxrwx 1 jeremy jeremy   23 Nov 26  2015 Link to pictures -> /mnt/filestore/pictures
lrwxrwxrwx 1 jeremy jeremy   10 Nov 26  2015 Link to video -> /mnt/video
drwxr-xr-x 2 jeremy jeremy 4096 Feb  6 16:54 PC-problems
then,
Code:
ls -l ~/Desktop/Links/'Link to video/'
total 54980
drwxrwxr-x 28 nobody public     4096 Mar 25 23:35 iplayer
-r--r--r--  1 jeremy public 45196229 May 10  2013 MVI_0357.mp4
drwxrwxr-x 10 nobody public     4096 Jul  3 00:27 radio
-r--r--r--  1 jeremy public  1930185 May 10  2013 Vid 001.mp4
-r--r--r--  1 jeremy public  1680817 May 10  2013 Vid 002.mp4
-r--r--r--  1 jeremy public  2798262 May 10  2013 Vid 003.mp4
-r--r--r--  1 jeremy public  2097162 May 10  2013 Vid 004.mp4
-r--r--r--  1 jeremy public  2565052 May 10  2013 Vid 006.mp4
drwxrwxr-x 10 nobody public     4096 Jul 20 16:20 video
 
Old 07-22-2017, 08:14 AM   #13
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
Quote:
Originally Posted by scasey View Post
Or -- lets talk about the "errors in certain operating systems doing certain procedures" -- perhaps flattening the directory structure is not the best solution to the problem.
Let's talk about backups.
 
Old 07-22-2017, 11:19 AM   #14
scasey
LQ Veteran
 
Registered: Feb 2013
Location: Tucson, AZ, USA
Distribution: CentOS 7.8.2003
Posts: 5,442

Rep: Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070Reputation: 2070
Quote:
Originally Posted by Habitual View Post
Let's talk about backups.
Certainly. I suspect a
Code:
cp -R /* /some/destination
got "too many arguments" sometime in the past, but (as you know), that's probably not the best way to make a backup.

I use rsnapshot, which uses rsync, to make off-site backups of my production server and same-site backups of my backup server.
 
Old 07-22-2017, 01:07 PM   #15
JeremyBoden
Senior Member
 
Registered: Nov 2011
Location: London, UK
Distribution: Debian
Posts: 1,896

Rep: Reputation: 479Reputation: 479Reputation: 479Reputation: 479Reputation: 479
One of the problems for backups is that it is easier to take a backup than to do a proper restore!

rsync has quite a complex set of possible parameters...

I use backup2l which produces daily tar.gz files, so in a dire emergency, I could do a "manual" restore; even if I'd lost all details of the backup parameters.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: Refactoring Deeply Nested Code LXer Syndicated Linux News 0 08-17-2011 04:10 AM
creating a link to a deeply nested folder like ~ = /home/username/ AlphaLexman Linux - Newbie 9 09-17-2009 06:31 PM
Convert directory structure from long file names in Linux to DOS 8.3 structure? manorina Linux - Software 5 09-12-2009 09:18 AM
Home Jail Folder Structure like Gobolinux Directory Structure luispt Linux - General 3 07-26-2008 06:46 PM
LXer: Retain and recall long paths with rr utility LXer Syndicated Linux News 0 10-16-2007 06:42 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 03:17 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration