LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 02-11-2022, 05:47 PM   #1
babag
Member
 
Registered: Aug 2003
Posts: 419

Rep: Reputation: 31
using tree or du to display directory list with sizes


I'm using the tree command to get a list of directories:

Code:
tree -ifd --du
This, however, seems to be only displaying the size of the directory itself and not the total for its contents.

How would I modify this command to show the total space used by each directory WITH its contents?

I've also found that this du command seems to come close to doing what I want but has a sorting issue:

Code:
du -h ./* | sort -k 2
The above produces a list like this:

Code:
62G     ./Directory
78G     ./Directory 3-14-18
36G     ./Directory 3-14-18/#0986
42G     ./Directory 3-14-18/#0987
38G     ./Directory 3-21-18
38G     ./Directory 3-21-18/#1229
31G     ./Directory/#7852
31G     ./Directory/#7853
Note that, in the above, the first directory, the one with no date as part of its name, is at the top but it's subs are far below. I find that, if I add even a phony date to the end, something like 0-00-00, it sorts correctly:

Code:
62G     ./Directory 0-00-00
31G     ./Directory 0-00-00/#7852
31G     ./Directory 0-00-00/#7853
78G     ./Directory 3-14-18
36G     ./Directory 3-14-18/#0986
42G     ./Directory 3-14-18/#0987
38G     ./Directory 3-21-18
38G     ./Directory 3-21-18/#1229
Additional question here would be:

If I were to use du, how do I get the sorting to work WITHOUT having to add phony dates to everyplace I find this issue?

thanks,
babag
 
Old 02-11-2022, 06:46 PM   #2
dugan
LQ Guru
 
Registered: Nov 2003
Location: Canada
Distribution: distro hopper
Posts: 11,246

Rep: Reputation: 5323Reputation: 5323Reputation: 5323Reputation: 5323Reputation: 5323Reputation: 5323Reputation: 5323Reputation: 5323Reputation: 5323Reputation: 5323Reputation: 5323
Use a third party program, like ncdu.
 
Old 02-11-2022, 07:04 PM   #3
babag
Member
 
Registered: Aug 2003
Posts: 419

Original Poster
Rep: Reputation: 31
thanks, dugan.

hmm. that seems like it would add a layer of complexity i'd like to avoid. i was hoping to just use the above commands to direct out to a txt file and be done. this seems like i'd have to run the ncdu app, then figure out how to sort correctly, then export to txt file. is that right? is there no way to get du or tree, or a combination of the two to do this in a line?

edit:
just looked into ncdu and it doesn't easily seem to do what i'm looking for. what i want is a list of ONLY directories, each followed by its subdirectories. something like this:
Code:
548G    ./Buzz_Video-0_DRIVE
195G    ./Buzz_Video-0_DRIVE/Australia
190G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut
1.5G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Aukland-00
1.6G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Cervantes-00
7.6G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Cervantes-01
3.2G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Cervantes-02
1.2G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Coral_Bay-00
2.6G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Coral_Bay-01
1.8G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Coral_Bay-02
1.1G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Denham-00
4.5G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Denmark-00
2.5G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Esperance-00
3.2G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Esperance-01
1.9G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Esperance-02
1.1G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Esperance-03
745M    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Esperance-04
3.6G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Exmouth-00
762M    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Exmouth-01
1.1G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Kalbarri-00
2.2G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Kalbarri-01
1.6G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Kalbarri-02
151M    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Kalbarri-03
1.4G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Kalbarri-04
223M    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Kalbarri-05
1.1G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Kalbarri-06
2.1G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Kalbarri-07
3.4G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Kalbarri-08
3.7G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Kalbarri-09
1.2G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Marg_Riv-00
860M    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Marg_Riv-01
730M    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Marg_Riv-02
119M    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Melbourne-00
2.0G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Melbourne-01
544M    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Pemberton-00
2.3G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Perth-00
1.4G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Perth-01
126G    ./Buzz_Video-0_DRIVE/Australia/Final_Cut/Timelapse
3.7G    ./Buzz_Video-0_DRIVE/Australia/H4
1.6G    ./Buzz_Video-0_DRIVE/Australia/H4/Batch-00
1.6G    ./Buzz_Video-0_DRIVE/Australia/H4/Batch-01
504M    ./Buzz_Video-0_DRIVE/Australia/H4/Batch-02
1.6G    ./Buzz_Video-0_DRIVE/Australia/Stills
329G    ./Buzz_Video-0_DRIVE/dv_media
1.1M    ./Buzz_Video-0_DRIVE/fcp_projects
1.1M    ./Buzz_Video-0_DRIVE/fcp_projects/Buzzworld_Angels_Gate
20K     ./Buzz_Video-0_DRIVE/fcp_projects/Buzzworld_Angels_Gate/Audio
4.0K    ./Buzz_Video-0_DRIVE/fcp_projects/Buzzworld_Angels_Gate/Audio/FX
4.0K    ./Buzz_Video-0_DRIVE/fcp_projects/Buzzworld_Angels_Gate/Audio/Music
12K     ./Buzz_Video-0_DRIVE/fcp_projects/Buzzworld_Angels_Gate/Backups
4.0K    ./Buzz_Video-0_DRIVE/fcp_projects/Buzzworld_Angels_Gate/Encoded Files
4.0K    ./Buzz_Video-0_DRIVE/fcp_projects/Buzzworld_Angels_Gate/Exports
4.0K    ./Buzz_Video-0_DRIVE/fcp_projects/Buzzworld_Angels_Gate/Media
4.0K    ./Buzz_Video-0_DRIVE/fcp_projects/Buzzworld_Angels_Gate/Notes
4.0K    ./Buzz_Video-0_DRIVE/fcp_projects/Buzzworld_Angels_Gate/Stills
4.0K    ./Buzz_Video-0_DRIVE/fcp_projects/Buzzworld_Angels_Gate/Titles
the above have sorted correctly. as noted previously, though, there are cases in which the sorting seems to fail. how do i get around that?

/end edit

thanks again,
babag

Last edited by babag; 02-11-2022 at 07:24 PM.
 
Old 02-11-2022, 08:47 PM   #4
babag
Member
 
Registered: Aug 2003
Posts: 419

Original Poster
Rep: Reputation: 31
basic sorting question

I've been having a problem with sorting that is beginning to look like more of a system-wide issue than a specific command issue.

I'm trying to print a list of directories and their sub-directories. It's working well until I get to some directories with specific naming conventions.

Using some command line options I can print a list that, generally, is fine. Some directories, however list like this:

Code:
62G     ./Directory
78G     ./Directory 3-14-18
36G     ./Directory 3-14-18/#0986
42G     ./Directory 3-14-18/#0987
38G     ./Directory 3-21-18
38G     ./Directory 3-21-18/#1229
31G     ./Directory/#7852
31G     ./Directory/#7853
Note that, in the above, the first directory, the one with no date as part of its name, is at the top but it's subs are at the bottom. I find that, if I add even a phony date to the end, something like 0-00-00, it sorts correctly:

Code:
62G     ./Directory 0-00-00
31G     ./Directory 0-00-00/#7852
31G     ./Directory 0-00-00/#7853
78G     ./Directory 3-14-18
36G     ./Directory 3-14-18/#0986
42G     ./Directory 3-14-18/#0987
38G     ./Directory 3-21-18
38G     ./Directory 3-21-18/#1229
I've been trying to find options within the commands I've been using (tree and du) to try to address this issue. I've just found, however, that this seems to be a more general problem, hence, my coming to this, more general, forum.

I imported my list into LibreOffice Calc and tried sorting it there. To my surprise, it sorted the same way my command line options had sorted the list, with the non-dated sub-directories separated from the parent by dated directories.

Is there no way around this? (If it makes a difference, I'm using Kubuntu 20.04.)

thanks,
babag

Last edited by babag; 02-11-2022 at 08:53 PM.
 
Old 02-11-2022, 09:47 PM   #5
computersavvy
Senior Member
 
Registered: Aug 2016
Posts: 3,345

Rep: Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484
The system sorts in ascii sequence as do most apps.

Since the # (ascii 35) sign comes after the space " " (ascii 32) it always will sort as you are seeing. You can see the ascii sequence online with a quick search for 'ascii chart'.
If your directory names did not have the space in the name, then the # (ascii 35) would sort before the first number. The numbers are "0" (ascii 48) thru "9" (ascii 57). Even replacing the space with a hyphen (ascii 45) or an underscore (ascii 95) would make it sort the way you expect.
Even the directory separator (/) ascii 47 comes after the space which also puts those subdirectories after the names containing the space.
(Come to think of it, the hyphen would still sort before the "/" so the underscore would be the better choice to replace the spaces.)

I see the same thing when sorting numbers with single digits and 2 or 3 digit numbers. As an example, in most cases the number 2 will be sorted between 19 and 20 but 02 will be sorted properly with other 2 digit numbers.

Last edited by computersavvy; 02-11-2022 at 10:16 PM.
 
Old 02-11-2022, 10:47 PM   #6
babag
Member
 
Registered: Aug 2003
Posts: 419

Original Poster
Rep: Reputation: 31
thanks, computersavvy.

from what you say, it's sounding like the only way i can get around this is by renaming my directories. is that right? there's no switch or something i can apply to my searching and sorting?

edit:
i suppose what's surprising to me is that the du command, which i used to produce the lists above:
Code:
du -h ./* | sort -k2
seems to be constructing the string for each directory or sub-directory listing and then using that full string for its sort, rather than the actual directory structure. if it ordered by structure it would sort properly. since it seems to be looking at the string for each listing, it sees things like the "/" separator and sorts using that, causing the ascii ordering you describe. hmmm.
end edit

thanks again,
babag

Last edited by babag; 02-11-2022 at 11:32 PM.
 
Old 02-12-2022, 01:12 AM   #7
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,987

Rep: Reputation: 7337Reputation: 7337Reputation: 7337Reputation: 7337Reputation: 7337Reputation: 7337Reputation: 7337Reputation: 7337Reputation: 7337Reputation: 7337Reputation: 7337
Quote:
Originally Posted by babag View Post
thanks, computersavvy.

from what you say, it's sounding like the only way i can get around this is by renaming my directories.
Definitely not.

I guess you can specify / as delimiter for sort (see -t). Also you can have different keys (see KEYDEF).
 
Old 02-12-2022, 03:24 AM   #8
aus9
LQ 5k Club
 
Registered: Oct 2003
Location: Western Australia
Distribution: Icewm
Posts: 5,842

Rep: Reputation: Disabled
If you have only a few dirs....can I suggest for each dir run

Code:
ls -Slh
 
Old 02-12-2022, 05:27 AM   #9
GazL
LQ Veteran
 
Registered: May 2008
Posts: 6,915

Rep: Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033
'du -S .| sort -k 2' will give you a list of directories and their contents size, but it won't include in the totals the contents of any subdirectories: only the size of files in that directory itself.

Not sure if that's what your looking for or not.
 
Old 02-12-2022, 05:44 AM   #10
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,146

Rep: Reputation: 4124Reputation: 4124Reputation: 4124Reputation: 4124Reputation: 4124Reputation: 4124Reputation: 4124Reputation: 4124Reputation: 4124Reputation: 4124Reputation: 4124
Seems like a duplicate of this. Consolidation would seem appropriate.
 
Old 02-12-2022, 06:16 AM   #11
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 19,872
Blog Entries: 12

Rep: Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053
Duplicate of https://www.linuxquestions.org/quest...7/#post6328503
 
Old 02-12-2022, 08:32 AM   #12
goumba
Senior Member
 
Registered: Dec 2009
Location: New Jersey, USA
Distribution: Fedora, OpenSUSE, FreeBSD, OpenBSD, macOS (hack). Past: Debian, Arch, RedHat (pre-RHEL).
Posts: 1,335
Blog Entries: 7

Rep: Reputation: 402Reputation: 402Reputation: 402Reputation: 402Reputation: 402
This looks like it does what you want to do:

Code:
find * -type d -print0 | xargs -0rn1 du
find sorts its output by directory
using '*' for spec to find means you won't get the current directory and leading './' in its output.
-print0 (find) and -0 (xargs) allow spaces and other special characters in filenames to be handled gracefully.

xargs -n 1 runs du on each directory individually -r just ignores the parameter if it's empty

Last edited by goumba; 02-12-2022 at 08:33 AM.
 
Old 02-12-2022, 02:23 PM   #13
babag
Member
 
Registered: Aug 2003
Posts: 419

Original Poster
Rep: Reputation: 31
thanks everyone.

@pan64:
interesting. i don't quite follow how to use that to get where i want but it seems to have potential.

@ondoho:
yes. i started out looking at this as more specific to certain commands but found the issue to be more general. i'm mainly working on it here now.

@goumba:
also interesting. i tried this command but it produced a lot of duplicates. sorting of longer entries was out of order too. nice to get rid of the ./ though!

i'm thinking that my original command along with @pan64's -t delimiter option might be the ticket. hope to find examples.

thanks again,
babag
 
Old 02-12-2022, 02:30 PM   #14
babag
Member
 
Registered: Aug 2003
Posts: 419

Original Poster
Rep: Reputation: 31
@aus9:
unfortunately, i have a LOT of directories.

@GazL:
tried this command and, unfortunately, it has the same sorting issue as my original command.

@syg00:
yes. i'm continuing with the thread you linked as i found the sorting issue to be more general than these two specific commands.

thanks everyone!
babag
 
Old 02-12-2022, 03:09 PM   #15
GazL
LQ Veteran
 
Registered: May 2008
Posts: 6,915

Rep: Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033
Quote:
Originally Posted by babag View Post
@GazL:
tried this command and, unfortunately, it has the same sorting issue as my original command.
That's an easy fix, just append a trailing / to the dirnames.
Code:
du -S . |sed 's:$:/:' | sort -k 2
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Get directory sizes, sorted by size and directory depth Bashed Linux - Software 1 11-27-2018 02:58 PM
what is the difference strict binary tree nad extended binary tree . tushar_pandey Programming 1 07-18-2012 11:30 AM
define different physical sizes on Multiple Monitors of different sizes MasterC Linux - Desktop 2 03-18-2008 04:24 PM
the bible = the tree of the knowledge of good and evil () Jesus = the tree of life Michael111 General 2 04-14-2004 04:28 PM
need a P-Tree (Patricia Tree) library manaskb Programming 1 11-02-2002 06:15 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 09:35 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration