[SOLVED] List files by size (easy-ish)

GPGAgent · 02-26-2024, 10:57 AM

I had a nice little script, sadly I deleted the notes I took and I cannot recreate it.

I want to loop through a folder of files, order by size - biggest first, then pass that to a bash script to process it.

Quote:

ls -lS | awk '{print $9}'

But some files have spaces and it only prints the first word of the filename :-(

teckk · 02-26-2024, 11:17 AM

Example:

Code:

files=$(du ~ | sort -nr)

echo "$files" | less

pan64 · 02-26-2024, 11:18 AM

ls is known to produce unparsable output, which is difficult to use in scripts.

Code:

ls -lS | awk ' { print substr($0, 48) } '

scasey · 02-26-2024, 11:29 AM

Code:

ls -1S

Works…yes? (That’s the number one. 1)
I don’t have any file names with imbedded spaces, but I’d expect the entire file name to be returned.
See the -Q option to ls to perhaps help with the input to your script.
My preference would be to “undo” the spaces in the file names, but that’s just me.

GPGAgent · 02-26-2024, 11:32 AM

Quote:

Originally Posted by pan64

ls is known to produce unparsable output, which is difficult to use in scripts.

Code:

ls -lS | awk ' { print substr($0, 48) } '

Almost, but I only want the file name

Quote:

$ ls -lS | awk ' { print substr($0, 48) } '

5 D01.iso
5 D01.txt
5 DVDdd.txt
3 00Started-dd.txt
3 This has spaces

GPGAgent · 02-26-2024, 11:34 AM

Quote:

Originally Posted by scasey

Code:

ls -1Sr

Works…yes? (That’s the number one. 1)

That's just fine

Quote:

$ ls -1S
D01.iso
D01.txt
DVDdd.txt
00Started-dd.txt
'This has spaces'

I removed the r because I want biggest first, I thought I tried this and it didn't do what I wanted, becuase i left the l in for long listing - doh!!!

Now how do I process this list?

GPGAgent · 02-26-2024, 11:35 AM

Quote:

Originally Posted by teckk

Example:

Code:

files=$(du ~ | sort -nr)

echo "$files" | less

Sorry teckk, not exactly what i wanted, I'm only looking at the current directory.

pan64 · 02-26-2024, 11:38 AM

Quote:

Originally Posted by GPGAgent

Almost, but I only want the file name

You need to adjust that 48 in your awk script to fit your needs (50), but anyway, the solution posted by scasey is much better and more elegant.

Code:

ls -1S
    ^  number one

michaelk · 02-26-2024, 11:56 AM

By default awk will split on spaces which is why you only see the first word in the filename. Parsing the output of ls is never a good idea.

Code:

find /dir/ -type f -printf '%p\n' | sort -nr

GPGAgent · 02-26-2024, 12:01 PM

Quote:

Originally Posted by pan64

You need to adjust that 48 in your awk script to fit your needs (50), but anyway, the solution posted by scasey is much better and more elegant.

Code:

ls -1S
    ^  number one

Yep, 48 to 50 works a treat

teckk · 02-26-2024, 12:07 PM

Quote:

Now how do I process this list?

Code:

files=($(ls -1S))

for i in "${files[@]}"; do
    echo "Doing something with "$i""
    sleep .2
done

GPGAgent · 02-26-2024, 12:30 PM

Quote:

Originally Posted by teckk

Code:

files=($(ls -1S))

for i in "${files[@]}"; do
    echo "Doing something with "$i""
    sleep .2
done

Of course, cheers

MadeInGermany · 02-26-2024, 12:56 PM

$(ls -1S) splits on whitespace, can go wrong.
Better is a while-read loop, where each read ends on newline.

Code:

ls -S |
  while IFS= read -r fn
  do
    echo "do sth with $fn"
  done

Standard shell and bash (unlike ksh and zsh) run the loop in a sub shell.
The following (if supported) runs the loop in the main shell:

Code:

while IFS= read -r fn <&3
do
  echo "do sth with $fn"
done 3< <(ls -S)

It's robust enough in practice. (Still not perfect, because a file name may contain a newline character.)

GPGAgent · 02-26-2024, 01:10 PM

Quote:

Originally Posted by MadeInGermany

$(ls -1S) splits on whitespace, can go wrong.
Better is a while-read loop, where each read ends on newline.

Code:

ls -S |
  while IFS= read -r fn
  do
    echo "do sth with $fn"
  done

Standard shell and bash (unlike ksh and zsh) run the loop in a sub shell.
The following (if supported) runs the loop in the main shell:

Code:

while IFS= read -r fn <&3
do
  echo "do sth with $fn"
done 3< <(ls -S)

It's robust enough in practice. (Still not perfect, because a file name may contain a newline character.)

Cheers Perfect, all in my notes now

Quote:

$ ls -S | while IFS= read -r fn; do echo "do sth with $fn"; done
do sth with D01.iso
do sth with D01.txt
do sth with DVDdd.txt
do sth with 00Started-dd.txt
do sth with This has spaces

MadeInGermany · 02-26-2024, 01:33 PM

Last method, a for loop where InputFieldSeparator is set to newline only:

Code:

  saveIFS=$IFS
  IFS="
"
  for fn in $(ls -S)
  do
    echo "do sth with $fn"
  done
  IFS=$saveIFS

The same method works for the array assignment

Code:

  saveIFS=$IFS
  IFS="
"
  files=($(ls -S))
  IFS=$saveIFS
  for fn in "${files[@]}"
  do
    echo "do sth with $fn"
  done