Some common file management using simple utilities and regex.

teckk · 02-27-2018, 04:40 PM

Some common file management using simple utilities and regex.
If you are a newbie, here you go.

Remove spaces in filenames

Code:

for file in *; do mv "$file" ${file// /}; done

Replace spaces in filenames with _

Code:

for file in *; do mv "$file" ${file// /_}; done

Remove _ in filenames

Code:

for file in *; do mv "$file" ${file//_/}; done

Remove 1st to 3rd chars in filenames

Code:

for i in *; do mv $i ${i##???}; done

Keep just chars 1-2 5-6 in filenames

Code:

for i in *; do mv $i $(echo $i | cut -c 1-2,5-6); done

Keep just digits in filenames

Code:

for i in *; do mv $i "${i//[!0-9]/}.${i#*.}"; done

Keep just alpha chars in filenames

Code:

for i in *; do mv $i "${i//[!A-Za-z]/}.${i#*.}"; done

Insert _ after 2nd char in filenames

Code:

for i in *; do echo mv "$i" "${i:0:2}_${i:2}"; done

Batch rename filenames from .MP3 to .mp3

Code:

for i in *.MP3; do mv "$i" "${i//MP3/mp3}"; done

Get filenames without extension

Code:

for i in *; do echo ${i%.*}; done

Get just the file extension

Code:

for i in *; do echo ${i#*.}; done

Keep just filename from path

Code:

i="/home/fred/somewhere/myfile.mp3"; echo "${i##*/}"

Keep just filename from url

Code:

i="http://path/to/a/file.mp4"; echo "${i##*/}"

Find 20 largest files/folders from current path

Code:

du -hsx * | sort -rh | head -20

Find 20 largest files in current dir and sub dirs

Code:

find . -type f -print0 | xargs -0 du | sort -n | tail -20 | cut -f2 | xargs -I{} du -sh {}

Find empty files folders

Code:

find . -empty

List all commands present on system by folder.

Code:

ls ${PATH//:/ }

List commands on machine

Code:

apropos --sections 1,8 -w '*' | less

alternate lines from 2 files to make 3rd file

Code:

paste -d "\n" 1st.txt 2nd.txt > 3rd.txt

Stop lines at of text at 65 characters

Code:

fold -sw 65 file1.txt > file2.txt

Find diff of 2 files

Code:

grep -Fxvf file1 file2
diff file1 file2
comm -3 <(sort file1) <(sort file2)

Get file from another machine over ssh

Code:

scp fred@192.168.0.2:/home/fred/file.txt /home/john/

Put file to another machine over ssh

Code:

scp file.txt fred@192.168.0.2:/home/fred/

Find existence and size of remote file over http

Code:

wget --spider url
curl -I url
lynx -head -dump url
HEAD url
w3m -dump_head url

Make a .pdf or .txt from a man page for easier reading

Code:

man -t ls | ps2pdf - output.pdf
man ls | roff2pdf > output.pdf
man -P cat ls > output.txt
man ls | roff2text > output.txt
man ls | col -b > output.txt
man -t ls | ps2ascii - output.txt

And as always, read those man pages.

Anyone else who wishes should add to this list.

pan64 · 02-28-2018, 08:57 AM

you forgot the rename command, which will do all the rename/move commands you posted.

syg00 · 02-28-2018, 04:23 PM

I also prefer "basename" - I have a real problem remembering all the special character combinations for bash string manipulation, although they certainly have their place.

MadeInGermany · 02-28-2018, 04:41 PM

Please edit your post#1 and put each command argument in "quotes"!
Examples

Code:

for file in *; do mv "$file" "${file//_/}"; done
for i in *; do mv "$i" "${i##???}"; done
for i in *; do mv "$i" "$(echo "$i" | cut -c 1-2,5-6)"; done

In the last example there are two nested commands, echo and mv; both must have their arguments quoted.

pan64 · 03-01-2018, 12:20 AM

Quote:

Originally Posted by syg00

I also prefer "basename" - I have a real problem remembering all the special character combinations for bash string manipulation, although they certainly have their place.

You are right. But:
basename is running in a new shell, therefore it is much more expensive. Even a function with a readable name can be faster. What I could suggest is: just add a comment line before that unreadable command to explain what's happening.