find all extensions than make directories of extensions using find command?

finalturismo · 09-18-2020, 02:29 AM

so iam trying to use find to locate all extensions recursively from all files recovered from a drive.

After i find all extensions i need to make folders for these extensions....

So far what i got is

Code:

find . -iname "*" -exec mkdir {} \;

Problem is i only need to make folders with the same name as the extensions. I some how need to subtract all text before the last period and push the results back into mkdir....

been messing with this for hours... i have found a few bash scripts that do the job, but i would rather get bits of code and put it together my self to help me understand bash scripting a little more.

The operators are what confuse me in linux....

pan64 · 09-18-2020, 02:50 AM

Code:

A="asdfarfe.dfg"
echo ${A##*.}

will give you the extension, but it cannot be put into that find command. You need to write a small script and/or loop to make it work

syg00 · 09-18-2020, 02:52 AM

Quote:

Originally Posted by finalturismo

i have found a few bash scripts that do the job, but i would rather get bits of code and put it together my self to help me understand bash scripting a little more.

I don't understand why you think you can't learn by studying code that works. When you understand why it works, you should be most of the way along that particular road.

finalturismo · 09-18-2020, 02:56 AM

Quote:

Originally Posted by pan64

Code:

A="asdfarfe.dfg"
echo ${A##*.}

will give you the extension, but it cannot be put into that find command. You need to write a small script and/or loop to make it work

Can you break down what ## means in the brackets?

Also if you can give me an example of the loop that would be great XD

i also found this

Code:

filename=foo.txt
echo "${filename}"

but it subtracts the extension... it seems simple enough but i need to reverse it...

Could i put the output in a variable and than run it again?

Turbocapitalist · 09-18-2020, 03:12 AM

Quote:

Originally Posted by finalturismo

Can you break down what ## means in the brackets?

It is called parameter expansion:

BudiKusasi · 09-19-2020, 07:00 AM

Code:

find . -execdir bash -c 'for p;{ [[ ${p#./} =~ ^.*\.(.+)$ ]]; mkdir ${BASH_REMATCH[1]}; }' dm '{}' \+ 2>/dev/null

get -exec specific feature explanation
$ man find
type / then -exec to get it, 'n' to point thru next

dm is any $0 bash argument, so here just named dm as a dummy

if just to create dirs in the starting search dir.

Code:

$ find ~+ -exec bash -c 'for p;{ [[ $p =~ ^.*\.(.+)$ ]]; mkdir ${BASH_REMATCH[1]}; }' dm '{}' \+ 2>/dev/null

Code:

2>/dev/null

just addition, may be left

berndbausch · 09-19-2020, 07:25 PM

Quote:

Originally Posted by finalturismo

Code:

filename=foo.txt
echo "${filename}"

but it subtracts the extension... it seems simple enough but i need to reverse it...

This doesn't subtract anything. It just echoes foo.txt.

Use pan64's ## construct. It removes the longest string that matches *. from the beginning of the word, i.e. foo. including the dot.

You can use find to generate all files, read the file list and use the ## syntax to generate a list of extensions, use sort -u to remove redundancy, then mkdir each list element.

Untested code:

Code:

find . |
while read file
do echo ${file##*.}
done |
sort -u |
xargs mkdir

EDIT: Or, now that I cleared my mind, replace the while loop with sed

Code:

sed 's/^.*\.//'

finalturismo · 09-19-2020, 07:59 PM

What about using variables?

I think i almost got it with variables....

anything iam doing wrong here?? the echo works with filename variable...

But if i try to call the $filename variable i get a permission denied?

What am i doing wrong here?

Code:

extension=$(echo "${filename##*.}")
makedirs=(mkdir $filename)
filename=$(find . -iname "*.testing")

berndbausch · 09-19-2020, 08:08 PM

Quote:

Originally Posted by finalturismo

But if i try to call the $filename variable i get a permission denied?

I suppose you get Permission Denied because you have no permissions to make this directory.

Quote:

What am i doing wrong here?

Code:

extension=$(echo "${filename##*.}")
makedirs=(mkdir $filename)
filename=$(find . -iname "*.testing")

The first line is overly complicated. It is equivalent to extension="${filename##*.}". You don't need the echo and the $(...) construct (named "command substitution").

Second line: It's not quite clear what you want to achieve. Normally, the parentheses have the effect of running the text between them in a subshell, but I am not sure how the shell i terprets your line. You also want to assign something to makedirs, but what? If you actually mean makedirs=$(mkdir $filename), makedirs will contain the empty string, since mkdir doesn't output anything.

Third line is correct. You assign the list of files named blabla.testing to the variable filename. Now, the question is how you want to use this variable.

finalturismo · 09-19-2020, 10:02 PM

Quote:

Originally Posted by berndbausch

I suppose you get Permission Denied because you have no permissions to make this directory.

The first line is overly complicated. It is equivalent to extension="${filename##*.}". You don't need the echo and the $(...) construct (named "command substitution").

Second line: It's not quite clear what you want to achieve. Normally, the parentheses have the effect of running the text between them in a subshell, but I am not sure how the shell i terprets your line. You also want to assign something to makedirs, but what? If you actually mean makedirs=$(mkdir $filename), makedirs will contain the empty string, since mkdir doesn't output anything.

Third line is correct. You assign the list of files named blabla.testing to the variable filename. Now, the question is how you want to use this variable.

Code:

filename=$(find . -iname "*")
extension="${filename##*.}"
makedirs=(mkdir $extension)

I rearranged the order for it to make more sense.

So this is for a recovery i did from a raid setup for a client. Iam a bit of a bash n00b to be honest.....
He has an incredible amount of files and its near impossible for me to sort through it manually. So iam setting variables in order to get the job done. This is about as far as i know bash, but iam learning slowly. Here are the goals iam trying to achieve in steps with this small script.

1. Find all files recursively
2. Remove all text but extension from files.
3. Make directories based on all found extensions.
4. Move files by extension to the created folders.

This allows me to present the recover data in a way that makes sense to the client.

berndbausch · 09-19-2020, 10:23 PM

Quote:

Originally Posted by finalturismo

Code:

filename=$(find . -iname "*")
extension="${filename##*.}"
makedirs=(mkdir $extension)

After the first line, the filename variable contains a list of all files found.

The second line removes EVERYTHING from that list, up to the last period. This is not your intention.

It seems to me that in the third line, you want to create directories, however I have no idea why you put parentheses around the mkdir command and what you want to assign to the makedirs variable. In any case, at this point extensions probably contains the very last extension from the find command, and not all extensions.

Quote:

1. Find all files recursively
2. Remove all text but extension from files.
3. Make directories based on all found extensions.
4. Move files by extension to the created folders.

The script I suggested should take care of the first three points. To incorporate point 4, I would write a script that

checks the extension of the file
creates a corresponding directory if it doesn't exist
moves the file there

then uses that script as a find -exec parameter.

The following script assumes that all files have an extension. If that is not the case, more testing needs to be added. Again I did not try it out:

Code:

#!/bin/bash
filename=$1
extension=${filename##*.}     # obtain extension
if [ ! -d $extension ]        # check if directory exists
then mkdir $extension
fi
mv $filename $extension

then

Code:

find . -name '*' -exec name-of-the-script-above {} \+

syg00 · 09-19-2020, 11:05 PM

Be aware that all the suggestions above appear to take no cognisance of hidden files. May not matter, but you might wind up with directories that don't seem like what a user might consider an extension - bash_history as a trivial example.

finalturismo · 09-19-2020, 11:54 PM

thats just what i was looking for , nice and simple but is there anyway i can put the bottom command inside of the bash file too? and why are we checking if a directory exists? sorry for all the questions i understand how to write most of the command but what does the ! mean

berndbausch · 09-20-2020, 05:27 AM

Quote:

Originally Posted by syg00

Be aware that all the suggestions above appear to take no cognisance of hidden files. May not matter, but you might wind up with directories that don't seem like what a user might consider an extension - bash_history as a trivial example.

Well, the find command explicitly excludes files that start with a dot. This may or may not be the intention; impossible to say.

berndbausch · 09-20-2020, 05:31 AM

Quote:

Originally Posted by finalturismo

thats just what i was looking for , nice and simple but is there anyway i can put the bottom command inside of the bash file too?

You could, but only with a certain amount of contorsion.

Quote:

and why are we checking if a directory exists?

To avoid making a directory that exists already.

Quote:

what does the ! mean

The ! in the first line is part of the syntax used to define the interpreter of this script.

The ! in the if condition negates the directory test, i.e. it means "not".