find string in filename and use string to create directories
 

Hi. I'm very new to this and want to do something I know is simple but can't figure it out so well. Please let me know if this is the right forum for this basic type of question.

I have a directory full of files that I want to put into subdirectories according to a string in the filename. Basically, I want to find the string, use mkdir, then mv files with that string in the filename to the newly created directories.

My files are named like this:

date_ID_moreinfo.type

I want the directories to be named ID. I can find the IDs based on the fact that it is two letters then two numbers (AB12, XZ97, etc) and this pattern does not appear anywhere else in the filenames. Also, there will be multiple files matching one particular ID with different dates, extensions, etc. if that will make a difference.

I have tried to use various methods like find and ls | grep to pull the ID information out of the filename and then use it as a variable but I can't seem to get it to work. I know that I can find the strings with regular expressions or probably something easier, but I can't seem to end up with just a list of the IDs.

Thanks for any suggestions.

This tells it to do the same commands for each file (a "for loop"). It uses awk and field separator underscore "_" to brake up the file name. It prints the second field which will be "ID".

Note the character before echo and at end of line is a back tick not a single quote. The character before and after the curly brackets IS a single quote.

It then runs an if/then conditional to determine if the "ID" directory exists. If it does it simply moves the file into that directory. If it doesn't it first makes the directory then moves the file.

FYI: The above should work but I didn't test it - make sure you do a basic test in another location before trying it on your real files.

or slightly differently:

Forrest

p.s. I didn't test this either.

Thanks!

the awk -F_ is exactly what I needed.

Unfortunately, neither of those seem to work perfectly. The first one is making files named ID and the second one puts *all* files into the the first of the directories (but does create the rest of the directories correctly).

Another thing I can't wrap my head around is why when I put these lines into a bash script do they behave differently (or give different errors at least...) than if I cut and paste them to the terminal.

Thanks again.

Sorry - there was typo in my mkdir command. It should be "mkdir" not "mmkdir". I've edited original post.

mkdir creates the directory if it isn't there. Since there is no mmkdir command that step was failing and the subsequent mv was simply renaming the file to the intended directory name. Using the mkdir command should solve that.

OK, it looks like it doesn't like the way I referenced the variable in the mv command. Try:

Normally I call my variables like this anyway. Don't know why I didn't here (probably because I was copying code).

Forrest

I caught that, but only when I was putting it into a bash script, my copy/paste to the terminal never did! It works now, thanks so much! (they both do!)

These really help me get a grasp on what is going on, and I will learn more awk.

Thanks again!

OK, now I'm trying to put files into subdirectories based on their extension, I have .hdr and .txt files for instance.

Why doesn't this work?

If I use this instead of find, it works, but gives me errors if those types do not exist which is fine but obviously wrong.I won't need to create a subdirectory for all filetypes, I just want to figure out how to put certain files in certain subdirectories.
Thanks!

OK, the reason the find isn't working is that you have to escape the *'s. But, even if you did escape them, you would be getting every file in the current directory AND subdirectories that have the name and then moving them to the specified directory. At the end of it, all the files would be located in the directory matching the last DIRNAME.

The mv's should work (you don't need the ';'). What are the errors they're giving?

Forrest

p.s. The mv commands should look like:

mv *_${DIRNAME}_*txt ${DIRNAME}/txt

Oh, I see the error of my ways..

You must make the DIRNAME list unique. Try this instead:

Forrest

What seems to be causing problems is I don't necessarily have to have .txt files for each one, I'm more trying to sort the files as a learning experience before I do the real task...

I was only getting the 'no such file or directory' error when it tried to move a .txt file that does not exist. Makes sense. I guess I can live with errors like that, but it's always nice to not have any!

I guess the find method would/could work with the -maxdepth switch though, right?

Eventually I will have a /hdr and other folders, and maybe a /misc catchall folder or something along those lines. I could then add a final mv command to find the "rest" and move them into the /misc or whatever. I guess it's a waste to have empty directories though (or is it?), but that can be cleaned up easily.

The sort -u does help for sure, as there will definitely be multiple files for each DIRNAME.

Thanks so much for all your help,

Well, if you don't want to have empty directories, you just need another for loop:

Forrest