LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 05-01-2009, 11:29 AM   #1
daberkow
LQ Newbie
 
Registered: May 2009
Posts: 11

Rep: Reputation: 0
find string in filename and use string to create directories


Hi. I'm very new to this and want to do something I know is simple but can't figure it out so well. Please let me know if this is the right forum for this basic type of question.

I have a directory full of files that I want to put into subdirectories according to a string in the filename. Basically, I want to find the string, use mkdir, then mv files with that string in the filename to the newly created directories.

My files are named like this:

date_ID_moreinfo.type

I want the directories to be named ID. I can find the IDs based on the fact that it is two letters then two numbers (AB12, XZ97, etc) and this pattern does not appear anywhere else in the filenames. Also, there will be multiple files matching one particular ID with different dates, extensions, etc. if that will make a difference.

I have tried to use various methods like find and ls | grep to pull the ID information out of the filename and then use it as a variable but I can't seem to get it to work. I know that I can find the strings with regular expressions or probably something easier, but I can't seem to end up with just a list of the IDs.

Thanks for any suggestions.

Last edited by daberkow; 05-01-2009 at 11:32 AM.
 
Old 05-01-2009, 11:39 AM   #2
MensaWater
Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 6,009
Blog Entries: 5

Rep: Reputation: 787Reputation: 787Reputation: 787Reputation: 787Reputation: 787Reputation: 787Reputation: 787
Code:
for FILE in *
do DIRNAME=`echo $FILE |awk -F_ '{print $2}'`
   if [ -d $DIRNAME ]
   then mv $FILE $DIRNAME
   else mkdir $DIRNAME
        mv $FILE $DIRNAME
   fi
done
This tells it to do the same commands for each file (a "for loop"). It uses awk and field separator underscore "_" to brake up the file name. It prints the second field which will be "ID".

Note the character before echo and at end of line is a back tick not a single quote. The character before and after the curly brackets IS a single quote.

It then runs an if/then conditional to determine if the "ID" directory exists. If it does it simply moves the file into that directory. If it doesn't it first makes the directory then moves the file.

FYI: The above should work but I didn't test it - make sure you do a basic test in another location before trying it on your real files.

Last edited by MensaWater; 05-01-2009 at 12:24 PM.
 
Old 05-01-2009, 11:54 AM   #3
forrestt
Senior Member
 
Registered: Mar 2004
Location: Cary, NC, USA
Distribution: Fedora, Kubuntu, RedHat, CentOS, SuSe
Posts: 1,288

Rep: Reputation: 99
or slightly differently:

Code:
for DIRNAME in `ls | awk -F_ '{print $2}'` ; do
   if [ ! -d $DIRNAME ] ; then
      mkdir $DIRNAME
   fi
   mv *_$DIRNAME_* $DIRNAME
done
Forrest

p.s. I didn't test this either.

Last edited by forrestt; 05-01-2009 at 11:55 AM. Reason: added p.s.
 
Old 05-01-2009, 12:17 PM   #4
daberkow
LQ Newbie
 
Registered: May 2009
Posts: 11

Original Poster
Rep: Reputation: 0
Thanks!

the awk -F_ is exactly what I needed.

Unfortunately, neither of those seem to work perfectly. The first one is making files named ID and the second one puts *all* files into the the first of the directories (but does create the rest of the directories correctly).

Another thing I can't wrap my head around is why when I put these lines into a bash script do they behave differently (or give different errors at least...) than if I cut and paste them to the terminal.

Thanks again.
 
Old 05-01-2009, 12:23 PM   #5
MensaWater
Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 6,009
Blog Entries: 5

Rep: Reputation: 787Reputation: 787Reputation: 787Reputation: 787Reputation: 787Reputation: 787Reputation: 787
Sorry - there was typo in my mkdir command. It should be "mkdir" not "mmkdir". I've edited original post.

mkdir creates the directory if it isn't there. Since there is no mmkdir command that step was failing and the subsequent mv was simply renaming the file to the intended directory name. Using the mkdir command should solve that.

Last edited by MensaWater; 05-01-2009 at 12:25 PM.
 
Old 05-01-2009, 12:32 PM   #6
forrestt
Senior Member
 
Registered: Mar 2004
Location: Cary, NC, USA
Distribution: Fedora, Kubuntu, RedHat, CentOS, SuSe
Posts: 1,288

Rep: Reputation: 99
OK, it looks like it doesn't like the way I referenced the variable in the mv command. Try:

Code:
for DIRNAME in `ls | awk -F_ '{print $2}'` ; do
   if [ ! -d ${DIRNAME} ] ; then
      mkdir ${DIRNAME}
   fi
   mv *_${DIRNAME}_* ${DIRNAME}
done
Normally I call my variables like this anyway. Don't know why I didn't here (probably because I was copying code).

Forrest
 
Old 05-01-2009, 12:33 PM   #7
daberkow
LQ Newbie
 
Registered: May 2009
Posts: 11

Original Poster
Rep: Reputation: 0
I caught that, but only when I was putting it into a bash script, my copy/paste to the terminal never did! It works now, thanks so much! (they both do!)

These really help me get a grasp on what is going on, and I will learn more awk.

Thanks again!

Last edited by daberkow; 05-01-2009 at 12:40 PM.
 
Old 05-01-2009, 01:29 PM   #8
daberkow
LQ Newbie
 
Registered: May 2009
Posts: 11

Original Poster
Rep: Reputation: 0
OK, now I'm trying to put files into subdirectories based on their extension, I have .hdr and .txt files for instance.

Why doesn't this work?

Code:
#!/bin/bash


for DIRNAME in `ls | awk -F_ '{print $2}'` ; do
   if [ ! -d ${DIRNAME} ] ; then
      mkdir ${DIRNAME}
      mkdir ${DIRNAME}/hdr
      mkdir ${DIRNAME}/txt
	
   fi

   find . -name *_${DIRNAME}_*hdr -exec mv {} ${DIRNAME}/hdr \;
   find . -name *_${DIRNAME}_*txt -exec mv {} ${DIRNAME}/txt \;
 
done
If I use this instead of find, it works, but gives me errors if those types do not exist which is fine but obviously wrong.
Code:
mv *${DIRNAME}*hdr ${DIRNAME}/hdr; 
mv *${DIRNAME}*txt ${DIRNAME}/txt;
I won't need to create a subdirectory for all filetypes, I just want to figure out how to put certain files in certain subdirectories.
Thanks!

Last edited by daberkow; 05-01-2009 at 01:35 PM.
 
Old 05-01-2009, 01:35 PM   #9
forrestt
Senior Member
 
Registered: Mar 2004
Location: Cary, NC, USA
Distribution: Fedora, Kubuntu, RedHat, CentOS, SuSe
Posts: 1,288

Rep: Reputation: 99
OK, the reason the find isn't working is that you have to escape the *'s. But, even if you did escape them, you would be getting every file in the current directory AND subdirectories that have the name and then moving them to the specified directory. At the end of it, all the files would be located in the directory matching the last DIRNAME.

The mv's should work (you don't need the ';'). What are the errors they're giving?

Forrest

p.s. The mv commands should look like:

mv *_${DIRNAME}_*txt ${DIRNAME}/txt

Last edited by forrestt; 05-01-2009 at 01:36 PM. Reason: added ps
 
Old 05-01-2009, 01:39 PM   #10
forrestt
Senior Member
 
Registered: Mar 2004
Location: Cary, NC, USA
Distribution: Fedora, Kubuntu, RedHat, CentOS, SuSe
Posts: 1,288

Rep: Reputation: 99
Oh, I see the error of my ways..

You must make the DIRNAME list unique. Try this instead:

Code:
#!/bin/bash

for DIRNAME in `ls | awk -F_ '{print $2}' | sort -u` ; do
   if [ ! -d ${DIRNAME} ] ; then
      mkdir ${DIRNAME}
      mkdir ${DIRNAME}/hdr
      mkdir ${DIRNAME}/txt
   fi

   mv *_${DIRNAME}_*txt ${DIRNAME}/txt
   mv *_${DIRNAME}_*hdr ${DIRNAME}/hdr
 
done
Forrest

Last edited by forrestt; 05-01-2009 at 01:40 PM.
 
Old 05-01-2009, 01:51 PM   #11
daberkow
LQ Newbie
 
Registered: May 2009
Posts: 11

Original Poster
Rep: Reputation: 0
What seems to be causing problems is I don't necessarily have to have .txt files for each one, I'm more trying to sort the files as a learning experience before I do the real task...

I was only getting the 'no such file or directory' error when it tried to move a .txt file that does not exist. Makes sense. I guess I can live with errors like that, but it's always nice to not have any!

I guess the find method would/could work with the -maxdepth switch though, right?

Eventually I will have a /hdr and other folders, and maybe a /misc catchall folder or something along those lines. I could then add a final mv command to find the "rest" and move them into the /misc or whatever. I guess it's a waste to have empty directories though (or is it?), but that can be cleaned up easily.

The sort -u does help for sure, as there will definitely be multiple files for each DIRNAME.

Thanks so much for all your help,

Last edited by daberkow; 05-01-2009 at 01:55 PM.
 
Old 05-01-2009, 02:12 PM   #12
forrestt
Senior Member
 
Registered: Mar 2004
Location: Cary, NC, USA
Distribution: Fedora, Kubuntu, RedHat, CentOS, SuSe
Posts: 1,288

Rep: Reputation: 99
Well, if you don't want to have empty directories, you just need another for loop:

Code:
#!/bin/bash

for DIRNAME in `ls | awk -F_ '{print $2}' | sort -u` ; do
    if [ ! -d ${DIRNAME} ] ; then
        mkdir ${DIRNAME}
    fi
    for EXTENSION in `ls *_${DIRNAME}_* | awk -F. '{print $NF}' | sort -u` ; do
        if [ ! -d ${DIRNAME}/${EXTENSION} ] ; then
            mkdir ${DIRNAME}/${EXTENSION}
        fi
        mv *_${DIRNAME}_*.${EXTENSION} ${DIRNAME}/${EXTENSION}
    done
done
Forrest
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Stripping a string to create a new string Sherlock Linux - Newbie 6 08-13-2008 02:49 AM
Trying to change String using sed with a string \/home\/user\/Desktop icecoolcorey Programming 10 06-12-2008 11:32 PM
Using /bin/mail command with a string for message, not a filename? SirTristan Linux - Newbie 4 03-26-2008 08:22 PM
Shell Script: Delete lines til string found or until particular string. bhargav_crd Linux - General 3 12-20-2007 11:14 PM
Filename string manipulation jango77 Linux - Newbie 2 03-27-2005 06:58 PM


All times are GMT -5. The time now is 04:24 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration