LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 10-10-2006, 09:37 AM   #16
muha
Member
 
Registered: Nov 2005
Distribution: xubuntu, grml
Posts: 451

Rep: Reputation: 37

I wrote this script underneath which works better then the first one.
It's a littlebit more robust and gives warnings if files already exist with similar names in the outputdir.
TEST IT on directories which are safe to destroy! I tried some things which I think might go wrong and it works over here but you never know what's happening at your side.
So make a copy of some directories and start playing with those first.
One thing I tried to build in is to prevent overwriting of
/task/faces/con0027.img
with
/stats/snod/con0027.img
to the same outputdir: outputdir/con_0027.img
This script should prevent from that situation.

You can now specify the source directories like /task/faces/ or /stats/snod/
If you need to know more about how I did things, feel free to ask.
Let me know how it works for you, m'kay?


Code:
#       -------------------------------------------------------------------
#
#       Shell program to copy all *.img and *.hdr files from subdirs and
#       rename them to ./<outputdir>/<dirnumber>_con<number>.extension
#
#       Copyright 2006, <scriptfreak at gmail.com>
#
#       This program is free software; you can redistribute it and/or
#       modify it under the terms of the GNU General Public License as
#       published by the Free Software Foundation; either version 2 of the
#       License, or (at your option) any later version.
#
#       This program is distributed in the hope that it will be useful, but
#       WITHOUT ANY WARRANTY; without even the implied warranty of
#       MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
#       General Public License for more details.


#       -------------------------------------------------------------------
#       Constants
#       -------------------------------------------------------------------

        PROGNAME=$(basename $0)
        VERSION="0.0.2"

#       -------------------------------------------------------------------
#       Variables
#       -------------------------------------------------------------------

        # set some variables
        # make sure the source_dir is not specified yet
        source_dir=
        # the name for the output directory (set it to what you want)
        output_dir="outputdir"
        # overwrite file in outputdir when it is already present when copying
        copy_overwrite="no"

#       -------------------------------------------------------------------
#       Functions
#       -------------------------------------------------------------------

function test_if_file_does_not_exist
{

#       -----------------------------------------------------------------------------------------------
#       Function to test if a file exists
#               argument is $filename which is the inputfile
#               returns 0 (succes) if the file does not exist
#               usage:
#                       test_if_file_exists <inputfile> || error_exit "<Filename> does not exist! Exiting"
#                       OR
#                       if test_if_file_exists <inputfile>;then echo "Succes!"; else echo "Failure"; fi
#       -----------------------------------------------------------------------------------------------

        if [ "$1" != "" ]; then
                if [ -e $1 ]; then
                        return 1
                else
                        # filename is not empty and filename does not exist
                        return 0
                fi
        else
                return 1
        fi
}


function find_files
{

#       -------------------------------------------------------------------
#       Function to find files called *.img and *.hdr from ./<number>/$source_dir/
#               argument is $source_dir which is the subdirectory to read from
#               So now the find command might look like find ./4001/task/faces/ -name etc ...
#               Exists with an error message if the specified subdirs do not exist.
#       -------------------------------------------------------------------

        # First find all files in subdirs called *.hdr and *.img
        # | sed -n "/^\.\/[0-9]*\/[a-zA-Z]/p" excludes all directories that do start with ./<letter> since we expect these
        # to be output directories and we don't want to copy from them. It also filters out subdirectories that start with numbers.
        find ./*/$1 -name "*.img" -o -name "*.hdr"| sed -n "/^\.\/[0-9]*\/[a-zA-Z]/p" || error_exit "Cannot find files in the subdirectory: $1"

}


function find_and_copy
{

#       -------------------------------------------------------------------
#       Function to find and copy from specified dir
#               argument is $source_dir which is the subdirectory (of ./<number>) to read from
#       -------------------------------------------------------------------

        # The find and copy command. A for loop for each file present.
        # Variable $i is now in the form of ./4001/task/faces/con_0027.img
        for i in `find_files $1`
        do
        # Specify the output file
        # ${i%%/[a-zA-Z]*} leaves only the ./4001 part of ./4001/task/faces/con0027.img
        # In other words, it deletes anything coming after ./4001 which has the form: slash letters; so /task/faces/con0027.img
        #
        # sed 's#^\.#\.\/'"${output_dir}"'#g'
        # This sed converts the output of the echo: ./4001
        # First it looks for 'the begin of the sentence' followed immediately by a .
        # it then replaces . by ./outputdir and echos out the rest, which is /4001
        # So in total this first echo outputs: ./outputdir/4001
        #
        # followed by a replace of ./4001/task/faces/con0027.img into _con0027.img
        # What sed does exactly: replace anything (.*) (here: ./4001/task/faces/) followed by con and the trailing bit,
        # with _con followed by the trailing bit (0027.img)
        outfile="`echo -n ${i%%/[a-zA-Z]*}|sed 's#^\.#\.\/'"${output_dir}"'#g'; echo -n ${i}|sed 's#.*con#_con#g'`"

        # test if file exists in outputdir. If it exists program continues with a warning message.
        test_if_file_does_not_exist "$outfile" || echo "$outfile already exists! Not overwritten by $i" >&2

        # The copy bit; -i --reply=no is not to overwrite files in $output_dir
        # -n to delete the trailing endline to keep it on one line
        # -v is for verbose mode when copying (not essential)
        # What comes out of the echo is: cp -v -i --reply=no ./4001/task/faces/con_0027.img
        echo "cp -v -i --reply=${copy_overwrite} ${i} ${outfile}"
        done
}


function error_exit
{

#       -----------------------------------------------------------------------
#       Function for exit due to fatal program error
#               Accepts 1 argument:
#                       string containing descriptive error message
#       -----------------------------------------------------------------------

        echo "${PROGNAME}: ${1:-"Unknown Error"}" >&2
        exit 1
}


function usage
{

#       -----------------------------------------------------------------------
#       Function to display usage message (does not exit)
#               No arguments
#       -----------------------------------------------------------------------

        local tab=$(echo -en "\t\t")

        cat <<- -EOF-

        Usage: ${PROGNAME} [-h | --help]

        SYNOPSIS
        ./${PROGNAME} [-s | --source] [-o | --outputdir]
        Copies all files named con<number>.img and con<number>.hdr
        from the subdirectories (in ./4001 etc ..) specified by SOURCE
        to ./OUTPUTDIR/con_<number>.<extension>

-EOF-

}


function helptext
{

#       -----------------------------------------------------------------------
#       Function to display help message for program
#               No arguments
#       -----------------------------------------------------------------------

        local tab=$(echo -en "\t\t")

        cat <<- -EOF-

        ${PROGNAME} ver. ${VERSION}

        Copy all *.img and *.hdr files from subdirs and rename them to ./<outputdir>/<dirnumber>_con<number>.extension
        If the files were not present in the output directory, it gives feedback in the form of:
        'old filename' -> 'new filename' (because of the -v verbose option in the copy)
        If the files were present in the outputdir it gives NO feedback.

        # Rename script that copies files called old, into new:
        # old ./4001/task/faces/con0027.img
        # new ./outputdir/4001_con0027.img
        # etc ..

        Usage:
        1) Save this script as rename_script.sh
           Put it in the same dir as the ./4001 directories and the outputdir.
        2) chmod the script to make it executable: chmod +x rename_script.sh
        3) When invoked like ./rename_script.sh -s task/faces -o outputdir
           it will output a test.
           Look at the output and when that looks ok:
        5) Invoke the script like so: ./rename_script -s task/faces -o outputdir |sh
           in order to execute the output of the script (this runs the actual copy commands).

        assumptions:
        1) We only copy files from subfolders that start with a letter. In the example t for task
        2) None of the main folders can start with a letter but must be called as a number: 4001
           If they start with letters we don't copy from them.
        3) It should never overwrite files in the outputdir. See variable copy_overwrite.
        4) All files to be copied are named con<number>.<img or hdr>
        5) The directory-names do not contain the letters con.

        $(usage)

        Options:

        [NO OPTIONS]    Show the usage text.
        -s, --source    Dir specified by user is the subdirectory to copy from.
        -o, --outputdir Dir specefied by user is the outputdirectory to copy to.
                        By default this is ./outputdir so this argument is optional.
        -h, --help      Display this help message and exit.

-EOF-
}


#       -------------------------------------------------------------------
#       Program starts here
#       -------------------------------------------------------------------

##### Command Line Processing #####

    while [ "$1" != "" ]; do
        case $1 in
            -s | --source )         shift
                                    if [ "$1" != "" ]; then
                                            # filter out ./ in front of directories
                                            source_dir=`echo $1 |sed 's#^[.]\{0,1\}\/\{0,1\}##g'`
                                    else
                                            helptext >&2
                                            exit
                                    fi
                                    ;;
            -o | --outputdir )      shift
                                    if [ "$1" != "" ]; then
                                            output_dir=$1
                                    else
                                            helptext >&2
                                            exit
                                    fi
                                    ;;
            -h | --help )           helptext >&2
                                    exit
                                    ;;
            * )                     usage >&2
                                    exit 1
        esac
        shift
    done

# test if outputdir exists
# Does $output_dir exist; otherwise create $output_dir
if ! [ -d $output_dir ]; then
    mkdir $output_dir || error_exit "Cannot create directory $output_dir Exiting."
fi

# call the function to find and copy
find_and_copy "$source_dir"

# all done so exit
exit
 
Old 10-10-2006, 08:04 PM   #17
shelfitz
LQ Newbie
 
Registered: Feb 2006
Posts: 9

Original Poster
Rep: Reputation: 0
Hi Muha,

Ah, I see what happened. I just made some con*.img and con*.hdr files to test things and I used the wrong naming convention. The actual files are named as such: con_00*.img and con_00*.hdr and this is consistent (not con00* as the output above suggests). Sorry! I tested it with the right naming convention and it works!

But there is this other issue. The directory where the files live is 4001/stats/snod. I want to avoid getting other con.img and con.hdr files from other subdirectories within the subject directory (there is no way to uniquely specify these in the file name--there will be files with the same names in both directories I'm afraid. Is it possible to specify the directory to search in as a variable--it will have the same path for every subject (i.e.: subnum (4001)/stats/snod)? This would help when applying this to differently named directories as well. The task/faces directory, by the way, was just an example I provided.

Thanks for your awesome help!

Best,
SF
 
Old 10-10-2006, 09:40 PM   #18
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,248
Blog Entries: 8

Rep: Reputation: 235Reputation: 235Reputation: 235
hello. please try this one.

Code:
SUBJECT=4001

OLDIFS=$IFS; IFS=$'\n'
for a in $(find . -type f -name *.img -print; find . -type f -name *.hdr -print); do
	DIRNAME="$(dirname ${a})"
	BASENAME="$(basename ${a})"
	NEWNAME="${DIRNAME}/${SUBJECT}_${BASENAME/_}"
	
	echo "renaming ${a} to ${NEWNAME}
	# remove the comment marks below if everything's already correct
	#echo mv "${a}" "${NEWNAME}" || {
	#	echo "error renaming ${a} to ${NEWNAME}
	#}
done
IFS=$OLDIFS
 
Old 10-12-2006, 06:52 AM   #19
muha
Member
 
Registered: Nov 2005
Distribution: xubuntu, grml
Posts: 451

Rep: Reputation: 37
@konsolebox: I just would like to comment that your script doesn't do anything with the outputdir
It looks like it keeps it all in the same dir?
Quote:
renaming ./4991/stats/snod/con991.img to ./4991/stats/snod/4001_con991.img
Quote:
Originally Posted by shelfitz
Here's what I want to do:
old name: con_0027.img
new name: 4001_con0027.img

old name: con_027.hdr
new name: 4001_con027.hdr

The directories consist of both .hdr and .img files and there will be 27-40 of each (i.e., con_027 ... con_028 ... etc.). All the files in a given directory will get the same numerical prefix. As I navigate to each new subject's directory (I'm dealing with fMRI data), I will have to do the same thing, only I will change the prefix to that particular subject's identification number. I need to do this because I have to create one directory for all the subject files
So what shelfitz wants, as I read it is:
old name: ./4991/stats/snod/con_0027.img
new name: ./outputdir/4001_con0027.img

@shelfitz:
You asked for the subdir part and I already had included that in my script:
Quote:
You can now specify the source directories like /task/faces/ or /stats/snod/
Use the script like so to get all *.img/hdr from ./4001/task/faces/ and ./4002/task/faces/
Code:
./rename_script.sh -s task/faces
Use the script like so to get all *.img/hdr from ./4001/stats/snod and ./4002/stats/snod
Code:
./rename_script.sh -s stats/snod
You can also specify the outputdir with -o so this will work even better:
Use the script like so to get all *.img/hdr from ./4001/task/faces/ and ./4002/task/faces/
to outputdir all_task_faces
Code:
./rename_script.sh -s task/faces -o all_task_faces
Use the script like so to get all *.img/hdr from ./4001/stats/snod and ./4002/stats/snod
to outputdir all_stats_snod
Code:
./rename_script.sh -s stats/snod -o all_stats_snod
Note: include |sh behind the command to make it work!

I made a small change so now it get's con_0027.img or con0027.img and copies that to
4001_con0027.img. So the underscore before the number is optional
The updated script:
Code:
#!/bin/bash

#       -------------------------------------------------------------------
#
#       Shell program to copy all *.img and *.hdr files from subdirs and
#       rename them to ./<outputdir>/<dirnumber>_con<number>.extension
#
#       Copyright 2006, <scriptfreak at gmail.com>
#
#       This program is free software; you can redistribute it and/or
#       modify it under the terms of the GNU General Public License as
#       published by the Free Software Foundation; either version 2 of the
#       License, or (at your option) any later version.
#
#       This program is distributed in the hope that it will be useful, but
#       WITHOUT ANY WARRANTY; without even the implied warranty of
#       MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
#       General Public License for more details.


#       -------------------------------------------------------------------
#       Constants
#       -------------------------------------------------------------------

        PROGNAME=$(basename $0)
        VERSION="0.0.3"

#       -------------------------------------------------------------------
#       Variables
#       -------------------------------------------------------------------

        # set some variables
        # make sure the source_dir is not specified yet
        source_dir=
        # the name for the output directory (set it to what you want)
        output_dir="outputdir"
        # overwrite file in outputdir when it is already present when copying
        copy_overwrite="no"

#       -------------------------------------------------------------------
#       Functions
#       -------------------------------------------------------------------

function test_if_file_does_not_exist
{

#       -----------------------------------------------------------------------------------------------
#       Function to test if a file exists
#               argument is $filename which is the inputfile
#               returns 0 (succes) if the file does not exist
#               usage:
#                       test_if_file_exists <inputfile> || error_exit "<Filename> does not exist! Exiting"
#                       OR
#                       if test_if_file_exists <inputfile>;then echo "Succes!"; else echo "Failure"; fi
#       -----------------------------------------------------------------------------------------------

        if [ "$1" != "" ]; then
                if [ -e $1 ]; then
                        return 1
                else
                        # filename is not empty and filename does not exist
                        return 0
                fi
        else
                return 1
        fi
}


function find_files
{

#       -------------------------------------------------------------------
#       Function to find files called *.img and *.hdr from ./<number>/$source_dir/
#               argument is $source_dir which is the subdirectory to read from
#               So now the find command might look like find ./4001/task/faces/ -name etc ...
#               Exists with an error message if the specified subdirs do not exist.
#       -------------------------------------------------------------------

        # First find all files in subdirs called *.hdr and *.img
        # | sed -n "/^\.\/[0-9]*\/[a-zA-Z]/p" excludes all directories that do start with ./<letter> since we expect these
        # to be output directories and we don't want to copy from them. It also filters out subdirectories that start with numbers.
        find ./*/$1 -name "*.img" -o -name "*.hdr"| sed -n "/^\.\/[0-9]*\/[a-zA-Z]/p" || error_exit "Cannot find files in the subdirectory: $1"

}


function find_and_copy
{

#       -------------------------------------------------------------------
#       Function to find and copy from specified dir
#               argument is $source_dir which is the subdirectory (of ./<number>) to read from
#       -------------------------------------------------------------------

        # The find and copy command. A for loop for each file present.
        # Variable $i is now in the form of ./4001/task/faces/con_0027.img
        for i in `find_files $1`
        do
        # Specify the output file
        # ${i%%/[a-zA-Z]*} leaves only the ./4001 part of ./4001/task/faces/con0027.img
        # In other words, it deletes anything coming after ./4001 which has the form: slash letters; so /task/faces/con0027.img
        #
        # sed 's#^\.#\.\/'"${output_dir}"'#g'
        # This sed converts the output of the echo: ./4001
        # First it looks for 'the begin of the sentence' followed immediately by a .
        # it then replaces . by ./outputdir and echos out the rest, which is /4001
        # So in total this first echo outputs: ./outputdir/4001
        #
        # followed by a replace of ./4001/task/faces/con0027.img into _con0027.img
        # What sed does exactly: replace anything (.*) (here: ./4001/task/faces/) followed by con, an optional _ and the trailing bit,
        # with _con followed by the trailing bit (0027.img)
        outfile="`echo -n ${i%%/[a-zA-Z]*}|sed 's#^\.#\.\/'"${output_dir}"'#g'; echo -n ${i}|sed 's#.*con[_]\{0,1\}#_con#g'`"

        # test if file exists in outputdir. If it exists program continues with a warning message.
        test_if_file_does_not_exist "$outfile" || echo "$outfile already exists! Not overwritten by $i" >&2

        # The copy bit; -i --reply=no is not to overwrite files in $output_dir
        # -n to delete the trailing endline to keep it on one line
        # -v is for verbose mode when copying (not essential)
        # What comes out of the echo is: cp -v -i --reply=no ./4001/task/faces/con_0027.img
        echo "cp -v -i --reply=${copy_overwrite} ${i} ${outfile}"
        done
}


function error_exit
{

#       -----------------------------------------------------------------------
#       Function for exit due to fatal program error
#               Accepts 1 argument:
#                       string containing descriptive error message
#       -----------------------------------------------------------------------

        echo "${PROGNAME}: ${1:-"Unknown Error"}" >&2
        exit 1
}


function usage
{

#       -----------------------------------------------------------------------
#       Function to display usage message (does not exit)
#               No arguments
#       -----------------------------------------------------------------------

        local tab=$(echo -en "\t\t")

        cat <<- -EOF-

        Usage: ${PROGNAME} [-h | --help]

        SYNOPSIS
        ./${PROGNAME} [-s | --source] [-o | --outputdir]
        Copies all files named con<number>.img and con<number>.hdr
        from the subdirectories (in ./4001 etc ..) specified by SOURCE
        to ./OUTPUTDIR/con_<number>.<extension>

-EOF-

}


function helptext
{

#       -----------------------------------------------------------------------
#       Function to display help message for program
#               No arguments
#       -----------------------------------------------------------------------

        local tab=$(echo -en "\t\t")

        cat <<- -EOF-

        ${PROGNAME} ver. ${VERSION}

        Copy all *.img and *.hdr files from subdirs and rename them to ./<outputdir>/<dirnumber>_con<number>.extension
        If the files were not present in the output directory, it gives feedback in the form of:
        'old filename' -> 'new filename' (because of the -v verbose option in the copy)
        If the files were present in the outputdir it gives NO feedback.

        # Rename script that copies files called old, into new:
        # old ./4001/task/faces/con0027.img
        # new ./outputdir/4001_con0027.img
        # etc ..

        Usage:
        1) Save this script as rename_script.sh
           Put it in the same dir as the ./4001 directories and the outputdir.
        2) chmod the script to make it executable: chmod +x rename_script.sh
        3) When invoked like ./rename_script.sh -s task/faces -o outputdir
           it will output a test.
           Look at the output and when that looks ok:
        5) Invoke the script like so: ./rename_script -s task/faces -o outputdir |sh
           in order to execute the output of the script (this runs the actual copy commands).

        assumptions:
        1) We only copy files from subfolders that start with a letter. In the example t for task
        2) None of the main folders can start with a letter but must be called as a number: 4001
           If they start with letters we don't copy from them.
        3) It should never overwrite files in the outputdir. See variable copy_overwrite.
        4) All files to be copied are named con<number>.<img or hdr>
        5) The directory-names do not contain the letters con.

        $(usage)

        Options:

        [NO OPTIONS]    Show the usage text.
        -s, --source    Dir specified by user is the subdirectory to copy from.
        -o, --outputdir Dir specefied by user is the outputdirectory to copy to.
                        By default this is ./outputdir so this argument is optional.
        -h, --help      Display this help message and exit.

-EOF-
}


#       -------------------------------------------------------------------
#       Program starts here
#       -------------------------------------------------------------------

##### Command Line Processing #####

    while [ "$1" != "" ]; do
        case $1 in
            -s | --source )         shift
                                    if [ "$1" != "" ]; then
                                            source_dir=`echo $1 |sed 's#^[.]\{0,1\}\/\{0,1\}##g'`
                                    else
                                            helptext >&2
                                            exit
                                    fi
                                    ;;
            -o | --outputdir )      shift
                                    if [ "$1" != "" ]; then
                                            output_dir=$1
                                    else
                                            helptext >&2
                                            exit
                                    fi
                                    ;;
            -h | --help )           helptext >&2
                                    exit
                                    ;;
            * )                     usage >&2
                                    exit 1
        esac
        shift
    done

# test if outputdir exists
# Does $output_dir exist; otherwise create $output_dir
if ! [ -d $output_dir ]; then
    mkdir $output_dir || error_exit "Cannot create directory $output_dir Exiting."
fi

# call the function to find and copy
find_and_copy "$source_dir"

# all done so exit
exit

Last edited by muha; 10-12-2006 at 07:08 AM.
 
Old 10-12-2006, 11:23 PM   #20
shelfitz
LQ Newbie
 
Registered: Feb 2006
Posts: 9

Original Poster
Rep: Reputation: 0
Hi M,

Well, I tested the the marvelous script in a pseudo environment, and it worked brilliantly! But when I tried it in the real world it didn't work. That is, I executed the command and just got a prompt back instantaneously, with no result in the specified output directory (not even an error message, just nothing). I'm assuming it's because the directory in question (i.e., where I ran it from) violates one of your assumptions (which I should have realized it would sooner!). In addition to numerically named directories, it also has directories named with text. What's more, some of the numerically named directories are named as such: 4001_2. When I tested this in the pseudo environment (no text-based named dirs but dirs named with _2) it just ignored the _2 directories. I imagine the text-based named dirs are also a problem. One other thing: My outputdir is not in the directory in which I launch the script. The program seems to require that the outputdir be in the pwd.

I realize you've given this alot of time, so do know that I understand if you are busy with other things. I can probably work with this myself.

Thank you!!

Last edited by shelfitz; 10-18-2006 at 06:42 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Batch Script to rename files... jamie_barrow Linux - Newbie 16 06-14-2009 02:26 PM
Bash - Batch File Rename Help... emailarron Linux - Newbie 4 01-26-2006 08:35 AM
Batch rename question hellblade Linux - Software 4 05-03-2004 04:57 PM
batch vs ?????? munna_502 Linux - Software 6 04-01-2004 10:42 PM
Got a script to rename a batch of files? jamie_barrow Linux - General 1 08-08-2003 07:52 AM


All times are GMT -5. The time now is 07:48 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration