file name manipulation

chocolatetoothpaste · 03-24-2006, 12:01 AM

I have a script to quickly import pics off my camera. The pictures are stored on my camera in the following format:
DSCF0001.jpg
DSCF0002.jpg
etc.
Now, what if, in the morning I go out and take some pictures, come home, download them, go out and take more pictures, then come home to download them. Well, on the camera it starts back at DSCF0001.jpg. This is bad because it will overwrite the picture I took this morning. So here is the quesion, how could I check to see if the file exists, and if it does, rename the to-be-imported picture to continue where the other pictures left off? IE the last picture I took this morning is DSCF0023.jpg. How do I make the script start copying at DSCF0024.jpg?

nickj6282 · 03-24-2006, 12:24 AM

Can you post your code please?

Thanks
-Nick

chocolatetoothpaste · 03-24-2006, 12:48 AM

Code:

#!/bin/sh

picdir=`date +%m-%d-%y`
usage="Options: [-c camere name] [-d delete source pictures]"

while getopts ":c:d" options; do
  case $options in
    c ) newcam=$OPTARG;;
    d ) delsrc=$OPTIND;;
    h ) echo $usage;;
    \? ) echo $usage

         exit 1;;
    * ) echo $usage
          exit 1;;
  esac
done

if [ $newcam ]; then
camdir=$newcam
else
camdir='fuji-a303'
fi

mkdir -p "$camdir"
mkdir -p ~/"$camdir/$picdir"
cp -r /media/usbdisk/DCIM/100_FUJI/* ~/"$camdir/$picdir"

if [ $delsrc ]; then
rm /media/usbdisk/DCIM/100_FUJI/*
fi

drkstr · 03-25-2006, 05:13 PM

What you are specifically asking for will need a bunch of code to read in all the file names from the camera, then do a bunch of "interpretation" of the file names to figure out what to rename them to. Since I am a lazy person and don't feal like writing all that code for you, I give you the following easier solution:

The script is over righting the files if you run it more then once in the same day because the folder name it creates is based on the date. All you need to do is make this folder more specific by adding the time as well.

picdir=`date +%m-%d-%y_%l-%M`

This way you can run the script as many times as you want in the same day without over righting the same folder. If you want to get real specific, you can add seconds to the folder name too:

picdir=`date +%m-%d-%y_%l-%M-%S`

Hope this is a good enough solution for you.

...drkstr

archtoad6 · 03-26-2006, 06:14 AM

My camera does not start the numbers over. Of course, I don't ever delete all my pics either. I don't know if it is more sensibly programmed than yours (which I doubt because it is <$100 no-name PoS), OR if it always uses the next highest pic number. Maybe yours works the same way. If that is true, try leaving the last (highest #'d) pic in the camera. -- Or have you already?

drkstr · 03-26-2006, 01:34 PM

The script that chocolate posted will copy all of the pics without leaveing any behind. Leaving the last pic on the camera just to stop it from copying over previous files would be horrible design. You would end up copying that pic multiple times. I think adding the time to the folder name it creates to copy the pics to, would be a better solution. But I guess it's up to you chocolate.

regards,
...drkstr

archtoad6 · 03-27-2006, 06:38 AM

At this point we don't know if his camera works the same as mine, heck we don't even know if my camera works the way I think it might, that I described.

If I understand his problem correctly, he wants his pictures to to be #'d consecutively from 1 continuous set of serials. What I am proposing is to let the camera do the job of keeping track of which have been used -- that is if it will.

Even if his script has to a) eliminate 1 duplicate picture each time he dumps the camera & b) make the last picture on the camera r/o, that will still be simpler than having to make the script add the right increment to the pictures' names.

& BTW, even if he chooses to continue to put them in separate dated directories, each picture will have a unique name, which surely is "A Good Thing"(tm).

I believe the code to make the newest picture r/o is as simple as:

Code:

CAMERA='/media/usbdisk/DCIM/100_FUJI/'
chmod 444 `ls -v $CAMERA  | tail -1`   # The "-v" may not be necessary.

All this assumes that my speculation about how digital cameras control the picture serial #'s is correct. I would like to know if chocolate's camera does indeed work this way. Please consider an experiment to find out.

drkstr · 03-27-2006, 05:27 PM

I agree. It would be ideal for the camera itself to track the incremental naming of the pics. You also bring up a good point about moving all of the pics to one “master directory.” Not having unique names would cause some problems. I apologize for my poor choice of wording in my previous post. When I said “better solution,” I really meant a “simpler solution.” If it were me however, I would be bothered by having to leave one pic behind when I clean off my camera, mainly because I have the option of deleting all my pics, or deleting them one by one. The later of the two would cause me to through my camera at the wall.

What if we look at it another way; instead of copying the name the camera makes, use the script to make a completely new and unique name. You could store a number somewhere like /home/someusr/.picnum. Then have the script use that number in creating the name of the pic as it copies over, then store an incremented value. If you wanted to get real fancy you could add in the date to the picname and reset picnum if the date changes. Such as:

PIC0326060001
PIC0326060002
PIC0327060001 ..etc

While this would be easy to do in perl, I’m not quite sure about bash (not my area of expertise). What do you think archtoad? Is there an easy way to read the files in one at a time then copy over to a different name? If not, it wouldn’t be to hard to translate the script to perl and write it like that. Let me know what you think, I’m interested to see how this turns out.

Regards,
…drkstr

chocolatetoothpaste · 03-28-2006, 01:47 AM

Wow, you guys have some interesting ideas. Keep going, I like the things you are coming up with. I have actually been considering a few options of how I want to handle copying the pictures. Let me just say up front, I am NOT interested in things like leaving 1 picture behind and stuff like that. I want this script to be dynamic and do things without me intervening too much. As far as my camera goes, when you take pictures, then delete them all from the card, the camera starts back at DSCF0001.JPG. I initially wanted it to find the very highest number in a dir, and then continue from that point to rename the pictures. That way, I don't worry about things, I just press a launcher, and it takes care of the rest. I have updated the code since I last posted, so I will post it tomorrow after I wrap a few more changes.

muha · 03-28-2006, 02:23 AM

Here's what i do to run a backup for all files in a folder:
It is not reeeeeally clean code but it works for me and you could use the random-directory part from it.
The inspiration for it (and layout/startscript) comes from: http://www.linuxcommand.org/

Code:

#!/bin/bash

#       -------------------------------------------------------------------
#
#       Shell program to backup files.
#
#       Description:
#               Copies only files in the current directory to a directory
#               ./backup/backitup<date>
#
#       Usage:
#
#               ./backitup [NO OPTIONS]
#
#       Options:
#
#               -d, --dir       Specify another main-backupdir
#               -h, --help      Display this help message and exit.
#
#
#       Revision History:
#
#       02/05/2006      File created by new_script ver. 2.1.0
#
#       -------------------------------------------------------------------


#       -------------------------------------------------------------------
#       Constants
#       -------------------------------------------------------------------

        PROGNAME=$(basename $0)
        VERSION="0.0.1"

#       -------------------------------------------------------------------
#       Variables
#       -------------------------------------------------------------------

        dir=
        main_dir=./backup


#       -------------------------------------------------------------------
#       Functions
#       -------------------------------------------------------------------


function set_dirs
{
#       -------------------------------------------------------------------
#       Funtion to set the dir variables
#               no arguments
#       -------------------------------------------------------------------

        # dir to create
        backup_dir_date=$main_dir/$PROGNAME`date -dtoday '+%Y%m%d'`
        backup_dir_date_time=$main_dir/$PROGNAME`date -dtoday '+%Y%m%d_%H%M%S'`
}


function error_exit
{

#       -----------------------------------------------------------------------
#       Function for exit due to fatal program error
#               Accepts 1 argument:
#                       string containing descriptive error message
#       -----------------------------------------------------------------------


        echo "${PROGNAME}: ${1:-"Unknown Error"}" >&2
        exit 1
}


function signal_exit
{

#       -----------------------------------------------------------------------
#       Function to handle termination signals
#               Accepts 1 argument:
#                       signal_spec
#       -----------------------------------------------------------------------

        case $1 in
                INT)    echo "$PROGNAME: Program aborted by user" >&2
                        exit
                        ;;
                TERM)   echo "$PROGNAME: Program terminated" >&2
                        exit
                        ;;
                *)      error_exit "$PROGNAME: Terminating on unknown signal"
                        ;;
        esac
}


function usage
{

#       -----------------------------------------------------------------------
#       Function to display usage message (does not exit)
#               No arguments
#       -----------------------------------------------------------------------

        echo "Usage: ${PROGNAME} [-h | --help] [-d | --dir]"
        echo -e "SYNOPSIS
       ./${PROGNAME} [NO OPTIONS] - copies all files in current directory to $main_dir/$PROGNAME<date_today>/\n"
}


function helptext
{

#       -----------------------------------------------------------------------
#       Function to display help message for program
#               No arguments
#       -----------------------------------------------------------------------

        local tab=$(echo -en "\t\t")

        cat <<- -EOF-

        ${PROGNAME} ver. ${VERSION}
        This is a script to backup files.
        If the dir $main_dir/$PROGNAME<date_today> exists it will create the directory
        $main_dir/$PROGNAME<date_today_time>
        If that dir exists it will create the directory
        $main_dir/$PROGNAME<date_today_time>_<some_random_number>


        $(usage)

        Options:

        [NO OPTIONS]    Copies all files in current directory to $main_dir/$PROGNAME<date_today>/
        -d, --dir       Copy to a dir specified by user
        -h, --help      Display this help message and exit.

-EOF-
}


#       -------------------------------------------------------------------
#       Program starts here
#       -------------------------------------------------------------------

##### Initialization And Setup #####

# Set file creation mask so that all files are created with 600 permissions.

umask 066


# Trap TERM, HUP, and INT signals and properly exit

trap "signal_exit TERM" TERM HUP
trap "signal_exit INT"  INT

##### Command Line Processing #####

while [ "$1" != "" ]; do
    case $1 in
        -d | --dir )            shift
                                if [ "$1" != "" ]; then
                                        main_dir=`echo $1 |sed -e 's/\/$//g' -e 's/^.\///g'`
                                else
                                        helptext
                                        exit
                                fi
                                ;;
        -h | --help )           helptext
                                exit
                                ;;
        * )                     usage
                                exit 1
    esac
    shift
done

##### Main Logic #####

# run the function set_dirs to initialize the dirs to backup to
set_dirs

# Does $main_dir exist; otherwise create $main_dir
if ! [ -d $main_dir ]; then mkdir $main_dir; fi

# Does $main_dir/$PROGNAME<date_today> exist; otherwise create $main_dir/$PROGNAME<date_today_time>
# If that directory exists create the directory $main_dir/$PROGNAME<date_today_time>_<some_random_number>
if [ -d $main_dir ]; then
        if [ -d $backup_dir_date ]; then
                if mkdir $backup_dir_date_time; then
                        dir=$backup_dir_date_time
                else
                        dir=$backup_dir_date_time"_"$RANDOM
                        while ! mkdir $dir; do
                                dir=$backup_dir_date_time"_"$RANDOM
                        done
                        echo "Instead we try a random dir called $dir"
                fi
        else
                if mkdir $backup_dir_date; then
                        dir=$backup_dir_date
                else
                        echo "Creation failed of dir $dir"
                        exit 1
                fi
        fi
fi

# Copy all files in this dir to the dir just created; output message if copy succeded
if cp --preserve `find . -maxdepth 1 -type f -print` $dir ;then
        echo "Copy succeeded to $dir"
else
        echo "Copy failed to dir $dir!"
fi

exit

timmeke · 03-28-2006, 05:14 AM

drkstr, saving the next picture number is a great idea, indeed.
However, wouldn't it be easier to simply find the highest number already present and then increment one?
A simple command like:

Code:

maxnum=`ls pict_dir/* | cut -c 4- |sort -n -r |head -1`

should do the trick, no?

drkstr · 03-28-2006, 01:18 PM

Wow, figuring out the last number in the file name was much easier then I though. I am pretty new to Linux but continue to be amazed by its robustness every day. It looks like timmeke’s response would answer chocolate’s question. Since I also deal with a problem similar to chocolate’s, I would like to expand on the original question. Archtoad brought up the point that if you ever moved the pictures to another directory, the numbering scheme would no longer be valid. It would be nice to have a universal counter that would keep the total number of pics copied to the computer, thus storing it in a file.

To use either method, the file names must be read in one at a time and then copied over to the directory the script makes. I’m not quite sure how to do this in a bash script. To read in filenames one at a time in perl, all you need to do is:

Code:

while ( defined($filename =<*>) )
{
  # code to rename and copy file
}

Could anyone tell me how to do this in bash?

Thanks!
…drkstr

archtoad6 · 03-28-2006, 02:09 PM

Lucky, I was just getting ready to write my post when I saw drkstr's #12. I will give some thought to the bash increment Q, but 1st I want to describe the results of my "clear the camera" experiment:

In short, my guess was right -- my camera resets its counter when you clear it & doesn't when you don't. It even continues an earlier sequence when when the old last pic is put back on the cam (excuse chat room-ish abbrs., but I need to save typing).

So, as far as I am concerned, letting the cam provide the seq. #'s is the simple & easy way to do it. That leaves the problem of how to automate leaving the last pic on the cam. I think I have a scriptable solution for that.

Code:

# CAMERA=<path_to_pics_on_cam>
CAMERA=/mnt/flash2/dcim/100media           # on my sys., today -- "flash2" can vary

# DESTDIR=<path_to_pics_on_box>
DESTDIR=~/Pictures/Norcent-dcim-100media   # again on my sys.

chmod 755 $CAMERA/*                        # normalizing permissions for copying
cp  $CAMERA/* $DESTDIR
chmod -w $CAMERA/`ls $CAMERA |tail -1`     # this works because of the way 'ls' works
rm $CAMERA/*
chmod u+w $CAMERA/*                        # for safety

Warning, while I did test the 'chmod' cmds., I did not test the 'cp & 'rm' because I used Konqueror to do the file transfer. I DID use the cam's "delete all pics" cmd. & it did NOT touch the the ro last pic.

BTW, my cam is a Norcent DSC 5LF (label), seen as "ID 055f:c233 Mustek Systems, Inc." (per lsusb). It names pics "imag####.jpg"; what it will do when it hits 10k, I don't know.

archtoad6 · 03-28-2006, 02:23 PM

Quote:

Originally Posted by drkstr

... To use either method, the file names must be read in one at a time and then copied over to the directory the script makes. I’m not quite sure how to do this in a bash script. To read in filenames one at a time in perl, all you need to do is:

Code:

while ( defined($filename =<*>) )
{
  # code to rename and copy file
}

Could anyone tell me how to do this in bash?

Thanks!
…drkstr

Might it be as easy as:

Code:

for FILE in `ls $DIR`
do
   # code to rename and copy file
   # remembering that  "$DIR/$FILE"  may be needed
done

chocolatetoothpaste · 03-29-2006, 12:06 PM

Quote:

Originally Posted by drkstr

Archtoad brought up the point that if you ever moved the pictures to another directory, the numbering scheme would no longer be valid. It would be nice to have a universal counter that would keep the total number of pics copied to the computer, thus storing it in a file.
…drkstr

That would not be a good idea because if you delete the file or lose it accidentally, you are screwed. I think one thing a lot of programmers overlook is making things dynamic. I work programming a website for a real estate company. They have a very robust and complicated website. The programmer before me, when he setup his development environment, used WAY too many environment specific variables, and as a result, when I came to work on the site, it was broken on my computer. It worked perfect on HIS laptop and the server, but nowhere else. It took me 2 months to wade through all the garbage, and I had to do a lot of testing and development on the server which is DANGEROUS! Anyway enough rant. I have figured out how to get the largest file and add 1 to it. Now all I have to do is create a loop the size of the amount of picuters on the cam. Then transfer them and rename them 1 by 1 as they transfer. It really won't matter if a directory gets moved or renamed. When you go to transfer pics, if the destination dir does not exist, it creates it. If it does, it renames pics so they are not overwritten. The beauty is that all you have to do is copy the new script to $PATH on a new computer, and you are good to go! No environment specific vars. That is assuming you are running bash. But that is the same as the difference between C++ and Java; different but similar. So, I almost have the script to the point I want, and then I will post it for all to benefit. Maybe I will turn this into a project on my website and keep it going for a while and keep adding on to it. Thanks for your responses guys. Keep them coming, I like the ideas you have.
PS - If you want to see one of my current projects, go to http://www.perc.us/igenda. It is a scheduling system for client-side/admin-side.