How to automatically move files from one folder into another folder?

xmrkite · 11-04-2006, 12:29 AM

Hello, I want to do this:

1. Copy files from a windows box into a temp folder on my linux box
2. Once the files are copied, run ImageMagick's Mogrify in order to resize them
3. Once resized, move them into a permanent folder

Here are the two scripts i use to do this (and then the issue):

SCRIPT 1 (script1.sh)

Code:

#!/bin/bash
script2.sh
exit 0

SCRIPT 2 (script2.sh)

Code:

#!/bin/bash
cd /home/sharedfolder/
for fp in *; do
mv "$fp" "/home/tempfolder/$fp";
`/opt/imagemagick/bin/mogrify -monitor -limit memory 1 -limit map 2 -quality 75 -size 1024x768 "/home/tempfolder/$fp"`
done
cd /home/tempfolder/
for fp in *; do
mv "$fp" "/home/permanentfolder/$fp";
sleep 60
script1.sh
done

The idea is that the script runs every 60 seconds so that whenever pictures are moved to the shared folder over the network, they are auto resized and moved. It works nicely, and i'm sure there is a much better way to do the same thing, but my main issue is that while images are still copying over, they get moved before they finish copying over and then they're resized and look bad. Basically, the images look like they're half drawn, with the bottom half usually being black or pink or something.

So my question...how do i make it so that the image does not get moved until it is fully stored on the network drive? So if i am copying images over, and 10 have finished and the 11th is copying over, the move and mogrify only look at the 10 that have finished and leave the 11th for the next time the script runs, which will be in 60 seconds?

Thank you very much for any help/suggestions, and I hope that I explained this clearly.

jschiwal · 11-04-2006, 01:45 AM

I think that it would be better to put both copies and the mogrify command in the same for loop.
You could use "convert" instead. It can resize the file and write the output to a different filename. For example:

Code:

for fp in /home/sourcefolder/*.jpg
do
   convert "$fp" -resize 100x /home/perminantfolder/"$fp" && rm "$fp"
done

You might also want to look into writing a service that runs in the background and polls inotify to activate.
http://www-128.ibm.com/developerwork...l-inotify.html

xmrkite · 11-04-2006, 01:56 AM

Hello.

I will look into inotify. Convert doesn't work well for my application, as it takes far too long when compared to mogrify (convert takes 30 seconds or so, mogrify takes 1-2 seconds). But that is not part of the main problem. Even if i didn't use mogrify and just did the copy and move, I still would have the issue of the file being moved to the new directory before it was fully transfered from the windows box in the first place, leaving me with a partially finished jpg.

I was hoping there was a way that after the

Code:

for fp in *; do

I could put a line of code that would determine if the file was still being transfered from the windows computer or if the file was completely transfered. Also, if i could somehow lock that file down until the transfer completed, that would work too.

So i'd have something like:

Code:

for fp in *; do
if "check if file transfer is complete"   ---here is where i need the help
then
mv "$fp" "/home/tempfolder/$fp";
else "go onto the next image"
etc etc etc

Any help would be great.

jschiwal · 11-04-2006, 07:24 AM

I didn't realize your problem was in operating on partially copied files from a windows share.
Also, I didn't mention, that the backticks don't make sense.

Instead of using filename globbing, use the file command and select files that are over 1 minute old. You might want to adjust that time, I don't know that the speed of your connection is.

Code:

cd /home/sharedfolder
for picture in $(find /home/sharedfolder/ -maxdepth 1 -type f -iname "*.jpg" -mmin +1); do
  mv "$picture" /home/tempfolder/
  /opt/imagemagick/bin/mogrify -monitor -limit memory 1 -limit map 2 -quality 75 -size 1024x768 
  mv "$picture" /home/permanentfolder/
done

Instead of using two scripts calling eachother, you could instead just use an infinite loop.
You also might want to increase the sleep time and let the program process groups of files. Or the program could be dispatched every 15 minutes or so using a cron job and process groups of files in a batch.

xmrkite · 11-05-2006, 02:19 AM

Crazy, but my version of find does not allow the mmin option. I can do mtime, but not mmin.

but what about comparing the cksums of the files (or something like that)?

I have been at it for some time but just cant seem to figure out how to do that. Here is the script i tried...

Code:

cd /home/sharedfolder/
for fp in *; do
cp "$fp" "/home/tempfolder/$fp";
CKSUM=`cksum /home/sharedfolder/$fp`
CKSUM2=`cksum /home/tempfolder/$fp`
if [ "$CKSUM" -eq "$CKSUM2" ]; then
echo File Sizes are equal and run mogrify script
fi

Please let me know if you have a fix for that.

gruntwerk · 11-05-2006, 09:53 AM

are you sure that you dont have multiple copies of your script running at the same time?
ie. #1is going thru the convert and #2 is copying the files
copy one sees the name of the currently copying file and starts to convert it
maybe you need some sort of lock file so that it knows it is currently running
just a thought

xmrkite · 11-05-2006, 10:39 AM

I'm pretty sure it's ok the way it is because each command in the bash file wait for the previous one to finish before they run.

If i copy over 100 images, generally about 96 or so come out perfect, it's just those 4 that come out bad.