[SOLVED] Scheduled copy based on filename

AndySocial · 06-17-2010, 01:02 PM

Since I'm not building a specific piece of software but a shell script, I hope this is in the right forum.

At work, we use RHEL for the basic system but have Windows clients attached as well. On the Win side, there's a program that I'm trying to duplicate on the Linux side. Because of established procedures, I can't change the way files are named so please don't suggest I do that (my boss would not be happy). We have files stored on an NFS share we manipulate on either side.

The program is used to copy files from one directory to another, based on the filenames. The first two characters of the filename are ignored (they're for human-readable sorting purposes only), the next four characters are the time, and then all characters after that are ignored. The time is read and the file gets copied at that time. (This was all done in VC++ many years ago, and nobody knows where the source code is.) For example, a file named 2d0730abcd.txt would be copied from the source directory to the destination directory at 7:30am.

It seems to me that it should be fairly straightforward to build a shell script on the Linux side to do the same thing, but darned if I can figure out which magical combination of commands to use. Would the shell script end up creating a giant pile of AT commands? HELP!

crabboy · 06-17-2010, 03:12 PM

I don't quite follow you. Are you copying all files in the source directory or only files with a specific time value in the filename? Would that value be passed into the script or captured elsewhere? I don't see this as being too difficult, but based on your description, I'm not sure where you need help.

Looking at all the 2-6 character of the filename for every file in a directory you can do this:

Code:

for i in `ls`; do
echo ${i:2:4}
done

Sergei Steshenko · 06-17-2010, 03:12 PM

Quote:

Originally Posted by AndySocial

Since I'm not building a specific piece of software but a shell script, I hope this is in the right forum.

At work, we use RHEL for the basic system but have Windows clients attached as well. On the Win side, there's a program that I'm trying to duplicate on the Linux side. Because of established procedures, I can't change the way files are named so please don't suggest I do that (my boss would not be happy). We have files stored on an NFS share we manipulate on either side.

The program is used to copy files from one directory to another, based on the filenames. The first two characters of the filename are ignored (they're for human-readable sorting purposes only), the next four characters are the time, and then all characters after that are ignored. The time is read and the file gets copied at that time. This was all done in VC++ many years ago, and nobody knows where the source code is.

It seems to me that it should be fairly straightforward to build a shell script on the Linux side to do the same thing, but darned if I can figure out which magical combination of commands to use. Would the shell script end up creating a giant pile of AT commands? HELP!

So, what did you do to get the time part out of the file name Specifically, did you read

man cut

?

tsg · 06-17-2010, 04:01 PM

If you want easy:

Code:

# in case the script takes longer than one minute to run
timenow=`date +%H%M`    
for fname in *; do
     ftime=${fname:2:4}
     if [ $ftime -eq $timenow ]; then
          cp $fname /path/to/$fname;
     fi
done

Put it in a cron job run once a minute. Although, if the files don't ever get cleared from that directory, it could eventually take a long time to run.

To get more sophisticated, you could look into inotify which will tell you when something in the filesystem changes.

AndySocial · 06-18-2010, 05:15 AM

Quote:

Originally Posted by crabboy

I don't quite follow you. Are you copying all files in the source directory or only files with a specific time value in the filename? Would that value be passed into the script or captured elsewhere? I don't see this as being too difficult, but based on your description, I'm not sure where you need help.

Looking at all the 2-6 character of the filename for every file in a directory you can do this:

Code:

for i in `ls`; do
echo ${i:2:4}
done

We're copying the files only when their time is due. So, 1d1345.txt would be copied at 1:45pm.

GazL · 06-18-2010, 07:36 AM

Code:

find /srcdir -name "??$( date +%H%M )*" -exec cp -t /destdir {} +

Schedule that to run once a minute and you're sorted.

AndySocial · 06-18-2010, 07:48 AM

TSG - thanks. That worked perfectly. I put it into a while loop instead of relying on cron, and added a file existence check and le instead of eq so that if I start the script late, it will pick up the older files too.

I knew it wasn't complicated, but I just couldn't make the mental leap I needed to. Thanks again!

David the H. · 06-18-2010, 07:57 AM

Quote:

Originally Posted by crabboy

Looking at all the 2-6 character of the filename for every file in a directory you can do this:

Code:

for i in `ls`; do
echo ${i:2:4}
done

Just for the record, it's not recommended, and just plain unnecessary, to call on ls in a loop like this. Bash is perfectly able to glob the filenames on its own.

The very first entry here explains it in more detail.
http://mywiki.wooledge.org/BashPitfa...0ls_.2A.mp3.60

Note that the part about quoting the variable applies here too.

tsg · 06-18-2010, 08:39 AM

Quote:

Originally Posted by AndySocial

TSG - thanks. That worked perfectly. I put it into a while loop instead of relying on cron, and added a file existence check and le instead of eq so that if I start the script late, it will pick up the older files too.

If you're doing a straight copy instead of a move, the -le is going to keep matching files you've already copied.

Quote:

I knew it wasn't complicated, but I just couldn't make the mental leap I needed to. Thanks again!

You're welcome.

AndySocial · 06-18-2010, 12:01 PM

Quote:

Originally Posted by tsg

If you're doing a straight copy instead of a move, the -le is going to keep matching files you've already copied.

You're welcome.

That's why I've got a file existence check. This is only to handle a directory with around 100 files, so it doesn't need to be too robust.
:-)

tsg · 06-18-2010, 01:16 PM

Ah, okay. I misunderstood.