My first Question

I'm new to scripting but am learning. I need a shell script I believe.

I have a directory pretty deep we run a manual check.

ls -ltr /*/*/*/*/*/incoming/*

This list out all the files from the various incoming directories and there is another process that moves them off. However they don't all get picked up and if they're stuck in these directories we'd like to be able to move them off to an error folder say if they have not increased in size for the last 30 minutes.

Seems like this should be pretty simple but not sure where to begin.

<HELP>

Hello and welcome to LQ. Hope you like it here.

If clues are enough you need to:
- set the script up as a cronjob
- find to find the directory,
- then find to find any leftovers,
- a state file to save filesizes (or hashes) in,
- compare filesizes and then,
- move files.


am learning
elif clues are not enough because you need more feedback: post what you got (even pseudo code) and we'll work on that, NP.


not sure where to begin
else if you really just want/"need" a script then ask for a script. NP. (fi)

Originally Posted by unSpawn Hello and welcome to LQ. Hope you like it here. If clues are enough you need to: - set the script up as a cronjob - find to find the directory, - then find to find any leftovers, - a state file to save filesizes (or hashes) in, - compare filesizes and then, - move files. am learning elif clues are not enough because you need more feedback: post what you got (even pseudo code) and we'll work on that, NP. not sure where to begin else if you really just want/"need" a script then ask for a script. NP. (fi)

Ok I think I need a script

These files aren't in one directory there in a bunch of different directories but all the sub directories have an incoming forlder where people upload files that's why I was thinking

Here's how I started

rm -f /tmp/incoming01.txt > /dev/null 2>&1

#Updates timestamp and creats the file again.
touch /tmp/incoming01.txt

#List files in the directory
ls -ltr /*/*/*/*/*/incoming/* > /tmp/incoming01.txt

but not sure how to create like two files for a compare 30 minutes apart and not sure how to compare them and then clean them up....

xgringo(jason)

What if I said please?

The other dilema is I may not be able to use cron maybe something that has to start and run in the background.

I was thinking if I created two files and then used diff and put the diff output to a third file.

Not sure how to check and see if file one exist create file two and then how to make file one file two after the diff.... thinking outloud.

And then some how used awk to move the files to the error folder....

not sure...

I think you can just use find to search for files that haven't been modified in the last 30 minutes.

find / -regex '.*/incoming/.*' -mmin +30 -exec mv '{}' /errorfolder ';'

This should move all files older than 30 minutes and that have /incoming/ somewhere in their path to /errorfolder. Is this what you wanted?

The manual for find says there are security risks in using -exec. I confess I don't really understand what it was saying. Maybe someone with more experience can say what security risks the above command might pose.

For me the mmin didn't work so taking ntubski's "find" I, slightly convoluted, do:

#!/bin/sh
# Uncomment next line for debug/halt-on-error mode
# set -xe
# Whoami
progn=${0//*\//}
# Find anchored in /upload/root so we don't have to search the whole fs, and
# only looking for files, and print epoch, username and /full/path/filename:
find /upload/root -regex '.*/incoming/.*' -type f -printf "%A@ %u %p\n" | while read e u f; do
 # If epochtime matches thirty minutes minus one for ops
 if [ "$[$(/bin/date '+%s')-${e}]" -gt "1759" ]; then
  # move (force) the file if not in use (fuser) and prefix it with the epochtime,
  # username and full path (slashes replaced by underscores) for whatever purposes:
  /sbin/fuser "${f}" >/dev/null 2>/dev/null && \
  mv -f "${f}" "/error/${e}_${u}${f//\//_}"; ret="$?
  # ...do something useful with the move exit code:
  case "$ret" in 0) r=OK;; 1) r=FAILED;; *) r="exited with errorcode $?";; esac
  # ..and log what we did:
  logger "${progn}: moved ${f}: ${r}."
 fi
# End of find loop
done
# Always exit script properly
exit 0

* epoch time format (as in "date '+%s'") is seconds since that memorable date in the seventies. Since we use epoch the resolution is that big you could run this like every n seconds. Not that you would want that though, mainly for performance reasons...

Wow this is great thank you so much? Wondering how you guys think in code and how long is it going to take me.... I will give this a shot but I'm amazed....Thanks

and how long is it going to take me....
Three days. Yes! You've read it right! You can be a world-class shellscripter in THREE DAYS with my new improved sleep-while-u-learn method! Surprise Friends and Co-workers! Force your Employer into Submission! Get Girls to Really (I mean *Really*) Like You! Gain Access to Power and Wealth! Only USD 50 for the First Nine Lessons! Now with free handy mouse cleaner that doubles as rash remover! (Abrasives should not be used on Sensitive Skin) Ask for the brochure! (Only where not prohibited by Law). Call:


Heh ;-p
Just go and script.
You'll get there when you get there.

lol,
apart from shell scripting, you can write advs too.

Heh. Does that mean I've enticed you enough to buy it?

lol, its like the spam I ignore.

Hmm. But this isn't spam. Honestly.

Originally Posted by unSpawn For me the mmin didn't work

Why wouldn't it work? It definitely works for me.
Version perhaps?
~% find --version
GNU find version 4.2.27
Features enabled: D_TYPE O_NOFOLLOW(enabled) LEAF_OPTIMISATION

Why wouldn't it work?
Beats me.


Version perhaps?
GNU find version 4.1.7 here, but it would seem unlikely -mmin functionality changed over time, right?