Download your favorite Linux distribution at LQ ISO.
Go Back > Forums > Linux Forums > Linux - Newbie
User Name
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!


  Search this Thread
Old 08-06-2009, 02:21 PM   #1
LQ Newbie
Registered: Mar 2009
Posts: 4

Rep: Reputation: 0
How to search for missing files and pass their names on to another shell script

Hi all,

I'm pretty n00bish, If you can help me with this question, I will be eternally grateful. I have a folder full of data, but occasionally some data goes missing. I need to write a script that scans through my data directory, finds the missing files, and sends the file name to a script that downloads it. The file names follow a particular pattern. Here's an example:


In this case, my script should notice that Q3 is missing and run a shell script (that I've already written) to get it. It needs to be able to search through all of the folders in in the data directory. I'm thinking I can create a list of file "prefixes" like "BRIB.Q1", "BRIB.Q2"... etc that it needs to match up with.

What kind of commands do I need to look at to write this script? How do I recursively search through directories, checking to make sure a list of files following a particular pattern exist?

Old 08-06-2009, 04:45 PM   #2
LQ 5k Club
Registered: May 2003
Location: London, UK
Distribution: Debian "Testing"
Posts: 6,117

Rep: Reputation: 416Reputation: 416Reputation: 416Reputation: 416Reputation: 416
I have a folder full of data, but occasionally some data goes missing.
A more logical approach might be to find out just why "some data goes missing".

Then you could fix the problem that is causing the problem (rather than "patching it up") and the problem would go away. Writing scripts to mask a problem you have not completely understood is not going to help you in the long term.

You need to investigate exactly why, or how "some data goes missing", and fix that.

Then your problem will be solved.

If you need further advice, please post more details.
Old 08-06-2009, 04:51 PM   #3
Senior Member
Registered: Mar 2006
Posts: 1,896

Rep: Reputation: 61
It depends how general you need the script to be. For the example you've given, (and generalizable to any situation where the pattern is simply an incrimenting number, you could:


get_missing=<insert path and script name here>

for n in $(seq 1 4); do
   file=$(eval echo "$pattern")
   [ ! -e "$file" ] && $get_missing "$file"
Since you are new to this, here is a bit of a "blow by blow" explanation of the script:

I told the script to use bash because I've run into trouble on Ubuntu when it uses dash for sh. When I defined $pattern, I escaped the dollar sign on $n with a backslash so substitution was not done at that step. seq, in this case will generate the sequence "1 2 3 4" which the for loop will iterate through. Each time through the loop, $file will get assigned a new value based on $n. (If anybody knows of a cleaner way to do that step, please let us know.) Finally it tests for the existance of $file and if it doesn't exist, it calls your script with the missing file name. If the files you are looking for are regular files, you could use -f instead of -e. (Look at the bash man page for information on the builtin command test.)

More general scripts, or scripts that need to iterate through letters, will of course, be more complicated.

Last edited by blackhole54; 08-06-2009 at 04:54 PM. Reason: cleanup
Old 08-07-2009, 01:59 AM   #4
LQ Newbie
Registered: Mar 2009
Posts: 4

Original Poster
Rep: Reputation: 0
You need to investigate exactly why, or how "some data goes missing", and fix that.
I wrote that because I didn't want to make the post too long with unnecessary information. I have a cron job running that downloads the data, which is being constantly generated, from a remote server. To get the data, I run someone else's program to generate it, which messes up sometimes. It's just weird and I don't have much control over it.


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
Shell script help: How to pass a directory path to the script? dv502 Programming 13 09-10-2008 04:07 AM
pass php variables to shell script viveksnv Programming 1 03-05-2008 12:46 AM
shell script to move files from one system to another sytem with file names in a txt coral_km Linux - Newbie 3 02-13-2008 11:23 PM
Need a script to search and replace text in file using shell script unixlearner Programming 14 06-21-2007 11:37 PM
PHP pass argument to shell script monzter Programming 2 08-14-2004 07:16 AM > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 12:26 PM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration