Need help with a script to compare md5sums of 4 files in loop

bsipple · 10-10-2016, 10:05 AM

Greetings,
I have searched through threads and online and cannot find anything that would help me in a specific way.

I am trying to check to see if 4 files listed in "files.txt" have tasks running on them.
Each file needs to be check for running tasks before moving on to the next file in the list.

example:

Code:

Checking file1
No change detected... checking file1
No change detected... checking file1
Tasks are running on file1 
No change detected... checking file2
No change detected... checking file2
No change detected... checking file2
No change detected... checking file2
Tasks are running on file2
[...]

I have been trying to use md5sum to do the check --which does seem to work, except that my script is not checking to completion before moving on to the next file in the list.

Tasks have to be running before the script moves on.
The script I have works perfectly for a single file, but when I add a for loop, it fails to actually check...

Here is the script that works for a single file:

Code:

filename="/path/to/myfiles/file1"
lastfile=$(ls -l $filename)
while true
do
  sleep 1
     newfile=$(md5sum "$filename")
     if [ "$newfile" != "$lastfile" ]
     then
     echo "Tasks are running on $filename"
      lastfile="$newfile"
      break
else 
 echo "Checking for running tasks on $filename"
fi
done

I need to be able to go through files listed in files.txt...

Which looks like this:

Code:

/path/to/myfiles/file1
/path/to/myfiles/file2
/path/to/myfiles/file3
/path/to/myfiles/file4

Here is the script for the loop:

Code:

for filename in $(cat files.txt); do
lastfile=`ls -l $filename`
while true
echo "Checking $filename"
do
	sleep 1  
	newfile=`md5sum "$filename"`
	if [ "$newfile" != "$lastfile" ]
then
    echo "Tasks are running on $filename"
    lastfile="$newfile"
    break
    else 
    echo "No change detected... $filename"
fi
done
done

The output I get is below:

Code:

-rw-rw-r-- 1 www www 1230 Oct 10 09:26 /path/to/myfiles/file1
1c05ebb1488d5dbfbcc0d34c2f7730d9 /path/to/myfiles/file1
Tasks Running /path/to/myfiles/file1
-rw-r--r-- 1 www www 1230 Oct 10 09:26 /path/to/myfiles/file2
c61d58de60268dc87194a3f76951b9cb /path/to/myfiles/file2
Tasks Running /path/to/myfiles/file2
-rw-r--r-- 1 www www 1229 Oct 10 09:26 /path/to/myfiles/file3
2204ee4bb052ccb13bb489a18a5962af /path/to/myfiles/file3
Tasks Running /path/to/myfiles/file3
-rw-r--r-- 1 www www 1229 Oct 10 09:26 /path/to/myfiles/file14
4f714eb328b7784b8485ce8a2538825b /path/to/myfiles/file4
Tasks Running /path/to/myfiles/file4

I have turned off updates to these files to see if the script is even working, and it is not...

I could really use some help with making this work, or... if someone knows a better way to do this -- Please let me know

Note -- I have no access to install software -- so I cannot add a 3rd party software to do this task... --I Wish--

Thanks in advance

keefaz · 10-10-2016, 04:05 PM

You need to revise your logic, comparing md5 checksum with ls -l output will lead to nowhere useful

First store each file checksum somewhere, then compare new checksum with old checksum

michaelk · 10-10-2016, 05:01 PM

Welcome to LinuxQuestions.

It appears that you are checking to see if the output of the ls command changes. If some process is writing to the file then the access time and size will change.

You can use the command set -x at the top of the script to see how your script runs and outputs a trace of the commands and arguments to help with debugging.

By the way you have the same problem in your first script but since the output does not match the script we can not compare the two. Since the initial values are different your logic fails.

lastfile = -rw-rw-r-- 1 www www 1230 Oct 10 09:26 /path/to/myfiles/file1
and
newfile = 1c05ebb1488d5dbfbcc0d34c2f7730d9 /path/to/myfiles/file1

aragorn2101 · 10-11-2016, 03:43 AM

Hi,

Yes, you should check when was the file last modified. md5sum is not appropriate in this case. md5sum is used to check for file integrity for security purposes.

Try the stat command:

Code:

stat FILENAME

Then if use sed you can retrieve a particular line from there and check if the file was recently accessed or modified.

bsipple · 10-18-2016, 08:55 AM

Thanks everyone... Sorry for the delay in my response.. I did figure out what the issue was, with double checking the ls output:

Code:

for FILE in $(cat servers.txt); do
LAST=$(md5sum $FILE)

while true
do
   sleep 1
   NEW=$(md5sum "$FILE")
if [ "$NEW" != "$LAST" ]
then
		echo "Tasks RUNNING $FILE"
        LAST="$NEW"
		break
		fi
done
done

Is how it should have been written... It was def the "ls" issue

Habitual · 10-18-2016, 10:07 AM

Quote:

Originally Posted by bsipple

Is how it should have been written... It was def the "ls" issue

Then you'll love this

bsipple · 10-18-2016, 10:38 AM

Thank you for this!! I really appreciate the guidance.
This is extremely useful

Habitual · 10-18-2016, 12:57 PM

You are very welcome.

Have another: https://www.linuxquestions.org/quest...llected-35954/

grail · 10-18-2016, 01:07 PM

I am a little confused on how this script is supposed to work??

Firstly, if the file being checked is never touched this script will run forever?

Now if I am following correctly, you will 'break' from your infinite loop to go to the next server / FILE only if the files are not the same. To me this implies you would need to run the script again until they are the same, but in this case you will never leave the loop ... is this really the desired outcome?

szboardstretcher · 10-18-2016, 01:12 PM

Quote:

I am trying to check to see if 4 files listed in "files.txt" have tasks running on them.

If this is what you are trying to do,. then you will want to do this:

Code:

fuser /some/file/name

#example
fuser /var/lib/pgsql/data
/var/lib/pgsql/data:   567c 15641c 15642c 15644c 15645c 15646c 15647c 15648c

You can see that 8 processes are using the file 'data'

What you are asking for and what you are showing your script doing are two VERY different things.