LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 10-10-2016, 11:05 AM   #1
bsipple
LQ Newbie
 
Registered: Oct 2016
Location: Deep in the heart of Texas
Distribution: CentOS 7.2
Posts: 3

Rep: Reputation: Disabled
Need help with a script to compare md5sums of 4 files in loop


Greetings,
I have searched through threads and online and cannot find anything that would help me in a specific way.

I am trying to check to see if 4 files listed in "files.txt" have tasks running on them.
Each file needs to be check for running tasks before moving on to the next file in the list.

example:
Code:
Checking file1
No change detected... checking file1
No change detected... checking file1
Tasks are running on file1 
No change detected... checking file2
No change detected... checking file2
No change detected... checking file2
No change detected... checking file2
Tasks are running on file2
[...]
I have been trying to use md5sum to do the check --which does seem to work, except that my script is not checking to completion before moving on to the next file in the list.

Tasks have to be running before the script moves on.
The script I have works perfectly for a single file, but when I add a for loop, it fails to actually check...

Here is the script that works for a single file:

Code:
filename="/path/to/myfiles/file1"
lastfile=$(ls -l $filename)
while true
do
  sleep 1
     newfile=$(md5sum "$filename")
     if [ "$newfile" != "$lastfile" ]
     then
     echo "Tasks are running on $filename"
      lastfile="$newfile"
      break
else 
 echo "Checking for running tasks on $filename"
fi
done
I need to be able to go through files listed in files.txt...

Which looks like this:
Code:
/path/to/myfiles/file1
/path/to/myfiles/file2
/path/to/myfiles/file3
/path/to/myfiles/file4
Here is the script for the loop:
Code:
for filename in $(cat files.txt); do
lastfile=`ls -l $filename`
while true
echo "Checking $filename"
do
	sleep 1  
	newfile=`md5sum "$filename"`
	if [ "$newfile" != "$lastfile" ]
then
    echo "Tasks are running on $filename"
    lastfile="$newfile"
    break
    else 
    echo "No change detected... $filename"
fi
done
done
The output I get is below:
Code:
-rw-rw-r-- 1 www www 1230 Oct 10 09:26 /path/to/myfiles/file1
1c05ebb1488d5dbfbcc0d34c2f7730d9 /path/to/myfiles/file1
Tasks Running /path/to/myfiles/file1
-rw-r--r-- 1 www www 1230 Oct 10 09:26 /path/to/myfiles/file2
c61d58de60268dc87194a3f76951b9cb /path/to/myfiles/file2
Tasks Running /path/to/myfiles/file2
-rw-r--r-- 1 www www 1229 Oct 10 09:26 /path/to/myfiles/file3
2204ee4bb052ccb13bb489a18a5962af /path/to/myfiles/file3
Tasks Running /path/to/myfiles/file3
-rw-r--r-- 1 www www 1229 Oct 10 09:26 /path/to/myfiles/file14
4f714eb328b7784b8485ce8a2538825b /path/to/myfiles/file4
Tasks Running /path/to/myfiles/file4
I have turned off updates to these files to see if the script is even working, and it is not...

I could really use some help with making this work, or... if someone knows a better way to do this -- Please let me know

Note -- I have no access to install software -- so I cannot add a 3rd party software to do this task... --I Wish--

Thanks in advance
 
Old 10-10-2016, 05:05 PM   #2
keefaz
LQ Guru
 
Registered: Mar 2004
Distribution: Slackware
Posts: 5,393

Rep: Reputation: 409Reputation: 409Reputation: 409Reputation: 409Reputation: 409
You need to revise your logic, comparing md5 checksum with ls -l output will lead to nowhere useful

First store each file checksum somewhere, then compare new checksum with old checksum
 
2 members found this post helpful.
Old 10-10-2016, 06:01 PM   #3
michaelk
Moderator
 
Registered: Aug 2002
Posts: 16,316

Rep: Reputation: 1905Reputation: 1905Reputation: 1905Reputation: 1905Reputation: 1905Reputation: 1905Reputation: 1905Reputation: 1905Reputation: 1905Reputation: 1905Reputation: 1905
Welcome to LinuxQuestions.

It appears that you are checking to see if the output of the ls command changes. If some process is writing to the file then the access time and size will change.

You can use the command set -x at the top of the script to see how your script runs and outputs a trace of the commands and arguments to help with debugging.

By the way you have the same problem in your first script but since the output does not match the script we can not compare the two. Since the initial values are different your logic fails.

lastfile = -rw-rw-r-- 1 www www 1230 Oct 10 09:26 /path/to/myfiles/file1
and
newfile = 1c05ebb1488d5dbfbcc0d34c2f7730d9 /path/to/myfiles/file1

Last edited by michaelk; 10-10-2016 at 06:03 PM.
 
1 members found this post helpful.
Old 10-11-2016, 04:43 AM   #4
aragorn2101
Member
 
Registered: Dec 2012
Location: Mauritius
Distribution: Slackware
Posts: 509

Rep: Reputation: 241Reputation: 241Reputation: 241
Hi,

Yes, you should check when was the file last modified. md5sum is not appropriate in this case. md5sum is used to check for file integrity for security purposes.

Try the stat command:
Code:
stat FILENAME
Then if use sed you can retrieve a particular line from there and check if the file was recently accessed or modified.
 
Old 10-18-2016, 09:55 AM   #5
bsipple
LQ Newbie
 
Registered: Oct 2016
Location: Deep in the heart of Texas
Distribution: CentOS 7.2
Posts: 3

Original Poster
Rep: Reputation: Disabled
Thanks everyone... Sorry for the delay in my response.. I did figure out what the issue was, with double checking the ls output:

Code:
for FILE in $(cat servers.txt); do
LAST=$(md5sum $FILE)

while true
do
   sleep 1
   NEW=$(md5sum "$FILE")
if [ "$NEW" != "$LAST" ]
then
		echo "Tasks RUNNING $FILE"
        LAST="$NEW"
		break
		fi
done
done
Is how it should have been written... It was def the "ls" issue

Last edited by bsipple; 10-18-2016 at 09:57 AM. Reason: used incorrect code tag
 
Old 10-18-2016, 11:07 AM   #6
Habitual
LQ 5k Club
 
Registered: Jan 2011
Location: Nowhere near you, thank God.
Distribution: OSX Sierra
Posts: 8,573
Blog Entries: 14

Rep: Reputation: Disabled
Quote:
Originally Posted by bsipple View Post
Is how it should have been written... It was def the "ls" issue
Then you'll love this
 
1 members found this post helpful.
Old 10-18-2016, 11:38 AM   #7
bsipple
LQ Newbie
 
Registered: Oct 2016
Location: Deep in the heart of Texas
Distribution: CentOS 7.2
Posts: 3

Original Poster
Rep: Reputation: Disabled
Thank you for the guidance

Thank you for this!! I really appreciate the guidance.
This is extremely useful
 
Old 10-18-2016, 01:57 PM   #8
Habitual
LQ 5k Club
 
Registered: Jan 2011
Location: Nowhere near you, thank God.
Distribution: OSX Sierra
Posts: 8,573
Blog Entries: 14

Rep: Reputation: Disabled
You are very welcome.

Have another: https://www.linuxquestions.org/quest...llected-35954/
 
Old 10-18-2016, 02:07 PM   #9
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,550

Rep: Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898
I am a little confused on how this script is supposed to work??

Firstly, if the file being checked is never touched this script will run forever?

Now if I am following correctly, you will 'break' from your infinite loop to go to the next server / FILE only if the files are not the same. To me this implies you would need to run the script again until they are the same, but in this case you will never leave the loop ... is this really the desired outcome?
 
Old 10-18-2016, 02:12 PM   #10
szboardstretcher
Senior Member
 
Registered: Aug 2006
Location: Detroit, MI
Distribution: GNU/Linux systemd
Posts: 4,211

Rep: Reputation: 1612Reputation: 1612Reputation: 1612Reputation: 1612Reputation: 1612Reputation: 1612Reputation: 1612Reputation: 1612Reputation: 1612Reputation: 1612Reputation: 1612
Quote:
I am trying to check to see if 4 files listed in "files.txt" have tasks running on them.
If this is what you are trying to do,. then you will want to do this:
Code:
fuser /some/file/name

#example
fuser /var/lib/pgsql/data
/var/lib/pgsql/data:   567c 15641c 15642c 15644c 15645c 15646c 15647c 15648c
You can see that 8 processes are using the file 'data'

What you are asking for and what you are showing your script doing are two VERY different things.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
NEWBIE: loop through and compare two files? hristo77 Programming 4 11-21-2015 04:22 AM
I need one shell script to compare two files. kathirvel Linux - Newbie 4 01-07-2014 10:19 AM
[SOLVED] Compare 2 files with script. elalexluna83 Programming 4 09-25-2012 07:50 PM
script to compare users in files s_linux Programming 7 04-05-2011 11:34 AM
How do I compare md5sums of directories (and of course everything in it)? natalinasmpf General 1 12-08-2003 08:03 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 01:58 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration