LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Programming (http://www.linuxquestions.org/questions/programming-9/)
-   -   bash and delayed write / write is not in sync (http://www.linuxquestions.org/questions/programming-9/bash-and-delayed-write-write-is-not-in-sync-731576/)

Marko Hyvärinen 06-09-2009 01:11 AM

bash and delayed write / write is not in sync
 
Hi !

Couldn't find answer to my problem yet. Problem is that I use bash script which ping clients and put results to file, like ping xxx> database.file.

i run multiple processes at the same time for different ip-adresses. I have noticed that sometimes there can be strange situation while watching database (text file), for example with WATCH. My script writes first date to file, and after it ping response, which is filtered as 1 or 0

sometimes things happend like this:
26.05.09.10.25.37 1
26.05.09.10.25.53 1
26.05.09.10.26.08 1
26.05.09.10.26.23 1
26.05.09.10.26.39 1
26.05.09.10.26.54 1
26.05.09.10.27.09 1
26.05.09.10.27.14 1
26.05.09.10.27.30 1
26.05.09.10.27.45 1
26.05.09.10.28.01 26.05.09.10.28.02 1
1
26.05.09.10.28.16 1
26.05.09.10.28.32 1
26.05.09.10.28.47 1
26.05.09.10.29.03 1

reply is not written to file as should be ocasionally, can't see why..
then it can work even many hours without makind this error, couldn't figure out any repeating pattern for failure.

here is a script which starts pinger for all ip-adresses foung from file and should wait till all are finished:

Note: Some comment are in Finnish, maby programmes don't need my comments to understand what i'm looking after. Code is messy, i know and there is many testing thing inside still...
________________________________
#!/bin/bash

while (true) do

exec 3< iplista

while read <&3
do
./ping_ping $REPLY &
done
exec 3>&-

#Odota edellisiä ping_ping:ejä
maara=1
while [[ "$maara" -ne "0" ]] ;do
maara=`ps axf|grep -E "ping -c"|wc -l`
#echo -n "$maara "
#sleep 0.1
done

sleep 5
#sync
#blockdev --flushbufs /dev/hda
sleep 10
sync
touch -f reply/*
grep -i -E " 0" reply/*.ping>/dev/null

rm -rf tmp/*
cp -f reply/*.tilatieto tmp/
done
___________________________________________________

here is ping_ping:
#!/bin/bash

#NOLLATAAAN CACHE
ls -lR reply/>/dev/null
sync;sync

date +%d.%m.%y.%H.%M.%S" "|tr -d '\n'>>reply/$1.ping
PING_RESULT=`ping -c 3 -w 5 $1 | grep -E "bytes from"`
if [ -z "$PING_RESULT" ] ; then
echo "0">>reply/$1.ping
echo "0">reply/$1.tilatieto
else
echo "1">>reply/$1.ping
echo "1">reply/$1.tilatieto
fi

#NOLLATAAAN CACHE
ls -lR reply/>/dev/null
sync;sync
__________________________________________________________

Any help would be great !

Marko

noctilucent 06-09-2009 01:57 AM

Hello.

A quick look over your post would suggest that you have multiple processes running in parallel, all writing to the same file during their respective executions. If this is the case, you need to sync them [semaphore, or such].

Marko Hyvärinen 06-09-2009 03:40 AM

maby you missed this:

in pinger is echo "1">>reply/$1.ping

$1 gets ip from upper deamon.
___________________________________

There is upper deamon running which starts pinger for each ip-address and their results go for each own xxx.xxx.xxx.xxx.ping - file.

Or did i miss your point ?

Marko

ntubski 06-09-2009 12:46 PM

But you must have duplicates in your iplista file, since the ping_ping script only writes a single result at a time.

Code:

#Odota edellisiä ping_ping:ejä
maara=1
while [[ "$maara" -ne "0" ]] ;do
maara=`ps axf|grep -E "ping -c"|wc -l`
#echo -n "$maara "
#sleep 0.1
done

sleep 5
#sync
#blockdev --flushbufs /dev/hda
sleep 10

Is this code supposed to wait until ./ping_ping $REPLY & ends? The loop will exit after the ping commands finishes but before the result is written to the file. I see you put sleep commands after it to compensate, but this is not guaranteed to work. Perhaps this is the source of your problem. You could just do wait $! instead.

Although I don't really understand why you are running ping_ping asynchronously if you just end up waiting for it afterwards...

Marko Hyvärinen 06-10-2009 02:21 AM

Quote:

Originally Posted by ntubski (Post 3568193)
Is this code supposed to wait until ./ping_ping $REPLY & ends? The loop will exit after the ping commands finishes but before the result is written to the file. I see you put sleep commands after it to compensate, but this is not guaranteed to work. Perhaps this is the source of your problem. You could just do wait $! instead.

Although I don't really understand why you are running ping_ping asynchronously if you just end up waiting for it afterwards...

No, there was not duplicates in iplista-file, i checked ;)

Reason running parallel is that in short time, i can ping many targets, like 10 targets within ~5 sec are they up or down. Running pings serially mean longer time to see state of the target if targetcount is bigger. Because you have to ping target reliably, not just ping once to figure out its current state, ping can take even 5 sec + more delay before i get output to file. I have other deamon which sees each targets state in ip-address.tila - file.( Tila is state in finnish) and make alarm.

Any simple advise, how can i figure out that ping_ping output is fully written to each one's ping-file ?
I try that "wait $!", but if problem lies on buffering or in bash (piping), i can see that only way is in each ping_ping to make sure that ALL output is written fully, before exiting script. Maby that's best solution here ?

Here's some ps listings:

This is ps list when main deamon run all ping_pings:
valvonta:~ # ps axf|grep ping
6179 pts/0 S 0:00 \_ grep ping
31899 ? S 20:04 /bin/bash /valvonta/ping.deamon
6165 ? S 0:00 \_ /bin/bash ./ping_ping 192.168.1.1
6167 ? S 0:00 \_ /bin/bash ./ping_ping 192.168.1.33
6170 ? S 0:00 \_ /bin/bash ./ping_ping 192.168.1.34
6173 ? S 0:00 \_ /bin/bash ./ping_ping 192.168.1.60
6176 ? S 0:00 \_ /bin/bash ./ping_ping 192.168.1.65
6178 ? R 0:00 \_ /bin/bash ./ping_ping 192.168.1.97
6180 ? S 0:00 \_ /bin/bash ./ping_ping 194.251.244.241
6183 ? R 0:00 \_ /bin/bash /valvonta/ping.deamon
6185 ? S 0:00 \_ grep -E ping -c
6186 ? R 0:00 \_ /bin/bash /valvonta/ping.deamon
31105 ? S 11:05 /bin/bash /valvonta/ping.deamon

Little after that, list looks like this:
31105 ? S 11:05 /bin/bash /valvonta/ping.deamon
7004 ? S 0:00 \_ /bin/bash ./ping_ping 192.168.1.1
7075 ? S 0:00 | \_ /bin/bash ./ping_ping 192.168.1.1
7076 ? S 0:00 | \_ ping -c 3 -w 5 192.168.1.1
7006 ? S 0:00 \_ /bin/bash ./ping_ping 192.168.1.33
7066 ? S 0:00 | \_ /bin/bash ./ping_ping 192.168.1.33
7067 ? S 0:00 | \_ ping -c 3 -w 5 192.168.1.33
7007 ? S 0:00 \_ /bin/bash ./ping_ping 192.168.1.34
7046 ? S 0:00 | \_ /bin/bash ./ping_ping 192.168.1.34
7047 ? S 0:00 | \_ ping -c 3 -w 5 192.168.1.34
7010 ? S 0:00 \_ /bin/bash ./ping_ping 192.168.1.60
7051 ? S 0:00 | \_ /bin/bash ./ping_ping 192.168.1.60
7052 ? S 0:00 | \_ ping -c 3 -w 5 192.168.1.60
7013 ? S 0:00 \_ /bin/bash ./ping_ping 192.168.1.65
7088 ? S 0:00 | \_ /bin/bash ./ping_ping 192.168.1.65
7089 ? S 0:00 | \_ ping -c 3 -w 5 192.168.1.65
7015 ? S 0:00 \_ /bin/bash ./ping_ping 192.168.1.97
7061 ? S 0:00 | \_ /bin/bash ./ping_ping 192.168.1.97
7062 ? S 0:00 | \_ ping -c 3 -w 5 192.168.1.97
7016 ? S 0:00 \_ /bin/bash ./ping_ping 194.251.244.241
7083 ? S 0:00 | \_ /bin/bash ./ping_ping 194.251.244.241
7084 ? S 0:00 | \_ ping -c 3 -w 5 194.251.244.241

Guttorm 06-10-2009 04:30 AM

Hi

Not really sure what the problem is. But I don't think the "syncing" helps. It could help to make the appending to files "atomic" - I mean instad of this:

Code:

date +%d.%m.%y.%H.%M.%S" "|tr -d '\n'>>reply/$1.ping
PING_RESULT=`ping -c 3 -w 5 $1 | grep -E "bytes from"`
if [ -z "$PING_RESULT" ] ; then
echo "0">>reply/$1.ping
echo "0">reply/$1.tilatieto
else
echo "1">>reply/$1.ping
echo "1">reply/$1.tilatieto
fi

Code:

DATE=$(date +%d.%m.%y.%H.%M.%S" "|tr -d '\n')
PING_RESULT=`ping -c 3 -w 5 $1 | grep -E "bytes from"`
if [ -z "$PING_RESULT" ] ; then
echo "${DATE}0">>reply/$1.ping
echo "0">reply/$1.tilatieto
else
echo "${DATE}0">>reply/$1.ping
echo "1">reply/$1.tilatieto
fi

If the ping times out, there can be 5 seconds or more between the two appends to the files. It could be another process appends in between. At least that's what it looks like in the output.

Also, have you considered using nmap instead? It can ping lots of addresses at once (fast) and it has some nice options for how you want the output.

For example:

XML output:
nmap -sP -oX - $(cat iplista)

Greppable output:
nmap -sP -oG - $(cat iplista)

Marko Hyvärinen 06-10-2009 05:08 AM

Thanks, very good example !

We can say that i like "kiss", simple and existing things, which anyone can modify, and ping is "everywhere" and quite reliable. I have also nc-scripts to test pop. telnet etc, but as you said, nmap exist too.

I took freedom to "change" little and this is running now, let's see what happens but this works now, pls don't laugh, i'm not a coder, just get ideas... :

valvonta:/valvonta # cat ping_ping
#!/bin/bash

#if new ip, create file/s for it.
touch reply/$1.ping
touch reply/$1.tilatieto

#Get date and time
DATE=$(date +%d.%m.%Y.%H.%M.%S" "|tr -d '\n')

#Ping target
PING_RESULT=`ping -c 3 -w 5 $1 | grep -i -m 1 -E "bytes from"`

#Detect: is host on or off
if [ -z "$PING_RESULT" ] ; then
echo "${DATE}0">>reply/$1.ping
echo "0">reply/$1.tilatieto
else
echo "${DATE}1">>reply/$1.ping
echo "1">reply/$1.tilatieto
fi

#Just for making sure, wait all prosesses, does even need this ?
wait

#Final check that file is written actually, paranoid ?
#It is ok for me to hang if any fault in system.
WRITE_SUCCESS=`grep $DATE reply/$1.ping|grep -iE " 1| 0"`
while [ -z "$WRITE_SUCCESS" ];
do
:
done


Someone might use this for telnet, for example, but it makes log file to grow on server:
TELNET_RESULT=`echo ^c|netcat -w 1 $ip 23| grep '˙'`

ntubski 06-11-2009 06:00 PM

Quote:

Originally Posted by Marko Hyvärinen (Post 3568863)
No, there was not duplicates in iplista-file, i checked ;)

But then I don't understand how any of your files can have more than line in it, since ping_ping only writes a single result.

Marko Hyvärinen 06-12-2009 12:03 AM

Quote:

Originally Posted by ntubski (Post 3571037)
But then I don't understand how any of your files can have more than line in it, since ping_ping only writes a single result.

That is correct, but let me explain more, maby you understand after this or not...

I included earlier in this post both codes, first was ping.deamon's code, which seek trough host-list and start ping_ping for every ip-address it finds. And that is repeated forever, until kill etc. So like this:
31899 ? S 20:04 /bin/bash /valvonta/ping.deamon
6165 ? S 0:00 \_ /bin/bash ./ping_ping 192.168.1.1
6167 ? S 0:00 \_ /bin/bash ./ping_ping 192.168.1.33
6170 ? S 0:00 \_ /bin/bash ./ping_ping 192.168.1.34
6173 ? S 0:00 \_ /bin/bash ./ping_ping 192.168.1.60
6176 ? S 0:00 \_ /bin/bash ./ping_ping 192.168.1.65

after all ping_pings is little pause and same happens again and again. That's why every xxx.xxx.xxx.xxx.ping - file has "dd.mm.yy.mm.hh.ss 1" many times, like so:
26.05.09.10.25.37 1
26.05.09.10.25.53 1
26.05.09.10.26.08 1
26.05.09.10.26.23 1
26.05.09.10.26.39 1

Or then i just can't understand what YOU are after, language/nationality barrier eh?

bw,
marko

ntubski 06-12-2009 12:12 PM

Oops, you're right, I misread the script.

Quote:

#Final check that file is written actually, paranoid ?
This is not necessary, the reason for your problem is because you didn't wait for the ping_ping's to finish in the main script. It could run through iplista again before they are done, which results in multiple ping_ping's writing to the same file.

I think this would work:
Code:

while true ; do
  xargs -n 1 -P 0 ./ping_ping < iplista
done

xargs only finishes after it runs all commands.
Code:

man xargs
...
      --max-procs=max-procs, -P max-procs
              Run up to max-procs processes at a time; the default is  1.  If
              max-procs  is 0, xargs will run as many processes as possible at
              a time.
  Use the -n option with -P; otherwise chances  are  that
              only one exec will be done.



All times are GMT -5. The time now is 04:21 AM.