bash scripting sending "cat: write error: Broken pipe,"

rojoblandino · 04-19-2010, 05:39 PM

I move a page from a server in Suse to Debian and this script was working well but when i move the page to a Debian base server it is sending in the logs the following message:

cat: write error: Broken pipe, referer: http://domain.example/cgi-bin/search.sh

the cgi does the following, read a db file plain text that repeat each 15 lines an information where to look the seraching info. if it is there sends to the output the 15 lines block. but it was not sending errors in suse distro, does anybody knows why the cat's line are sending "cat: write error: Broken pipe," if i am not writing or making a pipe with ">" to none file.

Quote:

np=`echo $POST_DATA | wc -c`
for data in `ls /srv/virtual/wfolder/$seccion/db`;do
seccion_search=`echo $data | awk -F/ '{print $5}'`
n=`cat $data | wc -l`
#Look the first line is o not empty
fl=`cat $data | head -1 | wc -w`
if let "fl==0";then
ini=15
else
ini=14
fi
echo "<tr bgcolor=\"#CCCC99\"><td><A NAME=\""`echo $data | awk -F/ '{print $5}' `"|region\">"$seccion_search"</td></tr>"
end=""
if let "np>1"; then
for (( i=0;i<=$n; ))
do
ishead=`cat $data | head -$i | tail -1 | grep \<\!--begin | wc -w`
let $j "j=i+1"
iddate=`cat $data | head -$j | tail -1 | awk '{print $2}' | awk -F"-" '{print $1}'`
if let "ishead>0";then
let $i "i+=13"
content=`cat $data | head -$i | tail -14 `
iffound=`echo $content | grep $browser | wc -l`
if let "iffound>0";then
echo "<tr><td>"
datline=`cat $data | head -$i | tail -14`
title=`echo $datline | sed "s/\//._./g"`
sectitle=`echo $title | awk -F">" '{print $8}' | awk -F"<" '{print $1}' `
title1=`echo $title | sed "s/>${sectitle}</><a href=\"http:\/\/domain.example\/$seccion_search\/index.html#$seccion_search|$iddate\">$sectitle<._.a></g"`
echo $title1 | sed "s/._./\//g"

#cat $data | head -$i | tail -14
echo "<br><A HREF=\"#home|region\"><b>Regresar al Inicio</A></td></tr>"
fi
fi
let $i "i+=1"
done
fi
done

kbp · 04-19-2010, 08:41 PM

What are the permissions on this directory and the files inside it?: '/srv/virtual/wfolder/$seccion/db'

rojoblandino · 04-20-2010, 01:06 PM

Quote:

Originally Posted by kbp

What are the permissions on this directory and the files inside it?: '/srv/virtual/wfolder/$seccion/db'

i have a user wwwu and a group wwwg for web, in apache it is configurated to work so.

for db it has 775 so wwwu and wwwg has permissions for read and write but others only read and execute.

another thing the script does not write in the db file it just read information from it and send the info to the output in this case the browser.

What i am doing is searching not editing or writing on the db files

kbp · 04-20-2010, 07:24 PM

You are chaining commands together several times, eg. 'cat $data | wc -l'. You may need to put in some 'echo' lines to provide output to the browser or even to a log file, so you can see which command it's failing on, like:

'echo "attempting to run command X"'

catkin · 04-21-2010, 03:26 AM

In case anyone wants to understand and discuss the code, it might be easier like this

Code:

  1 np=`echo $POST_DATA | wc -c`
  2 for data in `ls /srv/virtual/wfolder/$seccion/db`
  3 do
  4     seccion_search=`echo $data | awk -F/ '{print $5}'`
  5     n=`cat $data | wc -l`
  6     #Look the first line is o not empty
  7     fl=`cat $data | head -1 | wc -w`
  8     if let "fl==0";then
  9         ini=15
 10     else
 11         ini=14
 12     fi
 13     echo "<tr bgcolor=\"#CCCC99\"><td><A NAME=\""`echo $data | awk -F/ '{print $5}' `"|region\">"$seccion_search"</td></tr>"
 14     end=""
 15     if let "np>1"; then
 16         for (( i=0;i<=$n; ))
 17         do
 18             ishead=`cat $data | head -$i | tail -1 | grep \<\!--begin | wc -w`
 19             let $j "j=i+1"
 20             iddate=`cat $data | head -$j | tail -1 | awk '{print $2}' | awk -F"-" '{print $1}'`
 21             if let "ishead>0";then
 22                 let $i "i+=13"
 23                 content=`cat $data | head -$i | tail -14 `
 24                 iffound=`echo $content | grep $browser | wc -l`
 25                 if let "iffound>0";then
 26                     echo "<tr><td>"
 27                     datline=`cat $data | head -$i | tail -14`
 28                     title=`echo $datline | sed "s/\//._./g"`
 29                     sectitle=`echo $title | awk -F">" '{print $8}' | awk -F"<" '{print $1}' `                                                            30                     title1=`echo $title | sed "s/>${sectitle}</><a href=\"http:\/\/domain.example\/$seccion_search\/index.html#$seccion_search|$iddate\"    >$sectitle<._.a></g"`
 31                     echo $title1 | sed "s/._./\//g"
 32                     echo "<br><A HREF=\"#home|region\"><b>Regresar al Inicio</A></td></tr>"
 33                 fi
 34             fi
 35             let $i "i+=1"
 36         done
 37     fi
 38 done

AIUI "cat: write error: Broken pipe" means that cat was attempting to write stdout to the stdin of another process but the other process terminated. This will happen for every one of the cat <some file> | head -<n> commands when there are more than <n> lines in <some file>. The crude solution would be to discard the error messages, for example by

Code:

fl=`cat $data 2>/dev/null | head -1 | wc -w`

but it is a crude solution because it makes error detection and reporting impossible. Maybe that's OK; the script has no error detection and reporting; it's an example of the "optimistic" school of scripting, easy to write but hard to support.

bakdong · 04-21-2010, 03:32 AM

Also, unless I'm very much mistaken, lots of unnecessary 'cat's ....

catkin · 04-21-2010, 03:52 AM

Quote:

Originally Posted by bakdong

Also, unless I'm very much mistaken, lots of unnecessary 'cat's ....

The script does not embody "best practice" in a few ways; it would be great material for a scripting seminar.

Regards the original question, it could have been readily investigated by that powerful scripting technique of trying individual commands at the command prompt to familiarise with their behaviour.

rojoblandino · 04-23-2010, 10:19 AM

Thanks catkin, a lot! You are right, is not right without error and detection and reporting capabilities, well when i wrote this code i did it for my self so i did not care so much for handling errors detection or reporting, i have work with scripting for standard output but it is the first time that i write a script as a cgi, so that i am starting and testing it, i got surprised when i change the distro and i did not know why the meaning of that error, now i know and i will look how to send the output of a file without a cat, i hope not switching to perl at the end, i would like to do it with bash,sh or ksh.

well i am doing some changes to the code like:
5 n=`cat $data | wc -l`
5 n=`wc -l $data | awk '{print $1}'`

i have erase the cat's, ending the script as below:

Code:

  1 np=`echo $POST_DATA | wc -c`
  2 for data in `ls /srv/virtual/wfolder/$seccion/db`
  3 do
  4     seccion_search=`echo $data | awk -F/ '{print $5}'`
  5     n=`wc -l $data | awk '{print $1}'`
  6     #Look the first line is o not empty
  7     fl=`head -1 $data | wc -w`
  8     if let "fl==0";then
  9         ini=15
 10     else
 11         ini=14
 12     fi
 13     echo "<tr bgcolor=\"#CCCC99\"><td><A NAME=\""`echo $data | awk -F/ '{print $5}' `"|region\">"$seccion_search"</td></tr>"
 14     end=""
 15     if let "np>1"; then
 16         for (( i=0;i<=$n; ))
 17         do
 18             ishead=`head -$i $data | tail -1 | grep \<\!--begin | wc -w`
 19             let $j "j=i+1"
 20             iddate=`head -$j $data | tail -1 | awk '{print $2}' | awk -F"-" '{print $1}'`
 21             if let "ishead>0";then
 22                 let $i "i+=13"
 23                 content=`head -$i $data | tail -14 `
 24                 iffound=`echo $content | grep $browser | wc -l`
 25                 if let "iffound>0";then
 26                     echo "<tr><td>"
 27                     datline=`head -$i $data | tail -14`
 28                     title=`echo $datline | sed "s/\//._./g"`
 29                     sectitle=`echo $title | awk -F">" '{print $8}' | awk -F"<" '{print $1}' `                                                            30                     title1=`echo $title | sed "s/>${sectitle}</><a href=\"http:\/\/domain.example\/$seccion_search\/index.html#$seccion_search|$iddate\"    >$sectitle<._.a></g"`
 31                     echo $title1 | sed "s/._./\//g"
 32                     echo "<br><A HREF=\"#home|region\"><b>Regresar al Inicio</A></td></tr>"
 33                 fi
 34             fi
 35             let $i "i+=1"
 36         done
 37     fi
 38 done

no cat's any more, and it stop the message in the log!

greetings and thanks!

catkin · 04-23-2010, 10:23 AM

Glad you solved it. The script is looking a lot tighter -- nice evolution