LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Programming (http://www.linuxquestions.org/questions/programming-9/)
-   -   Why doe sed fail to produce any output when called within this function? (http://www.linuxquestions.org/questions/programming-9/why-doe-sed-fail-to-produce-any-output-when-called-within-this-function-4175432094/)

PTrenholme 10-13-2012 08:02 PM

Why doe sed fail to produce any output when called within this function?
 
I wrote this function:
Code:

# Conky helper function to
# 1) Verify that the user has configured $1 in ~/conkyrc and return if not configured
# 2) Discard any lines in /tmp/.$USER.tmp not starting with "${1}: "
# 3) Remove the "$1: " from the start of each line so labeled in /tmp/.${USER}.conky.tmp
# 4) Replace any _ characters in the remaining line by blanks
# 5) Replace any # characters in the line by | characters
# 6) If any other arguments are present, pipe the result of steps 2-5 to them.
function conky.parse.run_or_warn()
{
  local label
  label=${1}
  shift
  if [ -z "$(sed -n "/^${label}: /p" ~/.conkyrc)" ]
  then
      cat <<EOF
No uncommented ${label} entries were found in your ~/.conkyrc file.
       
       
       
       
EOF
      return
  fi
  if [ $# > 0 ]
  then
      sed -nr "/^${label}: /{s/${label}: //;s/_/ /g;s/#/|/g;p}" /tmp/.$USER.conky.tmp | $*
  else
      sed -nr "/^${label}: /{s/${label}: //;s/_/ /g;s/#/|/g;p}" /tmp/.$USER.conky.tmp
  fi
  echo
}

and used it like this:
Code:

#!/bin/bash

# wminfo plugin: current entropy and the size of system entropy pool

# command: wminfo -p conky.entropy.wmi -u 6 -b#000000 -f#ff0000
. conky.parse.run_or_warn
conky.parse.run_or_warn ENTROPY
echo ------
datafile="/tmp/.$USER.conky.tmp"

sed -rn '/^ENTROPY: /{s/ENTROPY: //;s/_/ /g;s/#/|/g;p}' $datafile

echo

and I got this output:
Code:

$ ./conky.entropy.wmi

------
ENTROPY
SIZ: 4096
AVL: 184
4%

When I ran it with the v and x options on, I got (in the relevant section) this:
Code:

+ sed -nr '/^ENTROPY: /{s/ENTROPY: //;s/_/ /g;s/#/|/g;p}' /tmp/.Peter.conky.tmp
+ echo

echo ------
+ echo ------
------
datafile="/tmp/.$USER.conky.tmp"
+ datafile=/tmp/.Peter.conky.tmp

sed -rn '/^ENTROPY: /{s/ENTROPY: //;s/_/ /g;s/#/|/g;p}' $datafile
+ sed -rn '/^ENTROPY: /{s/ENTROPY: //;s/_/ /g;s/#/|/g;p}' /tmp/.Peter.conky.tmp
ENTROPY
SIZ: 4096
AVL: 184
4%
         

echo
+ echo

So I copied the two lines from the bash -xv output and ran them:
Code:

$ sed -rn '/^ENTROPY: /{s/ENTROPY: //;s/_/ /g;s/#/|/g;p}' /tmp/.Peter.conky.tmp
ENTROPY
SIZ: 4096
AVL: 184
4%
         
$ sed -nr '/^ENTROPY: /{s/ENTROPY: //;s/_/ /g;s/#/|/g;p}' /tmp/.Peter.conky.tmp
ENTROPY
SIZ: 4096
AVL: 184
4%

So, can anyone explain why the sed inside the function writes nothing to stdout, but the same command from the terminal prompt works fine? :scratch:

Oh, here's the part of the data file that should be extracted and printed.
Code:

FILESYSTEM1: rootfs
FILESYSTEM1: USE 64.4G
FILESYSTEM1: 13%
FILESYSTEM1: FRE 392G
FILESYSTEM1: 81%
ENTROPY: ENTROPY
ENTROPY: SIZ: 4096
ENTROPY: AVL: 184
ENTROPY: 4%
ENTROPY: __________
TOP:  22.33 X             
TOP:  3.47 kwin           
TOP:  0.87 kworker/1:0   
TOP:  0.50 kworker/0:0   
TOP:  0.37 kworker/3:0


firstfire 10-13-2012 11:05 PM

Hi.

Change [ $# > 0 ] to [[ $# > 0 ]] or (( $# > 0 )). Or quote the 'greater than' character, like this: [ $# '>' 0 ]. For some reason this condition in your code always evaluates to true, so sed output is piped to ... well, I don't know where. Anyway you do not see any output. There are no warning or error message probably because this situation happens at run time, not at the time of interpreting function definition (this may or may not be true depending on bash internals). I'd say it is a bug in bash, as piping with empty right hand side is an error.

NevemTeve 10-13-2012 11:11 PM

For example:

Code:

old: if [ -z "$(sed -n "/^${label}: /p" ~/.conkyrc)" ]
new: if [ -z "$(sed -n \"/^${label}: /p\" ~/.conkyrc)" ]


grail 10-14-2012 05:00 AM

Code:

if [ -z "$(sed -n "/^${label}: /p" ~/.conkyrc)" ]
  then
      cat <<EOF
No uncommented ${label} entries were found in your ~/.conkyrc file.
       
EOF
      return
fi

# Could simply be
if ! grep -q "^${label}: " ~/.conkyrc
then
...

firstfire's advice is correct that you should use (()) when testing numbers. The alternative when using [ or [[ is to use -gt which equals >.

I am also not clear on why you ran the same sed code twice as an example? Clearly it would deliver the same output.

I also guess that there must be other lines in your input file that actually have underscores (_) or hashes (#) in them as your example data would not seem to need the rest of the sed:
Code:

sed -nr "/^${label}: /{s/${label}: //;s/_/ /g;s/#/|/g;p}" /tmp/.$USER.conky.tmp
(portion in red is never used)

Lastly, are you able to explain what the following is supposed to do??
Code:

sed -nr "/^${label}: /{s/${label}: //;s/_/ /g;s/#/|/g;p}" /tmp/.$USER.conky.tmp | $*
Specifically the part in red?

PTrenholme 10-14-2012 10:54 AM

O.K.: Solved. I changed the if [ $# > 0 ] to if [ $# -gt 0 ] and now it works. It's the little things that cause problems, eh? (In my defense, I did know better, but once my fingers did the deed, my eyes could not see it. Thanks to everyone.:))

@grail: The pipe into $* is to allow the function user an ability to pass additional arguments into which the result of the transformation is to be piped. In the context of the application of which this is intended to be a part (wminfo), that is usually a simple character set transformation program, but I thought the additional flexibility might be useful. As to the "unneeded" transformations you flagged, that's only true of the sample data I posted.

Also, yes the use of grep in the first test would be just as easy. (That was, in fact, what was being done in the scripts I'm trying to improve.) In the current context, many scripts using this function may be invoked every few seconds, so I thought that the system was more likely to find a needed program in the cached file list if I minimized the number of different programs used by the system. Since grep and sed can both accomplish the same "is it in the file" task, but grep can't transform its input stream, I thought it would be better to abandon grep in the scripts.

grail 10-14-2012 12:06 PM

Quote:

Since grep and sed can both accomplish the same "is it in the file" task
Depending on your view point the above is not exactly correct at least not in the 'how'.

sed may be used to return a value (or not) from a file but then your test of '-z' or similar is required to confirm that there was (or was not) any output returned.

In grep's case, not only does it have the option to return what is has found but it also sets an exit code, hence why it can be used directly by 'if' which tests for exit codes (or rather
the truth or false answer of an expression).

PTrenholme 10-15-2012 06:00 PM

Quote:

Originally Posted by grail (Post 4805476)
...sed may be used to return a value (or not) from a file but then your test of '-z' or similar is required to confirm that there was (or was not) any output returned...

That's quite correct.

As I pointed out, I was trying to minimize the number of different programs invoked by the scripts using that code in the hope that physical file accesses would be minimized. (The wminfo program creates a five line, nine character text windows and periodically calls scripts to generate text to be displayed in the windows. Logically this amounts to creating a new process every few seconds so the window can be redrawn. Minimizing the overhead of this system is a "good thing," eh?)

I haven't yet done any timing comparisons, but I'm hoping that only using sed rather than grep + sed might yield at least a slight improvement.

However I welcome any comments about my strategy for trying to speed things up, and any alternatives of which you might think. (As an alternative to bash, I've been considering seeing if these scripts could be converted to lua scripts. But I first wanted to "squeeze" as much out of bash as I could.)

grail 10-16-2012 02:19 AM

Well I see nothing in your code that would stop you from writing it all as an awk script.

Obviously lua, perl, python or ruby would all be good choices too :)

ntubski 10-16-2012 02:37 AM

Quote:

Originally Posted by PTrenholme (Post 4806550)
As I pointed out, I was trying to minimize the number of different programs invoked by the scripts using that code in the hope that physical file accesses would be minimized.

In an age of multi GB RAMs, trying to save a few hundred kB of disk cache isn't worth the trouble. grep is so common that it will probably already be in cache anyway.

PTrenholme 10-16-2012 08:00 PM

Quote:

Originally Posted by ntubski (Post 4806823)
In an age of multi GB RAMs, trying to save a few hundred kB of disk cache isn't worth the trouble. grep is so common that it will probably already be in cache anyway.

One of my boxes is an old desktop with all of 512Mb memory and a real Pentiun II processor. It runs Victor Linux (and Gentoo, and a few others), but its old ATA hard drives are fairly slow. Any disk caching is to be avoided.

So, yes, most systems have multi-GB RM, etc., but not all. However, the wminfo system is targeted to people using The Window Manager as their windowing system. Those are not, I think, too likely to be people with systems that have modern resources.


All times are GMT -5. The time now is 04:13 PM.