LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 10-13-2012, 08:02 PM   #1
PTrenholme
Senior Member
 
Registered: Dec 2004
Location: Olympia, WA, USA
Distribution: Fedora, (K)Ubuntu
Posts: 4,147

Rep: Reputation: 330Reputation: 330Reputation: 330Reputation: 330
Why doe sed fail to produce any output when called within this function?


I wrote this function:
Code:
# Conky helper function to
# 1) Verify that the user has configured $1 in ~/conkyrc and return if not configured
# 2) Discard any lines in /tmp/.$USER.tmp not starting with "${1}: "
# 3) Remove the "$1: " from the start of each line so labeled in /tmp/.${USER}.conky.tmp
# 4) Replace any _ characters in the remaining line by blanks
# 5) Replace any # characters in the line by | characters
# 6) If any other arguments are present, pipe the result of steps 2-5 to them.
function conky.parse.run_or_warn()
{
  local label
  label=${1}
  shift
  if [ -z "$(sed -n "/^${label}: /p" ~/.conkyrc)" ]
  then
      cat <<EOF
No uncommented ${label} entries were found in your ~/.conkyrc file.
         
         
         
         
EOF
      return
  fi
  if [ $# > 0 ]
  then
      sed -nr "/^${label}: /{s/${label}: //;s/_/ /g;s/#/|/g;p}" /tmp/.$USER.conky.tmp | $*
  else
      sed -nr "/^${label}: /{s/${label}: //;s/_/ /g;s/#/|/g;p}" /tmp/.$USER.conky.tmp
  fi
  echo
}
and used it like this:
Code:
#!/bin/bash

# wminfo plugin: current entropy and the size of system entropy pool

# command: wminfo -p conky.entropy.wmi -u 6 -b#000000 -f#ff0000
. conky.parse.run_or_warn
conky.parse.run_or_warn ENTROPY
echo ------
datafile="/tmp/.$USER.conky.tmp"

sed -rn '/^ENTROPY: /{s/ENTROPY: //;s/_/ /g;s/#/|/g;p}' $datafile

echo
and I got this output:
Code:
$ ./conky.entropy.wmi

------
ENTROPY
SIZ: 4096
AVL: 184
4%
When I ran it with the v and x options on, I got (in the relevant section) this:
Code:
+ sed -nr '/^ENTROPY: /{s/ENTROPY: //;s/_/ /g;s/#/|/g;p}' /tmp/.Peter.conky.tmp
+ echo

echo ------
+ echo ------
------
datafile="/tmp/.$USER.conky.tmp"
+ datafile=/tmp/.Peter.conky.tmp

sed -rn '/^ENTROPY: /{s/ENTROPY: //;s/_/ /g;s/#/|/g;p}' $datafile
+ sed -rn '/^ENTROPY: /{s/ENTROPY: //;s/_/ /g;s/#/|/g;p}' /tmp/.Peter.conky.tmp
ENTROPY
SIZ: 4096
AVL: 184
4%
          

echo
+ echo
So I copied the two lines from the bash -xv output and ran them:
Code:
$ sed -rn '/^ENTROPY: /{s/ENTROPY: //;s/_/ /g;s/#/|/g;p}' /tmp/.Peter.conky.tmp
ENTROPY
SIZ: 4096
AVL: 184
4%
          
$ sed -nr '/^ENTROPY: /{s/ENTROPY: //;s/_/ /g;s/#/|/g;p}' /tmp/.Peter.conky.tmp
ENTROPY
SIZ: 4096
AVL: 184
4%
So, can anyone explain why the sed inside the function writes nothing to stdout, but the same command from the terminal prompt works fine?

Oh, here's the part of the data file that should be extracted and printed.
Code:
FILESYSTEM1: rootfs
FILESYSTEM1: USE 64.4G
FILESYSTEM1: 13%
FILESYSTEM1: FRE 392G
FILESYSTEM1: 81%
ENTROPY: ENTROPY
ENTROPY: SIZ: 4096
ENTROPY: AVL: 184
ENTROPY: 4%
ENTROPY: __________
TOP:  22.33 X               
TOP:   3.47 kwin            
TOP:   0.87 kworker/1:0     
TOP:   0.50 kworker/0:0     
TOP:   0.37 kworker/3:0
 
Old 10-13-2012, 11:05 PM   #2
firstfire
Member
 
Registered: Mar 2006
Location: Ekaterinburg, Russia
Distribution: Debian, Ubuntu
Posts: 615

Rep: Reputation: 358Reputation: 358Reputation: 358Reputation: 358
Hi.

Change [ $# > 0 ] to [[ $# > 0 ]] or (( $# > 0 )). Or quote the 'greater than' character, like this: [ $# '>' 0 ]. For some reason this condition in your code always evaluates to true, so sed output is piped to ... well, I don't know where. Anyway you do not see any output. There are no warning or error message probably because this situation happens at run time, not at the time of interpreting function definition (this may or may not be true depending on bash internals). I'd say it is a bug in bash, as piping with empty right hand side is an error.

Last edited by firstfire; 10-14-2012 at 12:20 AM.
 
Old 10-13-2012, 11:11 PM   #3
NevemTeve
Senior Member
 
Registered: Oct 2011
Location: Budapest
Distribution: Debian/GNU/Linux, AIX
Posts: 1,571

Rep: Reputation: 471Reputation: 471Reputation: 471Reputation: 471Reputation: 471
For example:

Code:
old: if [ -z "$(sed -n "/^${label}: /p" ~/.conkyrc)" ]
new: if [ -z "$(sed -n \"/^${label}: /p\" ~/.conkyrc)" ]
 
Old 10-14-2012, 05:00 AM   #4
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,412

Rep: Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874
Code:
if [ -z "$(sed -n "/^${label}: /p" ~/.conkyrc)" ]
  then
      cat <<EOF
No uncommented ${label} entries were found in your ~/.conkyrc file.
         
EOF
      return
fi

# Could simply be
if ! grep -q "^${label}: " ~/.conkyrc
then
...
firstfire's advice is correct that you should use (()) when testing numbers. The alternative when using [ or [[ is to use -gt which equals >.

I am also not clear on why you ran the same sed code twice as an example? Clearly it would deliver the same output.

I also guess that there must be other lines in your input file that actually have underscores (_) or hashes (#) in them as your example data would not seem to need the rest of the sed:
Code:
sed -nr "/^${label}: /{s/${label}: //;s/_/ /g;s/#/|/g;p}" /tmp/.$USER.conky.tmp
(portion in red is never used)

Lastly, are you able to explain what the following is supposed to do??
Code:
sed -nr "/^${label}: /{s/${label}: //;s/_/ /g;s/#/|/g;p}" /tmp/.$USER.conky.tmp | $*
Specifically the part in red?
 
Old 10-14-2012, 10:54 AM   #5
PTrenholme
Senior Member
 
Registered: Dec 2004
Location: Olympia, WA, USA
Distribution: Fedora, (K)Ubuntu
Posts: 4,147

Original Poster
Rep: Reputation: 330Reputation: 330Reputation: 330Reputation: 330
O.K.: Solved. I changed the if [ $# > 0 ] to if [ $# -gt 0 ] and now it works. It's the little things that cause problems, eh? (In my defense, I did know better, but once my fingers did the deed, my eyes could not see it. Thanks to everyone.)

@grail: The pipe into $* is to allow the function user an ability to pass additional arguments into which the result of the transformation is to be piped. In the context of the application of which this is intended to be a part (wminfo), that is usually a simple character set transformation program, but I thought the additional flexibility might be useful. As to the "unneeded" transformations you flagged, that's only true of the sample data I posted.

Also, yes the use of grep in the first test would be just as easy. (That was, in fact, what was being done in the scripts I'm trying to improve.) In the current context, many scripts using this function may be invoked every few seconds, so I thought that the system was more likely to find a needed program in the cached file list if I minimized the number of different programs used by the system. Since grep and sed can both accomplish the same "is it in the file" task, but grep can't transform its input stream, I thought it would be better to abandon grep in the scripts.
 
Old 10-14-2012, 12:06 PM   #6
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,412

Rep: Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874
Quote:
Since grep and sed can both accomplish the same "is it in the file" task
Depending on your view point the above is not exactly correct at least not in the 'how'.

sed may be used to return a value (or not) from a file but then your test of '-z' or similar is required to confirm that there was (or was not) any output returned.

In grep's case, not only does it have the option to return what is has found but it also sets an exit code, hence why it can be used directly by 'if' which tests for exit codes (or rather
the truth or false answer of an expression).
 
Old 10-15-2012, 06:00 PM   #7
PTrenholme
Senior Member
 
Registered: Dec 2004
Location: Olympia, WA, USA
Distribution: Fedora, (K)Ubuntu
Posts: 4,147

Original Poster
Rep: Reputation: 330Reputation: 330Reputation: 330Reputation: 330
Quote:
Originally Posted by grail View Post
...sed may be used to return a value (or not) from a file but then your test of '-z' or similar is required to confirm that there was (or was not) any output returned...
That's quite correct.

As I pointed out, I was trying to minimize the number of different programs invoked by the scripts using that code in the hope that physical file accesses would be minimized. (The wminfo program creates a five line, nine character text windows and periodically calls scripts to generate text to be displayed in the windows. Logically this amounts to creating a new process every few seconds so the window can be redrawn. Minimizing the overhead of this system is a "good thing," eh?)

I haven't yet done any timing comparisons, but I'm hoping that only using sed rather than grep + sed might yield at least a slight improvement.

However I welcome any comments about my strategy for trying to speed things up, and any alternatives of which you might think. (As an alternative to bash, I've been considering seeing if these scripts could be converted to lua scripts. But I first wanted to "squeeze" as much out of bash as I could.)
 
Old 10-16-2012, 02:19 AM   #8
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,412

Rep: Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874Reputation: 1874
Well I see nothing in your code that would stop you from writing it all as an awk script.

Obviously lua, perl, python or ruby would all be good choices too
 
Old 10-16-2012, 02:37 AM   #9
ntubski
Senior Member
 
Registered: Nov 2005
Distribution: Debian
Posts: 2,386

Rep: Reputation: 808Reputation: 808Reputation: 808Reputation: 808Reputation: 808Reputation: 808Reputation: 808
Quote:
Originally Posted by PTrenholme View Post
As I pointed out, I was trying to minimize the number of different programs invoked by the scripts using that code in the hope that physical file accesses would be minimized.
In an age of multi GB RAMs, trying to save a few hundred kB of disk cache isn't worth the trouble. grep is so common that it will probably already be in cache anyway.
 
Old 10-16-2012, 08:00 PM   #10
PTrenholme
Senior Member
 
Registered: Dec 2004
Location: Olympia, WA, USA
Distribution: Fedora, (K)Ubuntu
Posts: 4,147

Original Poster
Rep: Reputation: 330Reputation: 330Reputation: 330Reputation: 330
Quote:
Originally Posted by ntubski View Post
In an age of multi GB RAMs, trying to save a few hundred kB of disk cache isn't worth the trouble. grep is so common that it will probably already be in cache anyway.
One of my boxes is an old desktop with all of 512Mb memory and a real Pentiun II processor. It runs Victor Linux (and Gentoo, and a few others), but its old ATA hard drives are fairly slow. Any disk caching is to be avoided.

So, yes, most systems have multi-GB RM, etc., but not all. However, the wminfo system is targeted to people using The Window Manager as their windowing system. Those are not, I think, too likely to be people with systems that have modern resources.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
mdio function not called.. =( rashyd80 Linux - Kernel 1 07-18-2010 05:11 AM
AWK/Sed do not produce any output MasterOfTheWind Linux - Newbie 6 05-24-2009 07:50 AM
Why doesn't tee $(tty) produce output on the current terminal? openSauce Linux - Newbie 10 03-17-2009 06:09 PM
LXer: Bash One-Liner Script To Produce Somewhat-Fancy Output Of Who's On Your Linux O LXer Syndicated Linux News 0 11-19-2008 01:40 AM
xterm will not produce output with -lf parameter jarobman Linux - General 2 03-06-2005 02:12 AM


All times are GMT -5. The time now is 08:36 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration