LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Programming (https://www.linuxquestions.org/questions/programming-9/)
-   -   Parse text for host list (https://www.linuxquestions.org/questions/programming-9/parse-text-for-host-list-864370/)

Neruocomp 02-22-2011 03:40 PM
Parse text for host list
 
To run a parallel chemistry program, I need to build the host list. The cluster already has SGE(grid engine) installed and it generates a host list file with the following contents:

compute-1-1.local 2 all.q@compute-1-1.local UNDEFINED
compute-2-1.local 2 all.q@compute-1-1.local UNDEFINED

The important bits are hostname(ex compute-1-1) and number of cpus to use(ex 2). And for this program, it wants them in this form, a shell variable: HOSTLIST=hostname:cpus=X hostname:cpus=X ....

I've tried this script, but it doesn't work
Code:
#!/bin/bash
spacer=":cpus="

let count=0

cat hostfile | while read line; do
  HOSTS[$count]=$line
  ((count++))
done

NNODES=${#HOSTS[@]}

for ((i=0;i<$NNODES;i++)); do
  host=`echo ${HOSTS[$i]}|cut -f1 -d" "`
  cpus=`echo ${HOSTS[$i]}|cut -f2 -d" "`
  HOSTS[$i]="$host$spacer$cpus"
done

for j in ${HOSTS[@]} 
do
  HOSTLIST=$HOSTLIST+$HOSTS[$j]
done
Any ideas? Thanks

Tinkster 02-22-2011 04:06 PM
Not sure I got it right, but:
Code:
export HOSTLIST=$( awk '{printf "%s:cpus=%s ",$1, $2}' hostlist )
echo $HOSTLIST
compute-1-1.local:cpus=2 compute-2-1.local:cpus=2


Cheers,
Tink

Neruocomp 02-22-2011 04:45 PM
Hey that worked like a charm. I guess I gotta just sit down and read up on awk >_< BTW can no one tell me why my script doesn't work?

Tinkster 02-22-2011 05:45 PM
Quote:
Originally Posted by Neruocomp (Post 4267786)
Hey that worked like a charm. I guess I gotta just sit down and read up on awk >_< BTW can no one tell me why my script doesn't work?


It *does* work. Your only problem is that outside your script
HOSTLIST is unknown.



Cheers,
Tink

Neruocomp 02-23-2011 11:43 AM
No, this would have ended up part of a larger script where HOSTLIST would have been referenced. The thing was while testing this snip of code, I would put in echo statements to watch the variables and NNODES would come up as zero and I don't know why.

Tinkster 02-23-2011 01:39 PM
Quote:
Originally Posted by Neruocomp (Post 4268706)
No, this would have ended up part of a larger script where HOSTLIST would have been referenced. The thing was while testing this snip of code, I would put in echo statements to watch the variables and NNODES would come up as zero and I don't know why.
Hmmm ... it worked here, w/ echoes strewn in.

Maybe your file had DOS line endings or something, as opposed to what you
copy & pasted here?
Code:
~/tmp$ time ./host.sh

compute-1-1.local:cpus=2 compute-2-1.local:cpus=2

real    0m0.013s
user    0m0.006s
sys    0m0.004s
Code:
~/tmp$ time export HOSTLIST=$( awk '{printf "%s:cpus=%s ",$1, $2}' hostlist )

real    0m0.005s
user    0m0.002s
sys    0m0.002s
~/tmp$ echo $HOSTLIST
compute-1-1.local:cpus=2 compute-2-1.local:cpus=2

Cheers,
Tink


All times are GMT -5. The time now is 06:31 PM.