Text-processing and loops ...

skipdashu · 04-28-2009, 09:33 PM

Quote:

Originally Posted by homey

Good! I charge extra if you appreciate it.

Just Kidding!

How much is an explanation of the command?

I have similiar situation except the string I want is a hostname, so variable in length up to 8 but always followed by a constant in the record. Then later in the same record I need to pull an 'architecture type' that will either be "x64" or "i386".

I'll be looping thru a few records to pull the same info from each and then doing some processing based on these two variables. So how do I construct the loop around each record (line) read?

ghostdog74 · 04-28-2009, 10:33 PM

what is your sample text like? and show your output.

Tinkster · 04-28-2009, 10:47 PM

Not much to work with ...
Assuming your file looks something like this:

Code:

Is this really hostname1 FIXED  i386 a working
example host2name FIXED  x64 of what the 
guy on LQ is looking 3rdhost FIXED  i386 for after all?
He didn't give us host5 FIXED  x64 too much to work
with, did he? confused4host FIXED  i386
No, not and6host FIXED  x64 really...

This will extract the hostname and the architecture from
the stream

Code:

sed -r 's/.* ([^ ]+) +FIXED +([^ ]+).*/\1 \2/' file

And piping that into the following:

Code:

sed -r 's/.* ([^ ]+) +FIXED +([^ ]+).*/\1 \2/' file |  while read host arch; do echo "host: "$host" arch: "$arch; done 
host: hostname1 arch: i386
host: host2name arch: x64
host: 3rdhost arch: i386
host: host5 arch: x64
host: confused4host arch: i386
host: and6host arch: x64

Cheers,
Tink

P.S.: I've split your post out of the 5 year old zombie ...

skipdashu · 04-29-2009, 07:57 PM

Quote:

Originally Posted by Tinkster

Not much to work with ...
Assuming your file looks something like this:

Code:

Is this really hostname1 FIXED  i386 a working
example host2name FIXED  x64 of what the 
guy on LQ is looking 3rdhost FIXED  i386 for after all?
He didn't give us host5 FIXED  x64 too much to work
with, did he? confused4host FIXED  i386
No, not and6host FIXED  x64 really...

This will extract the hostname and the architecture from
the stream

Code:

sed -r 's/.* ([^ ]+) +FIXED +([^ ]+).*/\1 \2/' file

And piping that into the following:

Code:

sed -r 's/.* ([^ ]+) +FIXED +([^ ]+).*/\1 \2/' file |  while read host arch; do echo "host: "$host" arch: "$arch; done 
host: hostname1 arch: i386
host: host2name arch: x64
host: 3rdhost arch: i386
host: host5 arch: x64
host: confused4host arch: i386
host: and6host arch: x64

Cheers,
Tink

P.S.: I've split your post out of the 5 year old zombie ...

Tink, Thanx for getting to a current thread and for the answer... I'll be working on this tonight... very shortly. So I'm sure there'll be questions waiting tomorrow ;-) such a noob i r, i r

Somebody ask for the input file format... a typical line looks like:

Code:

Sun Mar 29 00:29:04 CDT 2009 : Added client crunch21 ; IP : 192.168.218.21 ; MAC : 00:1D:7D:96:A9:9A ; CLIENT ROOT : \/diskless\/crunch21 ; architecture : x64

I'm after "crunch21" and "x64" out of this one. I'm trying to construct a loop and do processing based on these to variables out of each line, then loop to get the next line (2 variables) and do the same processing for those two... until I hit EOF.

Below is a fragment of the script I'm trying to write where it got to be mostly notes to myself about what I wanted to do. This is the 2nd script I've ever written and I might be taking on more than I can handle yet.

Thanx again, I'll try to work the code you gave me into this.

Code:

fi
#
# File exists so parse out of it each clients hostname and arch type
echo "Working..."
while read LINE
do

# Parse out all the client names, arch types from $CLIENT_LOG file
# each line looks like:
#Sun Mar 29 00:29:04 CDT 2009 : Added client crunch21 ; IP : 192.168.218.21 ; MAC : 00:1D:7D:96:A9:9A ; CLIENT ROOT : \/diskless\/crunch21 ; #architecture : x64 
# hostname= the string starting after "Added client " and ending before ";".
# arch= the string after "architecture : " and ending before the newline.
#
#
# Then start copying updated config file from /Diskless/master.$ARCH/$PATH_FILE to
# /Diskless/$HOST/$PATH_FILE
#
        echo "$LINE"
        :

done < $CLIENT_LOG

Skip

chrism01 · 04-29-2009, 09:55 PM

This should give you a starting point:

Code:

t='Sun Mar 29 00:29:04 CDT 2009 : Added client crunch21 ; IP : 192.168.218.21 ; MAC : 00:1D:7D:96:A9:9A ; CLIENT ROOT : \/diskless\/crunch21 ; architecture : x64'

echo $t|awk '{ print $10, $27 }'

crunch21 x64

eg http://www.ibm.com/developerworks/library/l-awk1.html

skipdashu · 04-30-2009, 02:02 AM

OK some I'm still working on this script and decided I'd better work top down... I'm having a problem in a step right before I start parsing the file lines... it's about permissions to execute a cp -vp file1 file2 > logfile. May I post here or where?

Skip

chrism01 · 04-30-2009, 09:42 PM

Maybe start a new one. Incidentally, you may be a newbie, but these are Prog qns, you should probably put them in the Prog forum.
You may well get a faster response as well.

skipdashu · 04-30-2009, 09:51 PM

Quote:

Originally Posted by chrism01

Maybe start a new one. Incidentally, you may be a newbie, but these are Prog qns, you should probably put them in the Prog forum.
You may well get a faster response as well.

OK. Is there a bash or scripting under Linux forum? Didn't see anything under Linux.

Tinkster · 04-30-2009, 10:21 PM

If you have permission problems as well as coding, here's just fine,
and our programming forum is quite generic (covers ANY type of coding,
from bash script to .NET on windows ;}).

Cheers,
Tink

skipdashu · 04-30-2009, 10:35 PM

OK then I have two questions but I'll put them in two posts... one is about using cp the other refers back to the AWK to parse from earlier.

copying inside my script (lower of the two on my priority list):

This works:

Code:

sudo cp -pv /diskless/master.i386/$PATH_FILE /diskless/master.x64/$PATH_FILE

This doesn't:

Code:

sudo cp -pv /diskless/master.i386/$PATH_FILE /diskless/master.x64/$PATH_FILE >> /var/dotsch_ux/propgt.log

I did set -x and it appears to be telling me I have a permissions error on the log file. At the top of the script I did:

Code:

sudo touch /var/opt/dotsch_ux/propgt.log

The first time I ran it stopped and ask me for the sudo password (I could tell it was the touch asking because it hadn't yet done some "echos" that follow it.

It's not critical but I would like to know how to push the output of the copy verbose to a log file if I really needed it. Or is this whole concept just silly?

skipdashu · 04-30-2009, 10:51 PM

This code:

Code:

# File exists so parse out of it each clients hostname and arch type
echo "Working..."
echo ""
while read LINE
do

# Parse out all the client names, arch types from /var/opt/dotsch_ux/add_diskless_client.log

        echo $LINE| awk '{ print $10, $27 }'
        HOST=$10
	ARCH=$27
	echo $HOST, $ARCH
        :

done < $CLIENT_LOG

Produces this output:

Code:

crunch21 x64
0, 7
crunch22 x64
0, 7

The "crunch21" and "x64" are just what I need. But I need them assigned to variables $HOST and $ARCH as they become part of a pathname for a subsequent copy command.

PS: Tried this also...

Code:

HOST="$LINE| awk '{ print $10 }'"

and this

Code:

ARCH="`$LINE| awk '{ print $27 }'`"

and these

Code:

HOST=`$LINE| awk '{ print $10 }'`
ARCH=$( $LINE| awk '{ print $27 }' )

AND FINALLY... this one worked:

Code:

HOST=`echo $LINE| awk '{ print $10 }'`