awk shell scripting

ApacheRoseXbones · 07-10-2008, 09:54 AM

hi everyone!

I'm trying to make a shell script, and I'm trying to use awk to define a variable that will later be used for bash, but i keep getting this error:

line 5: BEGIN {OFS = ","}; /The following Mo dRedundant/ {getline; print $2}: No such file or directory

heres my script:

Code:

#!/bin/bash

for i in `ls *.log`
do
       a=awk ' BEGIN {OFS = ","}; /The following ModRedundant/ {getline; print $2}'\
       b=awk ' BEGIN {OFS = ","}; /The following ModRedundant/ {getline; print $3}'\
       c=awk ' BEGIN {OFS = ","}; /The following ModRedundant/ {getline; print $4}'\
       ANG=${i/log/sh}
       chmod 777 $ANG 
       echo "scanf=$1" >>$ANG
       echo "awk -v mine=$mine 'BEGIN {found=0; ene=0; count=0; printf \"%2s %8s %12s\n", "#", "Ang1", "E\";} " >>$ANG
       echo "/E\(RHF\)/ {ene=$5; found=0;} " >>$ANG
       echo "/Stationary point found/ {found=1; count++} " >>$ANG
       echo "(found==1 && $0 ~ /Optimized Parameters/) {found=2} "  >>$ANG
       echo "(found==2 && $0 ~ /A\(x,y,z\)/) {found=3; Ang1=$4} " >>$ANG
       echo "(found==3) {found=4; printf \"%-2d %8.3f %12.6f\n\", count, Ang1, ene }' $1" >>$ANG
       awk '{ sub (/x/, $a); print }'
       awk '{ sub (/y/, $b); print }'
       awk '{ sub (/z/, $c); print }'
       $ANG < $i > $i.dat
done

also, in the last few lines, i'm trying to get awk to print into the new file $ANG. How would I do that?

Thanks!

ghostdog74 · 07-10-2008, 10:02 AM

that's so messy. some of your constructs are wrong too. Describe your problem and expected output, show samples of input files where possible.

ApacheRoseXbones · 07-10-2008, 10:14 AM

alrighty, heres the general jist of what the script is supposed to do.

from a program I run I get a ton of output files (.log).

From those .log files, I am trying to:
1. pull data out (variables a, b, and c)
2. write those variables into a new file which is going to be a shell script ($ANG)
3. $ANG is comprised of everything that is being echoed. Its purpose is to go back into that original .log file and pull out even more data. (The data corresponds to the values of a,b and c input.)
4. The final step would then be to run $ANG and have the results output into a file ending with extension .dat

script:

Code:

#!/bin/bash

for i in `ls *.log`
do
       a=awk ' BEGIN {OFS = ","}; /The following ModRedundant/ {getline; print $2}'\
       b=awk ' BEGIN {OFS = ","}; /The following ModRedundant/ {getline; print $3}'\
       c=awk ' BEGIN {OFS = ","}; /The following ModRedundant/ {getline; print $4}'\
       ANG=${i/log/sh}
       chmod 777 $ANG 
       echo "scanf=$1" >>$ANG
       echo "awk -v mine=$mine 'BEGIN {found=0; ene=0; count=0; printf \"%2s %8s %12s\n", "#", "Ang1", "E\";} " >>$ANG
       echo "/E\(RHF\)/ {ene=$5; found=0;} " >>$ANG
       echo "/Stationary point found/ {found=1; count++} " >>$ANG
       echo "(found==1 && $0 ~ /Optimized Parameters/) {found=2} "  >>$ANG
       echo "(found==2 && $0 ~ /A\(x,y,z\)/) {found=3; Ang1=$4} " >>$ANG
       echo "(found==3) {found=4; printf \"%-2d %8.3f %12.6f\n\", count, Ang1, ene }' $1" >>$ANG
       awk '{ sub (/x/, $a); print }'
       awk '{ sub (/y/, $b); print }'
       awk '{ sub (/z/, $c); print }'
       $ANG < $i > $i.dat
done

...hope that helps to clear things up, and I'm sorry this is messy, but this is my first time writing something like this.

Mr. C. · 07-11-2008, 01:31 AM

Let me kindly suggest that you rephrase what your ultimate goal is. Try not to explain step-by-step how you think it should be done, but rather explain what the starting point is, and what the end result should be. For another explanation, see:

http://www.catb.org/~esr/faqs/smart-questions.html#goal

ApacheRoseXbones · 07-11-2008, 06:19 AM

Okay, take three:

The ultimate goal of the program is to pull data from a file. The main problem, though, is that in order to pull data from the file, you need to use data thats already in the file, if that makes any sense. So as it stands I have a shell script that will pull data from the files, but every time I run it i have to edit it to correspond to each file, and it gets a bit hairy when I'm doing this for 30+ files at a time. The original text of that shell script that I already use is what is being "echoed" in the script I posted above.

And I'm sorry but I can't post an example file for what I'm trying to extract from.

..Was that any better?

ghostdog74 · 07-11-2008, 06:29 AM

that's not going to help either. Make up an example file and describe properly what you want to see as final result.

ApacheRoseXbones · 07-11-2008, 07:09 AM

Quote:

Originally Posted by ghostdog74

that's not going to help either. Make up an example file and describe properly what you want to see as final result.

It would be impossible for me to do that. The files are extremely complex (7000+ pages) and if I were to give you any sort of truncated version then the interdependence of the data wouldn't show properly. It would just be a mess.

Thanks for trying to help though, I guess I'll just ask someone in person.

Mr. C. · 07-11-2008, 12:42 PM

Anything impossible for you to describe would be impossible to program in a computer. It is through the translation of your goals into a sequence of steps that makes programming possible. If this is not possible of you, consider another line of work.

It doesn't matter if your input files are 1 line or 100,000 pages of lines. You still must describe (either to us, or to yourself) a basic set of patterns or sequence of events. This can't be avoided.

Explain your program conceptually, not literally, in broad, general steps.

1) find lines that match xyx
2) grab a pattern abc from that line
3) output pattern and line number
4) grab another pattern from line
...

As an aside, part of the difficult you face is getting stuck and cycling in language and procedure. Consider:

Quote:

The ultimate goal of the program is to pull data from a file. The main problem, though, is that in order to pull data from the file, you need to use data thats already in the file, if that makes any sense. So as it stands I have a shell script that will pull data from the files,..

This is more easily stated as: "process data from a file"

Quote:

...but every time I run it i have to edit it to correspond to each file, and it gets a bit hairy when I'm doing this for 30+ files at a time.

Now you've jumped back into implementation mode. Perhaps you mean to say: "store data for later output." You are also getting overwhelmed by what are very small numbers. 30 files, 7000 pages; that's trivial stuff *IFF* you organize your thoughts and generalize your concepts.

Quote:

The files are extremely complex (7000+ pages) and if I were to give you any sort of truncated version then the interdependence of the data wouldn't show properly. It would just be a mess.

Again, you're overwhelmed. If the files are that complex, then you need to get your ideas cleared up and written down, or communicated to others. What is a mess is your implementation above, so stop worrying about mess, and focus instead on communication and organization.

We're here to help, otherwise, best of luck.