LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Programming (https://www.linuxquestions.org/questions/programming-9/)
-   -   How to change my script to another so that all of it is done via awk? (https://www.linuxquestions.org/questions/programming-9/how-to-change-my-script-to-another-so-that-all-of-it-is-done-via-awk-4175503961/)

massy 05-05-2014 03:14 AM

How to change my script to another so that all of it is done via awk?
 
This code works correctly to sum column $2 and $3 where $1=IP:
Code:

while read IP
do
        grep $IP DayReport-May-03 |awk '{sumloss += $2;timeDelay += $3} END {print $1,"\t",sumloss,"\t",timeDelay}'
done < IPvar

I want to improve it by deleting grep, and doing all of that via awk, is there anyway to do that!?how?

pan64 05-05-2014 03:55 AM

I have already told you: http://www.linuxquestions.org/questi...9/#post5163847.
Have you tried that solution?

massy 05-05-2014 04:43 AM

Quote:

Originally Posted by pan64 (Post 5164784)
I have already told you: http://www.linuxquestions.org/questi...9/#post5163847.
Have you tried that solution?

yes, but it didn't work correctly! It shows the last line of IPvar repeatedly

pan64 05-05-2014 05:16 AM

http://www.linuxquestions.org/questi...9/#post5164239

massy 05-05-2014 06:00 AM

Quote:

Originally Posted by pan64 (Post 5164814)

This script is a test script, so I use awk in it;
It is an example of my DayReport-May-04

192.168.40.29 0% 1265.756
192.168.40.1 0% 693.506
192.168.40.4 10% 1112.107
192.168.40.1 10% 754.015
192.168.40.4 0% 635.516
192.168.40.29 0% 538.657
192.168.40.22 0% 640.583
192.168.40.4 0% 615.745
192.168.40.1 0% 507.429
...

--------------------------------------------------------------------------------
and I have some IP addresses in IPvar that is read line by line via while loop:
192.168.40.1
192.168.40.27
192.168.40.5
192.168.40.29
...
---------------------------------------------------------------------------------
I need to sum of amounts of column 2 and 3 for each IP, for example, the output for above files should be:

192.168.40.1 20% 1954.95
...
192.168.40.29 0% 1804.413
---------------------------------------------------------------------------------
I've tried this code via awk:
Code:

while read IP
do
        set -vx
        awk -v IP="$IP" '/IP/{sumloss += $2;timeDelay += $3} END {print $1,"\t",sumloss,"\t",timeDelay}' ./PingReportFolder/DayReport-May-04
done < IPvar

It just work for the last record of DayReport-May-04!!!!

pan64 05-05-2014 06:18 AM

/IP/ means awk will take IP as string, not as variable, therefore it will look for the text IP in your input file.
you can try the following instead:
Code:

awk -v IP="$IP" '$1 == IP {sumloss += $2;timeDelay += $3} END {print $1,"\t",sumloss,"\t",timeDelay}'
or
awk  '/'$IP'/{sumloss += $2;timeDelay += $3} END {print $1,"\t",sumloss,"\t",timeDelay}'

but even better it can be solved without that while cycle ...

grail 05-05-2014 07:09 AM

Simply read in the file with all of the addresses in it and then cycle through the second file creating your totals. You will then need to utilise the 'END' action within awk to display the data in the format you require

sundialsvcs 05-05-2014 07:34 AM

This is a case where I would replace some logic with, say, "a very small Perl script." Or, in this case, with "awk itself (you don't need a separate script at all)."

"You have some rather funky magic going on" in the present (convoluted ...) script, and, although it might (or might not) be argued that "well, it works ...", such things are so hard to maintain (and, as you now see, so hard to debug!) that I routinely replace them, and/or ask a member of my team to do so. (Yes, I take my turn at "sweeping out the garbage," when I'm not wearing my management-hat and thus sweeping in more garbage. ;) )

You see, what this script in-aggregate does is to read a file of "IP-addresses and other things," and to extract certain variables out of it to produce certain results. But does it, cleanly and with just one tool, do that? Uhh, no. It uses two tools clumsily piped together, and neither of the two bits of source-code provided to either of the two tools completely expresses the total of what you actually want to accomplish.

First of all, a short (but true) awk script could quite-effortlessly do this: it just requires, well, "a separate file instead of a one-liner." You simply invoke the awk command giving it the name of the IP-address file and the name of a short but separate awk script. All of the necessary logic now sits within that script; not in whatever invokes awk.

Perl is awk's big-brother, so to speak, in that it literally was conceived as "an extension of awk" in its early days and still bears noticeable resemblances to it. Aptly called "the Swiss ArmyŽ Knife of data processing," it's a tool that I use quite often. But once again, never as a one-liner in production settings.

ntubski 05-05-2014 09:52 AM

Code:

grep -Ff IPvar DayReport-May-04 | awk '{
    sumloss[$1] += $2;
    timeDelay[$1] += $3;
}
END {
    for (ip in sumloss) {
        print ip,"\t",sumloss[ip],"\t",timeDelay[ip]
    }
}'


massy 05-06-2014 04:35 AM

Quote:

Originally Posted by pan64 (Post 5164837)
/IP/ means awk will take IP as string, not as variable, therefore it will look for the text IP in your input file.
you can try the following instead:
Code:

awk -v IP="$IP" '$1 == IP {sumloss += $2;timeDelay += $3} END {print $1,"\t",sumloss,"\t",timeDelay}'
or
awk  '/'$IP'/{sumloss += $2;timeDelay += $3} END {print $1,"\t",sumloss,"\t",timeDelay}'

but even better it can be solved without that while cycle ...

The output of your code is like as the output of my script! showing the last IP in Day-Report-May-04 repeatedly!

massy 05-06-2014 04:37 AM

Quote:

Originally Posted by ntubski (Post 5164966)
Code:

grep -Ff IPvar DayReport-May-04 | awk '{
    sumloss[$1] += $2;
    timeDelay[$1] += $3;
}
END {
    for (ip in sumloss) {
        print ip,"\t",sumloss[ip],"\t",timeDelay[ip]
    }
}'


I've done it via grep myself! I want to do all of that via awk!!!

pan64 05-06-2014 04:42 AM

to #10: yes, sure, did you put it into that while cycle from port #5? (you just need to replace the line containing the awk command).
to #11: probably you can simply drop that grep and use only the awk by ntubski

massy 05-06-2014 07:07 AM

Quote:

Originally Posted by pan64 (Post 5165528)
to #10: yes, sure, did you put it into that while cycle from port #5? (you just need to replace the line containing the awk command).
to #11: probably you can simply drop that grep and use only the awk by ntubski

yes I've putted it there!


All times are GMT -5. The time now is 12:36 AM.