[SOLVED] add all numeric values from variable in a single command

pedropt · 09-22-2018, 05:06 PM

Hi , maybe this is not the right name for this thread , but i was unable to remember anything else for it .

I get a variable with multiples lines with numbers

variable 1 = $var1

Quote:

34
789
285
23
etc ...

The question is how to add all values in a single shot .
If it was a file , then i was able to count the lines and then insert a loop by adding each value to the next one , but even that would gave me issues ahead .

Is there anyway to get an output with the total value from this variable ?

Something like :
34+789+285+23 = 1131 expected as output

The initial variable is not fixed , this means that can have 2 lines only as also can have 10 lines to be counted .

Thanks

pedropt · 09-22-2018, 05:28 PM

I think i got it , i was just stubbling around here on the logic to work this out .
Eventually this will work , but i did not test it out yet because i have other programming to do to this script before running it .
If anyone have a better idea then post it here .

Here it is what came to my mind :
I called the variable with all the numbers to be counted as ipc2

Code:

#Here splits on the echo command line by line the numbers 
# If the numbers on echo are displayed as (34 789 285 23) , then the tr command will put each #one on a single line
a1=$(echo "$ipc2" | tr " " "\n" | wc -l)
# a1 = total number of lines to be added
# Defined out variable as an empty var
out=""
#Start the loop based on the number of lines to be counted before
for i in $(seq "$a1")
do
# read the first line
nmb=$(echo "ipc2" | sed -n ${i}p)

#Here it is the trick , if the out is empty then dont add nothing , just place the first #number
if [[ -z "$out" ]]
then
out="$nmb"
else
#case out variable is not empty then add its value to the one got from nmb var before
out=$((out+"$nmb"))
fi
done 
echo "total vlaue is : $out"

syg00 · 09-22-2018, 06:39 PM

Code:

echo $ipc2 | awk '{for (i=1; i<=NF; i++) sum+=$i} END{print "Total value is: "sum}'

KISS

lougavulin · 09-22-2018, 08:31 PM

I would have done it with Awk as syg00.

But with shell, that is not as hard as you do. Just for information :

Code:

SUM=0
for INT in $(echo ${ipc2} | tr -dc '[0-9] '); do
  SUM=$(( INT + SUM ))
done
echo "Total value is: ${SUM}"

The only trick might be tr -dc, which remove everything that is not a number or a space. So your ipc2 variable could be '123 345 abcd &+-' it will still work.

ondoho · 09-23-2018, 01:50 AM

i question the need to have multiple numeric values in a single variable like that.
can't you use an array instead?
assuming bash or another shell that supports them?

grail · 09-23-2018, 02:19 AM

I am with ondoho, please show how the "ipc2" variable is being populated?

MadeInGermany · 09-23-2018, 02:45 AM

Yes, out or SUM should be preset to 0 not ""
And yes, the for loop's list should be the input values, that are separated by IFS (space or tab or newline).

Code:

out=0
for nmb in $ipc2
do
  out=$((out+nmb))
done

bash supports

Code:

  ((out+=nmb))

pedropt · 09-23-2018, 03:12 AM

some how is not working , i have to do a loop inside a loop because it is not stopping , i will be back as soon as i get it working .

Reedited .

Basicaly this is a script to read the serverlog , and count how many times an ip access the server .
So , the serverfile is a raw file that an external tcp monitoring tool is writing all the time as soon as anyone access the server ip no matter witch port is accessed .

With a few commands i can filter all the stuff i dont want to be check in this count , by this i mean dns requests from server , igmp , 224.0.0.1 , requests to the main isp modem gateway ip ,etc

From the raw file i get a new file that is something like this next example :

Quote:

2 1.1.1.1
4 5.5.5.5
1 3.3.3.3
1 1.1.1.1
9 7.7.7.7
6 4.4.4.4
5 2.2.2.2
3 5.5.5.5

The repeated ips on same list means that same ip access the server on different times , by this is means :

Example : ip 1.1.1.1 first time ask 2 requests to server , later ahead made 1 request to server , and go on

Now , my objective is to count how many times an ip access the website and what was doing in the site , by this and very ahead on the script , by the ip address i have a command to check all logs to see what was doing , this way i can ban that specific ip or not if want it .
Some ips are legit , accessing webite normally , others are pentesting , by default the pentesting ips get banned from server on the firewall rules almost at the end of the sript .

Bu returning .
After the first filter i use a loop to read the filtered file (from raw tcp dump) , and read line by line , on reading line by line what i do is :
1st - i get the ip on a variable
2nd - i search from that ip on the filtered file and retrieve the number of accesses

something like this :

Code:

ffile="output"
function count (){
SUM=0
for INT in $(echo ${rdaccess} | tr -dc '[0-9] '); do
  SUM=$(( INT + SUM ))
# echo is a quick helper for me to know if everything is working as expected , it will be removed)
echo "$rdip is $SUM"
done

# Now i will create a new file with everything ring and without repeated ips

# if output file exists then check if the ip is already there ,
# i have to do it this way , because on the loop , the script will count same ip multiples #times in many parts of the filtered file
if [[ -f "$ffile" ]]
then
a1=$(grep "$rdip"< "$ffile")
if [[ -z "$a1" ]]
then
i case the specific ip is not yet in the list then add it to the final file
echo "$SUM $rdip">> $ffile
fi
else
# here is in case the final file does not exist yet , this will write the first ip to it
echo "$SUM $rdip"> $ffile
fi 
}

#tmp is the filtered file from original raw
cntlst=$(wc -l <tmp)
for i in $(seq "$cntlst")
do

#read the first ip
rdip=$(sed -n ${i}p < tmp | awk '{print$2}')

# get all the accesses from that ip in the filtered file
rdaccess=$(grep "$rdip" < tmp | awk '{print$1}')

# at this point rdaccess variable only have the accesses numbers from Quote added before

# count is a subfunction where it will count all the rdaccess variable numbers
count
done

pedropt · 09-23-2018, 03:49 AM

Well , i got it working , like it is in the previous code , it toke sometime because filtered file have 2,4M of data to be counted , and this took sometime

pan64 · 09-23-2018, 04:37 AM

it is far from optimal. Instead of:

Code:

cntlst=$(wc -l <tmp)
for i in $(seq "$cntlst")
do

#read the first ip
rdip=$(sed -n ${i}p < tmp | awk '{print$2}')

# get all the accesses from that ip in the filtered file
rdaccess=$(grep "$rdip" < tmp | awk '{print$1}')

# at this point rdaccess variable only have the accesses numbers from Quote added before

# count is a subfunction where it will count all the rdaccess variable numbers
count
done

you can try something like:

Code:

while read -r _ rdip;
do
# get all the accesses from that ip in the filtered file
rdaccess=$(awk "/$rdip/"'{print $1}' tmp)

# at this point rdaccess variable only have the accesses numbers from Quote added before

# count is a subfunction where it will count all the rdaccess variable numbers
count
done < tmp

I have not tested it (but will be definitely faster)
And it can be still improved if awk will do that count function.

pedropt · 09-23-2018, 09:24 AM

Thanks to all , after a few adjustments to the script and adding more things to it here it is the output :

https://i.imgur.com/j2AYCBJ.png

The first question is to ask to user after how many ip accesses he wants the script to display the results , because there are hundreds of inputs with 1 or 2 accesses , witch are normally legit and almost dont need to be checked .

This is just the script that prompts to user what is happening in the webserver inputs and give him an idea how to follow next , ahead this script comes another that searches the log files for that ip entries and display it to user so he can decide if that ip gets banned or not , and it makes changes to the firewall settings in real time .

Probably in future i will upgrade it if i get some time to a new level where statistics from target ip will retrieve ports accesses and witch files , etc....

Again , thanks for the help .

pan64 · 09-23-2018, 09:32 AM

I think even better to do something like this:

Code:

awk '{sum[$2] += $1 }
     END { for (i in sum) {
           print i,":",sum[i];
     } }' inputfile | sort -n | head

(again, not tested)