Scripting help

TSCollins · 05-06-2021, 08:35 PM

Have a multi line file that contains four fields, the first field is a label and the other three are numbers. I need to find the percentage difference between field two and four and if it is greater then 90% print the label and if it is less then 90% ignore it and move onto the next line.

Turbocapitalist · 05-06-2021, 09:26 PM

That sounds like an easy job for AWK. What have you tried so far, where are you stuck? Also, please show a few lines of sample input and a few lines of expected output.

TSCollins · 05-07-2021, 07:14 AM

Actually the source file has more then four fields but I only am interested in four of the fields, here are a few sample lines from the source file:

Name Id RootInode ParentId Created InodeSpace MaxInodes AllocInodes UsedInodes
root 0 3 -- Fri Aug 18 11:53:03 2017 0 100000000 1528320 4187
home 1 524291 0 Fri Aug 18 12:06:10 2017 1 100000000 22502400 5828699
apps 5 2621443 0 Fri Aug 18 14:38:12 2017 5 50000128 17380352 11488908

I have been able to extract the four that I want to work with (Name, MaxInodes, AllocInodes & UsedInodes) like this:

cat inode.txt | awk '{print $1,$11,$12,$13}'
root 100000000 1528320 4187
home 100000000 22502400 5828699
apps 50000128 17380352 11488908

The problem I am having is with keeping the name in sync with the numbers then doing the calculation and determining if it is > or < 90%. Here is what I have so far in the script:

#!/bin/bash
# check inodes of filesets
declare THRESHOLD=90
Key=`cat inode.txt | sed "1d" | awk '{print $1,$11,$12,$13}'`
for K in $Key
do
echo $K
done

When I run the script I get the following output:

root
100000000
1528320
4187
home
100000000
22502400
5828699
apps
50000128
17380352
11488908

I have tried setting different variables like this:

Name=`cat inode.txt | awk '{print $1}'`
MaxInodes=`cat inode.txt | awk '{print $11}'`
AllocInodes=`cat inode.txt | awk '{print $12}'`
UsedInodes=`cat inode.txt | awk '{print $13}'`

for N in $Name
for M in $MaxInodes
for A in $AllocInodes
for U in $UsedInodes

and then thought something like

do
echo $N
echo $U / $A \* 100 | bc -l
if (( ${above_result} >= ${THRESHOLD} )) ; then
echo "fileset $N is close to allocated inodes"
else
echo "fileset $N is ok"
fi
done

The first problem I of course am having is the multiple 'for' statements. I then tried nesting them but then I just get the first value for $N i.e. root with all the values for say $U i.e. 4187, 5828699 & 11488908.

shruggy · 05-07-2021, 07:33 AM

Well, you don't have any filesystem with more than 90% inodes used, so here it is with 66%:

Code:

awk 'NR>1&&$13/$12>0.66,$0=$1' inode.txt

Turbocapitalist · 05-07-2021, 07:34 AM

AWK can do much more, so Bash is not needed or desired probably:

Code:

awk '$13 && $13/$12>=0.9 {
        print "Fileset " $1 " is close to allocate inodes"
        next
        }
$13 {
        print "Fileset " $1 " is OK" 
}' inodes.txt

MadeInGermany · 05-08-2021, 04:36 AM

Quote:

Key=`cat inode.txt | sed "1d" | awk '{print $1,$11,$12,$13}'`

This retains the spaces and newlines, but

Quote:

for K in $Key

splits into words according to $IFS.
You can improve it by modifying IFS to only a newline

Code:

oIFS=$IFS
IFS="
"

and later restore IFS with

Code:

IFS=$oIFS

But better use a while-read loop where the read defaults to a newline, and can assign fields to variables.

Code:

awk 'NR>1 {print $1,$11,$12,$13}' inode.txt |
while read Name MaxInodes AllocInodes UsedInodes
do
  echo "do something with $MaxInodes and $UsedInodes e.g. $((UsedInodes * 100 / AllocInodes)) and print along with $Name" 
done

The pipe might force the while loop into a subshell so you cannot set variables in the main shell.
The following while loop runs in the main shell:

Code:

while read Name MaxInodes AllocInodes UsedInodes
do
  echo "do something with $MaxInodes and $UsedInodes $((UsedInodes * 100 / AllocInodes)) and print along with $Name" 
done < <(awk 'NR>1 {print $1,$11,$12,$13}' inode.txt)

This construct is called a "process substitution".

igadoter · 05-08-2021, 04:48 AM

First drop any unneeded fields. Just filter file and create new with only these fields required. Do by hand a little - look at coreutils - and I am sure you will solution. Bash automates tasks we can do by hands - but it is too tedious, error prone, or data are too large. But reworking small sample should definitely clarify situation. I am not posting ready solution - task is really very easy to do.