[SOLVED] Bash Scripting help for memory usage

TroubleBob · 04-16-2013, 10:37 AM

Hi,
I'm trying to write a bash script that outputs the process using the most memory when total free memory is <20%.
So I need to read the memory usage in from /proc/meminfo and just do arithmetic on Memtotal and Memfree
An then output ps to a log

Code:

#!/bin/bash
TOTALMEM=$(cat /proc/meminfo|head -n1|cut -c 18-24)
USEDMEM=$(cat /proc/meminfo|head -n2|tail -n1|cut -c 18-24)
echo $USEDMEM
echo $TOTALMEM
LOGSTRING=$(ps axo pid,args,pmem --sort -pmem|head -n 2|tail -n 1)
echo $LOGSTRING

How do I do the arithmetic in bash?

TenTenths · 04-16-2013, 11:14 AM

You can encase your expression in double brackets:

Code:

z=$(($z+3))

z=$((z+3))

Or make use of backticks and the expr command

Code:

z=`expr $z + 3`

For me, Mendel Cooper's Advanced Bash Scripting Guide is essential! (Google it)

konsolebox · 04-16-2013, 11:49 AM

@TroubleBob: see the bash manual.

Or run

Code:

man bash

You could use (( )) for that. And any parts of the script where arithmetic expressions are allowed like in indices of variables, special arguments in special parameter expansion forms, etc.

Code:

help '(('
help let

shivaa · 04-16-2013, 12:29 PM

Can you explan it little more that what exactly you want?

You can run following cmds:

Code:

~$ cat /proc/meminfo | sort -nr -k2
~$ cat /proc/meminfo | grep MemFree
~$ cat /proc/meminfo | grep MemTotal

In order to send output into a log file, simply redirect your script or commands into a file, as:

Code:

~$ ./script > /tmp/memorylog.txt

TroubleBob · 04-17-2013, 04:30 PM

I need to set it up as a scheduled job with cron to output the process using the most memory when memory usage goes over 80%.
I'll be back if I get stuck again thanks
Its homework so the more I can do without help the better

PTrenholme · 04-17-2013, 05:45 PM

If it's homework, I'd suggest you look at the man pages for head and tail, and eliminate the unnecessary cat <file> | cmd from your code. That idiom should be reserved for situations where it's necessary, and it a really bad habit to use it unnecessarily.

chrism01 · 04-17-2013, 08:23 PM

Good bash tutorials/howtos
http://rute.2038bug.com/index.html.gz
http://tldp.org/LDP/Bash-Beginners-G...tml/index.html
http://www.tldp.org/LDP/abs/html/

TroubleBob · 04-18-2013, 10:57 AM

Code:

#!/bin/bash
NUMCORES=$(grep "^core id" /proc/cpuinfo | sort -u | wc -l)
LOADAVG=`cut /proc/loadavg -d ' ' -f 1`
PERUSAGE=`echo $LOADAVG '/' $NUMCORES '* 100 '|bc -l`
PROCESSSTRING=$(ps axo pid,args,%cpu --sort -%cpu|head -n 2|tail -n 1)
DAY=`date +%d`
MONTH=`date +%m`
YEAR=`date +%Y`
if [ `echo PERUSAGE '>80'|bc -l` -eq 1 ];then
    if [ ! -d "log" ];then
        mkdir "log"
    fi
    cd "log"
    if [ ! -d "$YEAR-$MONTH" ];then
        mkdir "$YEAR-$MONTH"
    fi    
    cd $YEAR-$MONTH
    echo $PROCESSSTRING>>$DAY.log
    cd ".."
fi

Turns out I was doing the wrong assignment

but not too bad nearly got it finished, I just need to write code to delete the logs that are older than 14 days now, the rest of this code outputs a log showing the most CPU intensive process when the load is >80%

PTrenholme · 04-18-2013, 12:00 PM

Hint:

Quote:

Originally Posted by man mkdir

-p, --parents
no error if existing, make parent directories as needed

which can simplify

Code:

    if [ ! -d "log" ];then
        mkdir "log"
    fi
    if [ ! -d "$YEAR-$MONTH" ];then
        mkdir "$YEAR-$MONTH"
    fi

to

Code:

    mkdir -p log/${YEAR}-${MONTH}

TroubleBob · 04-18-2013, 01:14 PM

Code:

for((i=1; i<$MONTH; i++))
    do
        echo $i "loop counter"
        echo ${YEAR}-${i}
        echo ${YEAR}-"0"${i}
        rm -rf ${YEAR}-${i}
        rm -rf ${YEAR}-"0"${i}
    done

Any idea why this isn't working
In this context

Code:

#!/bin/bash
NUMCORES=`grep "^core id" /proc/cpuinfo | sort -u | wc -l`
LOADAVG=`cut /proc/loadavg -d ' ' -f 1`
PERUSAGE=`echo $LOADAVG '/' $NUMCORES '* 100 '|bc -l`
PROCESSSTRING=$(ps axo pid,args,%cpu --sort -%cpu|head -n 2|tail -n 1)
DAY=`date +%d`
MONTH=`date +%m`
YEAR=`date +%Y`
cd $HOME
#if [ `echo $PERUSAGE '>80 '|bc -l` -eq 1 ];then
    mkdir -p $HOME/log/${YEAR}-${MONTH}
    cd $HOME/log/${YEAR}-${MONTH}
    echo $PROCESSSTRING>>$DAY.log
    cd $HOME/log        
    if [ `echo $DAY '>14 '|bc -l` -eq 1 ];then
    for((i=1; i<$MONTH; i++))
    do
        echo $i "loop counter"
        echo ${YEAR}-${i}
        echo ${YEAR}-"0"${i}
        rm -rf ${YEAR}-${i}
        rm -rf ${YEAR}-"0"${i}
    done
    fi
#fi

PTrenholme · 04-18-2013, 03:36 PM

Not without you telling us the error(s) that code generated.

This works:

Code:

$ YEAR=2013;MONTH=5;for((i=1; i<$MONTH; i++));do echo ${YEAR}-"0"${i};echo ${YEAR}-${i};done
2013-01
2013-1
2013-02
2013-2
2013-03
2013-3
2013-04
2013-4

Another hint or two:

The test function includes numeric comparison operators (e.g.: -lt -le -eq -ge and -gt). So why pipe into bc to test if [ ${DAY} -gt 14 ] ? (Beware that a leading zero will make the number octal in both bc and test.)
There is a printf command (on most system) that can be used to format a number with leading zeros.
The form $(cmd) is usually preferred to `cmd`, although the effect is often the same.

TroubleBob · 04-18-2013, 04:15 PM

There wasn't an error I was looking in the wrong place it was working, thanks for your time folks, looking forward to contributing to the forum

This is my final code

Code:

#!/bin/bash
NUMCORES=`grep "^core id" /proc/cpuinfo | sort -u | wc -l`
LOADAVG=`cut /proc/loadavg -d ' ' -f 1`
PERUSAGE=`echo $LOADAVG '/' $NUMCORES '* 100 '|bc -l`
TIME=`date +%c`
PROCESSSTRING=$TIME$(ps axo pid,args,%cpu --sort -%cpu|head -n 2|tail -n 1)
DAY=`date +%d`
MONTH=`date +%m`
YEAR=`date +%Y`
cd $HOME
if [ `echo $PERUSAGE '>80 '|bc -l` -eq 1 ];then
    mkdir -p $HOME/log/${YEAR}-${MONTH}
    cd $HOME/log/${YEAR}-${MONTH}
    echo $PROCESSSTRING>>$DAY.log
    cd $HOME/log
    if [ `echo $DAY '>14 '|bc -l` -eq 1 ];then
    for((i=1; i<$MONTH; i++))
    do
        rm -rf ${YEAR}-${i}
        rm -rf ${YEAR}-"0"${i}
    done
    cd $HOME/log/${YEAR}-${MONTH}
    for((j=1; j<($DAY-14); j++))
    do
        rm -rf $j".log"
        rm -rf "0"$j".log"
    done
    fi
fi

PTrenholme · 04-18-2013, 07:02 PM

You might find this interesting. Here are two re-written versions of the first five lines of the solution you posted:

Code:

$ cat TroubleBob[12].sh 
#!/bin/bash
NUMCORES=$(grep "^core id" /proc/cpuinfo | sort -u | wc -l)
LOADAVG=$(cut /proc/loadavg -d ' ' -f 1)
PERUSAGE=$(echo ${LOADAVG} '/' ${NUMCORES} '* 100 '|bc -l)
TIME=$(date +%c)
echo NUMCORES=${NUMCORES}
echo LOADAVG=${LOADAVG}
echo PERUSAGE=${PERUSAGE}

#!/bin/bash
NUMCORES=$(grep "^core id" /proc/cpuinfo | sort -u | wc -l)
LOADAVG=$(cut /proc/loadavg -d ' ' -f 1)
PERUSAGE=$(printf "scale=2\n$LOADAVG/$NUMCORES* 100\n"|bc)
TIME=$(date +%c)
echo NUMCORES=${NUMCORES}
echo LOADAVG=${LOADAVG}
echo PERUSAGE=${PERUSAGE}

I ran these using the time function and got this:

Code:

TroubleBob1.sh

NUMCORES=4
LOADAVG=0.56
PERUSAGE=14.00000000000000000000

real    0m0.077s
user    0m0.005s
sys     0m0.080s
TroubleBob2.sh

NUMCORES=4
LOADAVG=0.56
PERUSAGE=14.00

real    0m0.106s
user    0m0.012s
sys     0m0.107s

<edit>
Note that I dropped the -l option from the bc command. That option is only necessary if you need transcendental functions, and it sets scale to 20.
</edit>

David the H. · 04-18-2013, 07:17 PM

Congratulations on figuring it out. However, there's a lot of room for improvement in your code. In particular there are a lot of unnecessary external process calls.

I've taken the liberty of re-writing it the way I would've done it. I'll just post it as-is, and hopefully you can analyze it as a learning exercise.

Code:

#!/bin/bash

IFS=- read -r year month day time < <( date '+%F-%c' )

read loadavg _ </proc/loadavg
numcores=$( grep -c '^core id' /proc/cpuinfo )
perusage=$( bc <<<"$loadavg / $numcores * 100" )
printf -v perusage '%.f' "$perusage"   #rounds it to the nearest integer

if (( perusage > 80 ));then

    cd "$HOME/log" || { echo "Can't enter log dir. Exiting." ; exit 1 ;}
    mkdir -p "$year-month"

    ps axo pid,args,%cpu --sort -%cpu --no-headers |\
      head -n 1 >> "$year-month/day.log"

    if (( ${day#0} > 14 ));then

        rm -rf $year-[1-$month] $year-0[1-$month]

        cd "$year-$month"

        (( day = day - 14 ))
        rm -f [1-$day].log 0[1-$day].log

    fi

fi

exit 0

David the H. · 04-21-2013, 02:46 PM

I just recognized a small flaw in my previous script.

Code:

if (( day > 14 ));then

this will not work properly in days 08 and 09, since the zero-padded numbers will be treated as octal values and error out. It's a simple fix though. Just strip off the leading zero:

Code:

if (( ${day#0} > 14 )); then

I'm updating my previous post to correct this too.