LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Newbie (http://www.linuxquestions.org/questions/linux-newbie-8/)
-   -   Bash Scripting help for memory usage (http://www.linuxquestions.org/questions/linux-newbie-8/bash-scripting-help-for-memory-usage-4175458329/)

TroubleBob 04-16-2013 09:37 AM

Bash Scripting help for memory usage
 
Hi,
I'm trying to write a bash script that outputs the process using the most memory when total free memory is <20%.
So I need to read the memory usage in from /proc/meminfo and just do arithmetic on Memtotal and Memfree
An then output ps to a log

Code:

#!/bin/bash
TOTALMEM=$(cat /proc/meminfo|head -n1|cut -c 18-24)
USEDMEM=$(cat /proc/meminfo|head -n2|tail -n1|cut -c 18-24)
echo $USEDMEM
echo $TOTALMEM
LOGSTRING=$(ps axo pid,args,pmem --sort -pmem|head -n 2|tail -n 1)
echo $LOGSTRING

How do I do the arithmetic in bash?

TenTenths 04-16-2013 10:14 AM

You can encase your expression in double brackets:

Code:

z=$(($z+3))

z=$((z+3))

Or make use of backticks and the expr command

Code:

z=`expr $z + 3`
For me, Mendel Cooper's Advanced Bash Scripting Guide is essential! (Google it)

konsolebox 04-16-2013 10:49 AM

@TroubleBob: see the bash manual.

Or run
Code:

man bash
You could use (( )) for that. And any parts of the script where arithmetic expressions are allowed like in indices of variables, special arguments in special parameter expansion forms, etc.
Code:

help '(('
help let


shivaa 04-16-2013 11:29 AM

Can you explan it little more that what exactly you want?

You can run following cmds:
Code:

~$ cat /proc/meminfo | sort -nr -k2
~$ cat /proc/meminfo | grep MemFree
~$ cat /proc/meminfo | grep MemTotal

In order to send output into a log file, simply redirect your script or commands into a file, as:
Code:

~$ ./script > /tmp/memorylog.txt

TroubleBob 04-17-2013 03:30 PM

I need to set it up as a scheduled job with cron to output the process using the most memory when memory usage goes over 80%.
I'll be back if I get stuck again thanks
Its homework so the more I can do without help the better

PTrenholme 04-17-2013 04:45 PM

If it's homework, I'd suggest you look at the man pages for head and tail, and eliminate the unnecessary cat <file> | cmd from your code. That idiom should be reserved for situations where it's necessary, and it a really bad habit to use it unnecessarily.

chrism01 04-17-2013 07:23 PM

Good bash tutorials/howtos
http://rute.2038bug.com/index.html.gz
http://tldp.org/LDP/Bash-Beginners-G...tml/index.html
http://www.tldp.org/LDP/abs/html/

TroubleBob 04-18-2013 09:57 AM

Code:

#!/bin/bash
NUMCORES=$(grep "^core id" /proc/cpuinfo | sort -u | wc -l)
LOADAVG=`cut /proc/loadavg -d ' ' -f 1`
PERUSAGE=`echo $LOADAVG '/' $NUMCORES '* 100 '|bc -l`
PROCESSSTRING=$(ps axo pid,args,%cpu --sort -%cpu|head -n 2|tail -n 1)
DAY=`date +%d`
MONTH=`date +%m`
YEAR=`date +%Y`
if [ `echo PERUSAGE '>80'|bc -l` -eq 1 ];then
    if [ ! -d "log" ];then
        mkdir "log"
    fi
    cd "log"
    if [ ! -d "$YEAR-$MONTH" ];then
        mkdir "$YEAR-$MONTH"
    fi   
    cd $YEAR-$MONTH
    echo $PROCESSSTRING>>$DAY.log
    cd ".."
fi

Turns out I was doing the wrong assignment :) but not too bad nearly got it finished, I just need to write code to delete the logs that are older than 14 days now, the rest of this code outputs a log showing the most CPU intensive process when the load is >80%

PTrenholme 04-18-2013 11:00 AM

Hint:
Quote:

Originally Posted by man mkdir
-p, --parents
no error if existing, make parent directories as needed

which can simplify
Code:

    if [ ! -d "log" ];then
        mkdir "log"
    fi
    if [ ! -d "$YEAR-$MONTH" ];then
        mkdir "$YEAR-$MONTH"
    fi

to
Code:

    mkdir -p log/${YEAR}-${MONTH}

TroubleBob 04-18-2013 12:14 PM

Code:

for((i=1; i<$MONTH; i++))
    do
        echo $i "loop counter"
        echo ${YEAR}-${i}
        echo ${YEAR}-"0"${i}
        rm -rf ${YEAR}-${i}
        rm -rf ${YEAR}-"0"${i}
    done

Any idea why this isn't working
In this context

Code:

#!/bin/bash
NUMCORES=`grep "^core id" /proc/cpuinfo | sort -u | wc -l`
LOADAVG=`cut /proc/loadavg -d ' ' -f 1`
PERUSAGE=`echo $LOADAVG '/' $NUMCORES '* 100 '|bc -l`
PROCESSSTRING=$(ps axo pid,args,%cpu --sort -%cpu|head -n 2|tail -n 1)
DAY=`date +%d`
MONTH=`date +%m`
YEAR=`date +%Y`
cd $HOME
#if [ `echo $PERUSAGE '>80 '|bc -l` -eq 1 ];then
    mkdir -p $HOME/log/${YEAR}-${MONTH}
    cd $HOME/log/${YEAR}-${MONTH}
    echo $PROCESSSTRING>>$DAY.log
    cd $HOME/log       
    if [ `echo $DAY '>14 '|bc -l` -eq 1 ];then
    for((i=1; i<$MONTH; i++))
    do
        echo $i "loop counter"
        echo ${YEAR}-${i}
        echo ${YEAR}-"0"${i}
        rm -rf ${YEAR}-${i}
        rm -rf ${YEAR}-"0"${i}
    done
    fi
#fi


PTrenholme 04-18-2013 02:36 PM

Not without you telling us the error(s) that code generated.

This works:
Code:

$ YEAR=2013;MONTH=5;for((i=1; i<$MONTH; i++));do echo ${YEAR}-"0"${i};echo ${YEAR}-${i};done
2013-01
2013-1
2013-02
2013-2
2013-03
2013-3
2013-04
2013-4

Another hint or two:
  1. The test function includes numeric comparison operators (e.g.: -lt -le -eq -ge and -gt). So why pipe into bc to test if [ ${DAY} -gt 14 ] ? (Beware that a leading zero will make the number octal in both bc and test.)
  2. There is a printf command (on most system) that can be used to format a number with leading zeros.
  3. The form $(cmd) is usually preferred to `cmd`, although the effect is often the same.

TroubleBob 04-18-2013 03:15 PM

There wasn't an error I was looking in the wrong place it was working, thanks for your time folks, looking forward to contributing to the forum

This is my final code
Code:

#!/bin/bash
NUMCORES=`grep "^core id" /proc/cpuinfo | sort -u | wc -l`
LOADAVG=`cut /proc/loadavg -d ' ' -f 1`
PERUSAGE=`echo $LOADAVG '/' $NUMCORES '* 100 '|bc -l`
TIME=`date +%c`
PROCESSSTRING=$TIME$(ps axo pid,args,%cpu --sort -%cpu|head -n 2|tail -n 1)
DAY=`date +%d`
MONTH=`date +%m`
YEAR=`date +%Y`
cd $HOME
if [ `echo $PERUSAGE '>80 '|bc -l` -eq 1 ];then
    mkdir -p $HOME/log/${YEAR}-${MONTH}
    cd $HOME/log/${YEAR}-${MONTH}
    echo $PROCESSSTRING>>$DAY.log
    cd $HOME/log
    if [ `echo $DAY '>14 '|bc -l` -eq 1 ];then
    for((i=1; i<$MONTH; i++))
    do
        rm -rf ${YEAR}-${i}
        rm -rf ${YEAR}-"0"${i}
    done
    cd $HOME/log/${YEAR}-${MONTH}
    for((j=1; j<($DAY-14); j++))
    do
        rm -rf $j".log"
        rm -rf "0"$j".log"
    done
    fi
fi


PTrenholme 04-18-2013 06:02 PM

You might find this interesting. Here are two re-written versions of the first five lines of the solution you posted:
Code:

$ cat TroubleBob[12].sh
#!/bin/bash
NUMCORES=$(grep "^core id" /proc/cpuinfo | sort -u | wc -l)
LOADAVG=$(cut /proc/loadavg -d ' ' -f 1)
PERUSAGE=$(echo ${LOADAVG} '/' ${NUMCORES} '* 100 '|bc -l)
TIME=$(date +%c)
echo NUMCORES=${NUMCORES}
echo LOADAVG=${LOADAVG}
echo PERUSAGE=${PERUSAGE}

#!/bin/bash
NUMCORES=$(grep "^core id" /proc/cpuinfo | sort -u | wc -l)
LOADAVG=$(cut /proc/loadavg -d ' ' -f 1)
PERUSAGE=$(printf "scale=2\n$LOADAVG/$NUMCORES* 100\n"|bc)
TIME=$(date +%c)
echo NUMCORES=${NUMCORES}
echo LOADAVG=${LOADAVG}
echo PERUSAGE=${PERUSAGE}

I ran these using the time function and got this:
Code:

TroubleBob1.sh

NUMCORES=4
LOADAVG=0.56
PERUSAGE=14.00000000000000000000

real    0m0.077s
user    0m0.005s
sys    0m0.080s
TroubleBob2.sh

NUMCORES=4
LOADAVG=0.56
PERUSAGE=14.00

real    0m0.106s
user    0m0.012s
sys    0m0.107s

<edit>
Note that I dropped the -l option from the bc command. That option is only necessary if you need transcendental functions, and it sets scale to 20.
</edit>

David the H. 04-18-2013 06:17 PM

Congratulations on figuring it out. However, there's a lot of room for improvement in your code. In particular there are a lot of unnecessary external process calls.

I've taken the liberty of re-writing it the way I would've done it. I'll just post it as-is, and hopefully you can analyze it as a learning exercise.

Code:

#!/bin/bash

IFS=- read -r year month day time < <( date '+%F-%c' )

read loadavg _ </proc/loadavg
numcores=$( grep -c '^core id' /proc/cpuinfo )
perusage=$( bc <<<"$loadavg / $numcores * 100" )
printf -v perusage '%.f' "$perusage"  #rounds it to the nearest integer

if (( perusage > 80 ));then

    cd "$HOME/log" || { echo "Can't enter log dir. Exiting." ; exit 1 ;}
    mkdir -p "$year-month"

    ps axo pid,args,%cpu --sort -%cpu --no-headers |\
      head -n 1 >> "$year-month/day.log"

    if (( ${day#0} > 14 ));then

        rm -rf $year-[1-$month] $year-0[1-$month]

        cd "$year-$month"

        (( day = day - 14 ))
        rm -f [1-$day].log 0[1-$day].log

    fi

fi

exit 0


David the H. 04-21-2013 01:46 PM

I just recognized a small flaw in my previous script.

Code:

if (( day > 14 ));then
this will not work properly in days 08 and 09, since the zero-padded numbers will be treated as octal values and error out. It's a simple fix though. Just strip off the leading zero:

Code:

if (( ${day#0} > 14 )); then
I'm updating my previous post to correct this too.


All times are GMT -5. The time now is 05:44 AM.