LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 11-15-2005, 07:53 PM   #1
thanhvn
Member
 
Registered: Mar 2005
Location: CA
Distribution: RHEL3, FC4
Posts: 46

Rep: Reputation: 15
how to programmatically monitor a process memory usage?


I need to obtain the memory usage (real and virtual) of a particular process every couple of seconds and write it to a log file. I'm thinking of writing either shell script (sh or bash) or a C/C++ program. What shell commands and/or C/C++ API's are available for this task?

Thanks,
 
Old 11-15-2005, 10:13 PM   #2
twantrd
Senior Member
 
Registered: Nov 2002
Location: CA
Distribution: redhat 7.3
Posts: 1,438

Rep: Reputation: 52
'bash' and 'ps' can do this:

Code:
twantrd:~# ps -o pid,vsz,rss,pmem,time,comm
  PID   VSZ  RSS %MEM     TIME COMMAND
  990  2580 1448  0.3 00:00:00 bash
 1205  1768  988  0.2 00:00:00 man
 1210  2272 1000  0.2 00:00:00 sh
 1214  1836  660  0.1 00:00:00 pager
 1281  2288  680  0.1 00:00:00 ps
This displays more than what you have asked, just giving you an idea. You can use this and pipe a grep to search for the process that you want. To capture this every couple of seconds, I would put this in an infinite while loop and sleep for X seconds and run the command.

-twantrd

Last edited by twantrd; 11-15-2005 at 10:16 PM.
 
Old 11-15-2005, 10:52 PM   #3
primo
Member
 
Registered: Jun 2005
Posts: 542

Rep: Reputation: 34
ps uses values from /proc/<pid>/stat
 
Old 11-16-2005, 02:45 AM   #4
thanhvn
Member
 
Registered: Mar 2005
Location: CA
Distribution: RHEL3, FC4
Posts: 46

Original Poster
Rep: Reputation: 15
I am wondering:

1) how to get this GetMemUsage script and the program it monitors to start up at the same time? I don't want to start up the program then open a second console (the program writes to stdout) to launch the script. This means I would miss the first second or two of the program.

2) how to get this GetMemUsage script to shut down cleanly and automatically when the program it monitors shutdowns, even it shuts down abnormally? I would like the ability to batch run the program several times.

Thanks,
 
Old 11-16-2005, 02:54 AM   #5
thanhvn
Member
 
Registered: Mar 2005
Location: CA
Distribution: RHEL3, FC4
Posts: 46

Original Poster
Rep: Reputation: 15
Also, how to get the elapsed time since the program started (preferrably in milliseconds or better resolution)? other than saving the start time and subtract it from the current time? what if I wanted CPU time and not wall-clock time?

Thanks,
 
Old 11-21-2005, 04:13 PM   #6
thanhvn
Member
 
Registered: Mar 2005
Location: CA
Distribution: RHEL3, FC4
Posts: 46

Original Poster
Rep: Reputation: 15
Here's what I've got so far. This is the GetMemUsage script:

Code:
#!/bin/sh

USAGE="Usage: $0 processName"

if [ $# -ne 1 ]; then
   echo $USAGE
   exit 1
fi

# In case the monitored process has not yet started
# keep searching until its PID is found
PROCESS_PID=""
while :        
do
   PROCESS_PID=`/sbin/pidof $1`

   if [ "$PROCESS_PID.X" != ".X" ]; then
      break
   fi
done

LOG_FILE="memusage.csv"

echo "ElapsedTime,VmSize,VmRSS" > $LOG_FILE

ELAPSED_TIME=0
PERIOD=2        # seconds

# Monitor memory usage forever until script is killed
while :        
do
   VM_SIZE=`awk '/VmSize/ {print $2}' < /proc/$PROCESS_PID/status`
   if [ "$VM_SIZE.X" = ".X" ]; then
      continue
   fi
   VM_RSS=`awk '/VmRSS/ {print $2}' < /proc/$PROCESS_PID/status`
   if [ "$VM_RSS.X" = ".X" ]; then
      continue
   fi
   echo "$ELAPSED_TIME,$VM_SIZE,$VM_RSS" >> $LOG_FILE
   sleep $PERIOD
   VM_SIZE=""
   VM_RSS=""
   # Needs to get actual elapsed time instead of doing this
   ELAPSED_TIME=`expr $ELAPSED_TIME + $PERIOD`
done
Anyone knows how to get the actual elapsed time of a process (preferrably in milliseconds resolution but seconds resolution is fine)?

I cannot think of any way to have the GetMemUsage script launches and shuts down in synchronization with the monitored process other than to have a launch script that launches and shuts down both processes. Here's the launch script:

Code:
#!/bin/sh

Cleanup() {
   kill -9 `/sbin/pidof hello.out`
   kill -9 `ps ux | awk '/GetMemUsage.sh/ {print $2}'`
   exit $1
}
trap 'Cleanup 1' 1 2 3 15
./GetMemUsage.sh hello.out 2>&1 > /dev/null &
./hello.out
Cleanup 0
I realized this is not the most elegant way of doing it. Any suggestions for improvements are more than welcome.
 
Old 07-12-2007, 07:43 AM   #7
ptobra
LQ Newbie
 
Registered: Jun 2007
Posts: 11

Rep: Reputation: 0
Monitoring the memory consumption of a process

How do i capture the memory usage of that particular process running in multiple threads. In linux every thread is displayed in the top statistics.
In solaris, the case is different it shows the number of threads as NLWP but one process.

Any clues. Can we do it using ps but if it list the /proc/pid/status

How can I get the memory consumption of a process in such case. Any ideas and suggestions.

Thanks in advance
 
Old 07-12-2007, 07:08 PM   #8
ta0kira
Senior Member
 
Registered: Sep 2004
Distribution: FreeBSD 9.1, Kubuntu 12.10
Posts: 3,078

Rep: Reputation: Disabled
Have you looked at man proc? That has a lot of info regarding how to interpret files in /proc/pid. I think everything you are looking for will be in /proc.

Are you talking about computer shutdown, or program exit when the monitored program exits?
ta0kira
 
Old 07-13-2007, 12:46 AM   #9
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,289

Rep: Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034
ptobra, what Linux kernel are you using.?
I did some dev in Perl thrs and found that 2.4.x Linux kernels show each thr as a separate process, but 2.6.x kernel shows only 1 process.
iirc, it's due to a change in the underlying kernel code.
 
Old 07-13-2007, 05:50 AM   #10
ta0kira
Senior Member
 
Registered: Sep 2004
Distribution: FreeBSD 9.1, Kubuntu 12.10
Posts: 3,078

Rep: Reputation: Disabled
Quote:
Originally Posted by chrism01
ptobra, what Linux kernel are you using.?
I did some dev in Perl thrs and found that 2.4.x Linux kernels show each thr as a separate process, but 2.6.x kernel shows only 1 process.
iirc, it's due to a change in the underlying kernel code.
Threads are shown under /proc/#/task in 2.6 and have most of the same info available as the main process.
ta0kira
 
Old 07-15-2007, 02:49 AM   #11
ptobra
LQ Newbie
 
Registered: Jun 2007
Posts: 11

Rep: Reputation: 0
yes chris, you are right,
I am using Kernel 2.5.x only. I am still searching for a good solution.

Taokira, my problem is the same as chris mentioned. Every thread is shown as a process, so which pid should i use for capturing memory details.

Thanks
ptobra
 
Old 07-15-2007, 08:42 PM   #12
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,289

Rep: Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034
If the mem is shared between thrs (it might be/ can't remember), then choose the main/parent thr.
Otherwise, you'd have to sum all thrs mem values.
One way to check is to run a test on a quiet system, and observe the top values when prog is running vs not running.
Incidentally, iirc, only even-num (2.4, 2.6) kernels are prod ready, odd nums (2.5, 2.7) are dev/test ???

Last edited by chrism01; 02-20-2009 at 06:13 AM.
 
Old 07-15-2007, 11:42 PM   #13
ptobra
LQ Newbie
 
Registered: Jun 2007
Posts: 11

Rep: Reputation: 0
I think getting the pid of parent thread and observing is most appropriate.
Summing up all thread mem values and the calculating the average also makes sense.

Thanks chris
 
Old 09-09-2007, 10:18 AM   #14
chandan_maddanna
LQ Newbie
 
Registered: Sep 2007
Posts: 6

Rep: Reputation: 0
Hi Guys, i have got a bit better solution

Hey All,

We can use smap to get accurate memory consumption , rather than using VMSIZE or RSS as above.

check these links out, i have just compiled.

Per process accurate memory usage , private region , public region and shared library page mapping included. better than VMSIZE and RSS values.

http://unixlive.editboard.com/Genera...ral-p14.htm#14

How much RAM / Memory does a program use. Note that this takes care of parent process etc, mentioned above, and gives usage for a program ( not a process ). so this is really helpful.


http://unixlive.editboard.com/Genera...program-t5.htm


Regards,

-- Chandan

Quote:
Originally Posted by ptobra View Post
I think getting the pid of parent thread and observing is most appropriate.
Summing up all thread mem values and the calculating the average also makes sense.

Thanks chris
 
Old 02-20-2009, 01:15 AM   #15
thinker0
LQ Newbie
 
Registered: Feb 2009
Posts: 1

Rep: Reputation: 0
MultiThread process... patch

Code:
#!/bin/sh

USAGE="Usage: $0 processName"

if [ $# -ne 1 ]; then
   echo $USAGE
   exit 1
fi


LOG_FILE="memusage.csv"

echo "ElapsedTime,VmSize,VmRSS" > $LOG_FILE

ELAPSED_TIME=0
PERIOD=1        # seconds

# Monitor memory usage forever until script is killed
while :
do
   SUM_VM_SIZE=0
   SUM_RSS_SIZE=0
   # In case the monitored process has not yet started
   # keep searching until its PID is found
   PROCESS_PIDS=""
   while :
   do
      PROCESS_PIDS=`/sbin/pidof $1`

      if [ "$PROCESS_PIDS.X" != ".X" ]; then
         break
      fi
   done

   for PID in ${PROCESS_PIDS} ; do
     VM_SIZE=`awk '/VmSize/ {print $2}' < /proc/$PID/status`
     if [ "$VM_SIZE.X" = ".X" ]; then
        continue
     fi
     #echo exprVM_ $SUM_VM_SIZE + $VM_SIZE
     SUM_VM_SIZE=`expr $SUM_VM_SIZE + $VM_SIZE`

     VM_RSS=`awk '/VmRSS/ {print $2}' < /proc/$PID/status`
     if [ "$VM_RSS.X" = ".X" ]; then
        continue
     fi
     SUM_RSS_SIZE=`expr $SUM_RSS_SIZE + $VM_RSS`
   done
   echo "$ELAPSED_TIME sec, $SUM_VM_SIZE KB, $SUM_RSS_SIZE KB"
   echo "$ELAPSED_TIME,$SUM_VM_SIZE,$SUM_RSS_SIZE" >> $LOG_FILE
   sleep $PERIOD
   VM_SIZE=""
   VM_RSS=""
   # Needs to get actual elapsed time instead of doing this
   ELAPSED_TIME=`expr $ELAPSED_TIME + $PERIOD`
done

Last edited by thinker0; 02-20-2009 at 01:34 AM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
how to show the memory usage per process ty263 Linux - General 3 09-12-2008 06:36 AM
how to programmatically monitor a process memory usage? thanhvn Linux - Software 1 11-15-2005 08:44 PM
about displaying process memory usage maginotjr Slackware 2 09-13-2005 11:43 AM
Process memory usage wombat53 Linux - Newbie 5 07-21-2005 07:42 PM
limiting cpu and memory usage by user process amitkush Linux - Security 2 07-04-2003 12:21 AM


All times are GMT -5. The time now is 06:10 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration