Im trying to understand several things regarding /proc and how I can manually calculate, directly from the files, values as they appear in ‘ps’ or ‘top’ commands.
Lets say I have a binary running, I want to know its memory & cpu usage in percentage. I have more leads regarding memory, cpu is more problematic. Lets start with the easy one :
1. Memory usage of app in percentage – I can calculate (/proc/pid/status/VmRSS) / (/proc/meminfo/MemTotal) which gives me Resident Set Size to MemTotal ratio. But as much as I understand VmRSS gives physical memory usage including shared libs, Can I calculate the size of it without shared libs? What is exactly the relation between VmRSS, VmData?

2. CPU usage of app in percentage – How can I calculate it? Are there any exceptions here, similar to the one in memory – such as VmRSS or VmSize?


I know I can get some libraries to my code and get it done quite easily (procfs for instance) but I want to understand how it is calculated directly from procfs

I don't know the specifics, but if I were to wish to tackle this I'd grab the source for top, I took a quick look and found that you can download a copy of it here. Hope that helps.

The kernel devs have attempted to resolve this by introducing pss into /proc/<pid>/smaps. How successful they were is somewhat debatable. A search on "linux pss" should be a good start to explaining it.

Ok, I understand, so if I were Compromising shared libs to be included in the calculation, is what I wrote before a good precise calculation of only physical memory used by app (including shared libs)?

Any leads how I get CPU details the same way?

see man 5 proc

cpu usage is calculated with the difference of total run time over some arbitrary time period (from /proc/[pid]/stat)

the easiest way to calculate shared memory seems to be from /proc/[pid]/statm, where the 3'rd field tells shared memory usage in pages (page=4k)
thats at least where 5min of research got me


PS
strace says htop uses /proc/pid/ io,stat,statm and /proc/ loadavg and uptime
learning strace is a Good Thing

There is no easy one.
The definition of "memory usage" is vague, and changes over time. A very good indication of how much work is involved in calculating this can be seen in ps_mem.py
Even if you don't know python, it is extremely well documented.

CPU usage is actually much easier - at any instant in time either you are using a core or you're not. Count the clock ticks that you are, divide by total ticks since start. All that is available in /proc.
But ...
- where you multi-threaded for any of that time ?. what proportion ?.
- the kernel accounts for CPU in several buckets (user, nice, sys,...) - gotta get them all.

Nothing is ever as easy as it may first appear.

I see.
I read extensively in proc man page but i still dont get the complete picture.
so far it seems that to calculate CPU of process I go it its /proc/<pid>/stat file and fetch
stime - scheduled time in system mode
utime - same in user mode
cstime & cutime - same for child processes
so I can actually sum all for of these values in some time in the system, wait a second, and then sum the new values again.
take the difference, and its the number of jiffies my process was scheduled for.
a jiffie is 1/100sec, so my cpu had in that time period 100 jiffies to schedule in, so the difference I calculated is actually cpu usage in %
but! I still dont know if these values take into consideration multi core systems, that can have several threads running in parallel.
what then?

Then you wind up with potentially more than 100.
top and such tools typically don't normalise the numbers, so results (much) more than 100% aren't considered as remarkable.

Ok, good. so Im in the correct path.
I assumed perhaps those values are per core, and then somehow I need to multiply them in cores number.
But from your words I understand that the mentioned time variables are times on all cores, together.
Thanks!

Is it possible to sample these CPU scheduled values for less than a second?
Since normally sysconf (_SC_CLK_TCK) is 100Hz, I decided to sample over a second, because a second has 100 ticks in it (1/100sec) so it gives me the correct usage, but Isnt it possible to sample over a shorter period of time?

process scheduler works in nanoseconds resolution (at least on "modern" x86)

Try using the time command. Or more specifically /bin/time which has better stats.
Be warned stopping the process created ny the time command may result in some output this may be the only output even though you go on to contine the process.