I am looking for how the files in the /proc dir get their information. Things like disk and network I/O, cpu load, and other information. How is that updated? I'm trying to write a script or program to pick up all the information for me, and since the /proc file can vary I want to get information from wherever it gets its information from.

If anyone knows where some documentation on it is, or can point me in the right direction it would be greatly appreciated.

Thanks,
Michael

The /proc directory gets its information from the kernel data structures. The files in there are a live view of those items in the kernel. Several command line utilities get their information from the same kernel data structures. That is why looking at files in the /proc directory can be just like running one of these utilities. Some of these utilities include vmstat, iostat, and lsmod. I'm not saying that these utiltiies read their information from the /proc directory. I'm just saying that much of the same information displayed by these utilities is also available in the /proc directory.

they do read their information from the /proc directory
a little snipit of code from lsmod
/proc is there to provide userland access to things that are generally unavailable to userspace. but there are no doubt some userspace syscalls that return some of the data represented in the /proc filesystem. Beyond that you would have to be in kernel space to have access to the data.

/proc is not just for reading . you can actually alter the kernel data structures by writing to the /proc filesystem as well.

If I correctly remember (and understand) what I have read, /proc does not refer to any actual files on disk---rather it is a "hook" to allow you to see what the kernel has stashed in RAM. Consistent with the Unix mantra that "everything is a file", you access the data as if it were a file. Actually, it is in RAM.

You're right pixellany. /proc is a "pseudo-filesystem". They're not real files, just representations of what your kernel has stored in memory. See also stress_junkie's post.

alright, this is all really cool information, thank you everyone who has posted. A few questions, if I were to write a script to collect data from the /proc dir and display everything that I wanted in a format that I like (so that I wouldn't be using lspci, uptime, ect. to find the information that I want) would that be portable across all linux platforms, or do some distros set their /proc dir up differently then others. And I was kind of wondering how the linux kernel wrote the information to the /proc dir, so that I could understand more about it in AIX, as I've found the /proc in there is very different, and maybe knowing more in depth details about the Linux /proc would help me figure it out for other Unix systems. Thanks again for all the input.

alright, this is all really cool information, thank you everyone who has posted. A few questions, if I were to write a script to collect data from the /proc dir and display everything that I wanted in a format that I like (so that I wouldn't be using lspci, uptime, ect. to find the information that I want) would that be portable across all linux platforms, or do some distros set their /proc dir up differently then others. And I was kind of wondering how the linux kernel wrote the information to the /proc dir, so that I could understand more about it in AIX, as I've found the /proc in there is very different, and maybe knowing more in depth details about the Linux /proc would help me figure it out for other Unix systems.

How do kernel data structures work? I get the feeling that the answer to that question isn't a short one, if it isn't and you don't want to explain it is there some good documentation that you could recommend?

Thanks again for all the input.

Are you asking what is a data structure? It is just a definition of how a data set is stored. Simple example: I'm going to transmit (store) a 32-bit segment (four bytes). Byte 1 is an ascii character, and bytes 2-4 are 3 8-bit integers whose value is derived from three voltages that I measured.
Real-world data structures are vastly more complex and may contain a mix of integer, floating-point, ascii, and single-bit flags.

Simpler answer?? Data structures are the means by which the kernel knows where it put a piece of data so it can find it again.

The information provided in /proc does change between kernel versions, and may also change depending on which drivers you have loaded and/or compiled into your kernel. For example, if your system uses only SCSI or SATA and you therefore don't have IDE drivers loaded, /proc/ide shouldn't exist. For your script to work across different distributions (and indeed, different versions and installations of the same distribution), it should be prepared to fail gracefully if some piece of information is missing.


The information is never 'written' to the /proc dir. Actually, whenever you read some data from the filesystem, the request goes to the kernel, and it is the kernel that reads the data from the appropriate place, then returns it to userspace. In the case of /proc, instead of reading the data from disk the proc fs driver looks up the appropriate information in the kernel, and returns it as if it were a real file.

I can explain the way filesystems work in Linux in more detail if you want, but the above should give you an adequate understanding of what's going on here. I don't know the specifics of /proc in AIX or other Unices, but I imagine the above is true of them too and that only the contents of the directory are different.


Are you a C programmer to at least an intermediate level? It would be hard to imagine any answer to your question being very useful unless the answer to mine is 'yes' .

That answers my question, thank you, I thought all the files in /proc were getting their information from some other step and weren't in real time basically. I guess I still have a question as to what goes into the files in /proc getting their information, to try to work around it, but it doesn't look like that would work too well and I'm better off with using awk on what is already in /proc

pixellany's definition kind of reminded me of what data structures are in C code, though that is the part in C that I started getting really confused so I don't really know much about structures in C if that is why you were asking if I'm a C programmer (and the answer is "no" I'm not at least an intermediate level C programmer).

Thanks to everyone, all the responses have showed me some of the holes in my attack plan, and I now have a much better idea of how the /proc dir works and what I can do with it.

if you wish to understand the /proc filesystem as its used from within kernel space
./include/linux/proc_fs.h is the interface
and there is documentation in
./Documentation/DocBook/procfs_example.c
./Documentation/DocBook/procfs-guide

and like that

They aren't files--they are only virtual files. The data is stored in RAM by the kernel as it executes various processes. To understand in more detail you would have to take apart what the kernel is doing when it runs a particular process.

A few days with the kernel source code will give you some clues---hopefully without causing irreversible brain damage in the bargain....

For many things in /proc, /proc is the only interface the kernel provides to get hold of that information, so you'll have to use it for those . For the other stuff I can't imagine any reason why it'd be worth the extra trouble.

I don't know if I was reading it correctly but it looked to me like the procfs-guide was describing how to make your own /proc calls. It was mostly way over my head, but interesting.

It looks like the best way is just to fish things out of the /proc, then if I hit any problems with different kernels or anything just to modify the script. I think I'll work with that for now.

Thanks to everyone for all your help.
-Michael