structure browser?
 

Long ago, back in the days of the Amiga, there was a tool called a structure browser.

Basically, it read in the C header files for the OS, parsed them, and built a display based upon them that could read the contents of all kernel structures that were in memory.

It was very cool, and was an enormously powerful tool when trying to figure out WTF was going on.

Well, something like that would be really a good thing for Linux. It would have to install as a kernel module and pass a lot of data to userspace for display. It would also represent a huge security problem but for a system under test that should be OK.

I went googling for such a thing, and drew a blank. Does anyone know of such a tool? Writing one would be a fairly substantial task, but could prove to be worth the effort.

...and after a few days there is no response.

Could it be that I have run across a tool that does not yet exist for Linux????

Now wouldn't that be cool. Actually, on the Amiga, I found the structure browser tool to be a great help in a lot of cases. Made it possible to just walk the system structures and see what was there. Clarified a LOT of things, and greatly simplified certain types of development and debugging.

I might be willing to undertake this and build this tool. It would not be a trivial job though. The most painful part of it, I think, would be writing a good parser to read the .h files, and extract all the relevant structure information from them. This information would then be used to build a display which would be populated with the values that actually exist in the system structures.

Hmmm...I think I'll try to find that amiga structure browser and see if the source is available...

Alternatively, does anyone here have a parser that would do something like this?

I've never come across such a program. It sounds like a nice idea. I suppose the /proc and /sys filesystems give you some access to kernal internals, although these are not 'as stored', nor anything like complete. The parsing does indeed sound hairy.

Hello,

That sounds like a great tool to have. Just a thought, something along these lines may be included in an IDE - if it is, you could take the source for that functionality and modify it to run as a standalone program.

Well, quite obviously there has to be a .h parser in the gcc source, but I don't know if that would exist in a form that was easily extracted, and gcc is a pretty big source tree anyway.

Hello,

Hmmm... a structure browser would certainly make short work of navigating the gcc source tree. :p

Actually, it wouldn't. The point here is to navigate structures that exist in memory, particularly in the kernel.

There would be a kernel module piece that could access the relevant kernel structures and extract the data, and a userspace piece that would tell what structure to obtain and would display the result.

Any structure members that were pointers would be clickable to get the structure pointed to, and as a result you would be able to just wander through the kernel.

I would give two start points to match the kernel abstraction model; one start point would be tasks and the other would be files. Pick your start point and browse to wherever you wanted to be in the system. Since this is a monolithic kernel, I might need a third start point to get to the kernel's own internal stuff, but that is just another detail.

There has to be a .h parser so that the structure browser is looking at the right kernel. I suppose it would run once on installation and every time the kernel was upgraded, and would then output its own config file which then would be used forever to handle this.

Problem is that this parser could be formidable to write. I would dearly love to find one that already did what I want.

Basically, the parser needs to go through all the kernel .h files (and probably .a files too), pick out the structures, and build entities (either objects or structures) that match the structures in the .h files with the requirements of a display and the need to tell the kernel driver piece what structure to access and how much data to pass back.

Thus, assume there is a kernel structure foo defined as:

struct foo {

ULONG mydata1;
int mydata2;
struct fee *myfeepointer;
}

Then the parser has to read that structure definition and fill in this structure (or object, or something similar) that is in the parser's organization:

mystructdata.structname = foo;
mystructdata.structmember = mydata1;
mystructdata.membersize = 8;
mystructdata.datatype = ULONG;

mystructdata.structname = foo;
mystructdata.structmember = mydata2;
mystructdata.membersize = 4;
mystructdata.datatype = int;

mystructdata.structname = foo;
mystructdata.structmember = myfeepointer;
mystructdata.membersize = 4;
mystructdata.datatype = pointer;

To do this, obviously, the parser has to find all the typedefs for the particular system in order to know the sizes of everything. Also, typedef struct, and union, has to be handled.

Now, when it comes time to display, the structure browser application is told to display structure foo, so it reads through its lists for all members of foo, and totals up the size of foo in bytes. It then commands the kernel driver component to return N bytes starting at the location of foo.

This also means the parser has to be smart enough to recognize that this reference in foo is a pointer:

struct fee *myfeepointer;

but this reference embeds a copy of the structure:

struct fah myfahstruct;

When the kernel driver component returns the requested quantity of data,
then the userspace app parses the binary string, filling in the values, perhaps as mystruct.value = val_as_extracted_from_returned_data, then puts up a display of that data, with any pointers being clickable.

Extracting and displaying the data is routine. The parser could take a long time to get right.

*Great* idea for an open source project!!!
 

Hi, jiml8 -

I think you've hit upon a really, really great idea, if you want to pursue it!

A couple of brief comments:
1) "Understand for C/C++" is arguably best commercial product
(at least the best one I'm aware of):

http://www.scitools.com/

2) I have no idea what scope you're looking at, or what programming languages you prefer...
... but all things being equal...
implementing a "structure browser" as an Eclipse plug-in might be an effective way to reach the largest audience.

IMHO .. PSM

PS:
If you're serious, I'd be happy to participate!

Actually, I am serious. I think it would be a very useful tool, and it apparently does not exist.

But I don't want to write the parser, which would be arduous and not particularly interesting (IMO).

The parser would only run at install time or whenever the kernel was upgraded, so it could be in any language such as perl - which has excellent string handling capability.

The kernel module would be C. The userspace command and display application could be any language, but if I do it it will be either C or C++, probably with a gtk frontend, though as I think about it, doing a variant in PHP so that it could work with a web browser could lead to some very interesting possibilities for supporting and debugging remote embedded systems.

I do not know what eclipse is so I won't comment.

Basically I am sketching out a design here, hoping someone will step up and say "I'll write the parser, if you'll do the rest of it". The kernel module and display package is IMO routine. It is the parser that makes me go "ugh".

Maybe GCC-XML would be useful?

Would ctags help with the parsing requirements?

What kind of output does it give?

I don't think so. Looks like ctags just generates an index. This is far simpler than a full-up parser that recognizes size and type as well...though I suppose I should look more closely at ctags before dismissing it.

The basic requirement for parsing was set forth in an earlier post. Does anyone here know if ctags would prove suitable for that? perhaps with relatively minor modification?

Have a read of this.

I liked the concept so much I recently even went and built a Centos system so I could have a play (I got so badly bitten by "rpm hell" a few years back I refuse to have anything to do with any distro using it).
Haven't managed to have a serious play, but this is probably as close as you're gunna get to what you are looking for.

Especially as some-one else is maintaining it for you.

Sounds like a great idea. But surely, we already have a parser, i.e. GCC. I haven't looked at the source, but if it's sufficiently modular, one could surely extract the parser and build it as "libgcc_parser.so"?