LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 05-12-2008, 01:12 PM   #1
jiml8
Senior Member
 
Registered: Sep 2003
Posts: 3,171

Rep: Reputation: 114Reputation: 114
structure browser?


Long ago, back in the days of the Amiga, there was a tool called a structure browser.

Basically, it read in the C header files for the OS, parsed them, and built a display based upon them that could read the contents of all kernel structures that were in memory.

It was very cool, and was an enormously powerful tool when trying to figure out WTF was going on.

Well, something like that would be really a good thing for Linux. It would have to install as a kernel module and pass a lot of data to userspace for display. It would also represent a huge security problem but for a system under test that should be OK.

I went googling for such a thing, and drew a blank. Does anyone know of such a tool? Writing one would be a fairly substantial task, but could prove to be worth the effort.
 
Old 05-15-2008, 07:54 PM   #2
jiml8
Senior Member
 
Registered: Sep 2003
Posts: 3,171

Original Poster
Rep: Reputation: 114Reputation: 114
...and after a few days there is no response.

Could it be that I have run across a tool that does not yet exist for Linux????

Now wouldn't that be cool. Actually, on the Amiga, I found the structure browser tool to be a great help in a lot of cases. Made it possible to just walk the system structures and see what was there. Clarified a LOT of things, and greatly simplified certain types of development and debugging.

I might be willing to undertake this and build this tool. It would not be a trivial job though. The most painful part of it, I think, would be writing a good parser to read the .h files, and extract all the relevant structure information from them. This information would then be used to build a display which would be populated with the values that actually exist in the system structures.

Hmmm...I think I'll try to find that amiga structure browser and see if the source is available...

Alternatively, does anyone here have a parser that would do something like this?
 
Old 05-15-2008, 08:13 PM   #3
matthewg42
Senior Member
 
Registered: Oct 2003
Location: UK
Distribution: Kubuntu 12.10 (using awesome wm though)
Posts: 3,530

Rep: Reputation: 62
I've never come across such a program. It sounds like a nice idea. I suppose the /proc and /sys filesystems give you some access to kernal internals, although these are not 'as stored', nor anything like complete. The parsing does indeed sound hairy.
 
Old 05-16-2008, 09:22 AM   #4
JMJ_coder
Member
 
Registered: Apr 2006
Distribution: Fedora
Posts: 478

Rep: Reputation: 30
Hello,

That sounds like a great tool to have. Just a thought, something along these lines may be included in an IDE - if it is, you could take the source for that functionality and modify it to run as a standalone program.
 
Old 05-16-2008, 11:04 AM   #5
jiml8
Senior Member
 
Registered: Sep 2003
Posts: 3,171

Original Poster
Rep: Reputation: 114Reputation: 114
Well, quite obviously there has to be a .h parser in the gcc source, but I don't know if that would exist in a form that was easily extracted, and gcc is a pretty big source tree anyway.
 
Old 05-16-2008, 12:31 PM   #6
JMJ_coder
Member
 
Registered: Apr 2006
Distribution: Fedora
Posts: 478

Rep: Reputation: 30
Hello,

Quote:
Originally Posted by jiml8 View Post
Well, quite obviously there has to be a .h parser in the gcc source, but I don't know if that would exist in a form that was easily extracted, and gcc is a pretty big source tree anyway.
Hmmm... a structure browser would certainly make short work of navigating the gcc source tree.
 
Old 05-16-2008, 01:34 PM   #7
jiml8
Senior Member
 
Registered: Sep 2003
Posts: 3,171

Original Poster
Rep: Reputation: 114Reputation: 114
Quote:
Originally Posted by JMJ_coder View Post
Hello,



Hmmm... a structure browser would certainly make short work of navigating the gcc source tree.
Actually, it wouldn't. The point here is to navigate structures that exist in memory, particularly in the kernel.

There would be a kernel module piece that could access the relevant kernel structures and extract the data, and a userspace piece that would tell what structure to obtain and would display the result.

Any structure members that were pointers would be clickable to get the structure pointed to, and as a result you would be able to just wander through the kernel.

I would give two start points to match the kernel abstraction model; one start point would be tasks and the other would be files. Pick your start point and browse to wherever you wanted to be in the system. Since this is a monolithic kernel, I might need a third start point to get to the kernel's own internal stuff, but that is just another detail.

There has to be a .h parser so that the structure browser is looking at the right kernel. I suppose it would run once on installation and every time the kernel was upgraded, and would then output its own config file which then would be used forever to handle this.

Problem is that this parser could be formidable to write. I would dearly love to find one that already did what I want.

Basically, the parser needs to go through all the kernel .h files (and probably .a files too), pick out the structures, and build entities (either objects or structures) that match the structures in the .h files with the requirements of a display and the need to tell the kernel driver piece what structure to access and how much data to pass back.

Thus, assume there is a kernel structure foo defined as:

struct foo {

ULONG mydata1;
int mydata2;
struct fee *myfeepointer;
}

Then the parser has to read that structure definition and fill in this structure (or object, or something similar) that is in the parser's organization:

mystructdata.structname = foo;
mystructdata.structmember = mydata1;
mystructdata.membersize = 8;
mystructdata.datatype = ULONG;

mystructdata.structname = foo;
mystructdata.structmember = mydata2;
mystructdata.membersize = 4;
mystructdata.datatype = int;

mystructdata.structname = foo;
mystructdata.structmember = myfeepointer;
mystructdata.membersize = 4;
mystructdata.datatype = pointer;

To do this, obviously, the parser has to find all the typedefs for the particular system in order to know the sizes of everything. Also, typedef struct, and union, has to be handled.

Now, when it comes time to display, the structure browser application is told to display structure foo, so it reads through its lists for all members of foo, and totals up the size of foo in bytes. It then commands the kernel driver component to return N bytes starting at the location of foo.

This also means the parser has to be smart enough to recognize that this reference in foo is a pointer:

struct fee *myfeepointer;

but this reference embeds a copy of the structure:

struct fah myfahstruct;

When the kernel driver component returns the requested quantity of data,
then the userspace app parses the binary string, filling in the values, perhaps as mystruct.value = val_as_extracted_from_returned_data, then puts up a display of that data, with any pointers being clickable.

Extracting and displaying the data is routine. The parser could take a long time to get right.
 
Old 05-16-2008, 02:13 PM   #8
paulsm4
Guru
 
Registered: Mar 2004
Distribution: SusE 8.2
Posts: 5,863
Blog Entries: 1

Rep: Reputation: Disabled
*Great* idea for an open source project!!!

Hi, jiml8 -

I think you've hit upon a really, really great idea, if you want to pursue it!

A couple of brief comments:
1) "Understand for C/C++" is arguably best commercial product
(at least the best one I'm aware of):

http://www.scitools.com/

2) I have no idea what scope you're looking at, or what programming languages you prefer...
... but all things being equal...
implementing a "structure browser" as an Eclipse plug-in might be an effective way to reach the largest audience.

IMHO .. PSM

PS:
If you're serious, I'd be happy to participate!

Last edited by paulsm4; 05-16-2008 at 02:14 PM.
 
Old 05-16-2008, 02:38 PM   #9
jiml8
Senior Member
 
Registered: Sep 2003
Posts: 3,171

Original Poster
Rep: Reputation: 114Reputation: 114
Actually, I am serious. I think it would be a very useful tool, and it apparently does not exist.

But I don't want to write the parser, which would be arduous and not particularly interesting (IMO).

The parser would only run at install time or whenever the kernel was upgraded, so it could be in any language such as perl - which has excellent string handling capability.

The kernel module would be C. The userspace command and display application could be any language, but if I do it it will be either C or C++, probably with a gtk frontend, though as I think about it, doing a variant in PHP so that it could work with a web browser could lead to some very interesting possibilities for supporting and debugging remote embedded systems.

I do not know what eclipse is so I won't comment.

Basically I am sketching out a design here, hoping someone will step up and say "I'll write the parser, if you'll do the rest of it". The kernel module and display package is IMO routine. It is the parser that makes me go "ugh".

Last edited by jiml8; 05-16-2008 at 02:48 PM.
 
Old 05-16-2008, 03:16 PM   #10
ntubski
Senior Member
 
Registered: Nov 2005
Distribution: Debian
Posts: 2,396

Rep: Reputation: 814Reputation: 814Reputation: 814Reputation: 814Reputation: 814Reputation: 814Reputation: 814
Quote:
Originally Posted by jiml8 View Post
Actually, I am serious. I think it would be a very useful tool, and it apparently does not exist.

But I don't want to write the parser, which would be arduous and not particularly interesting (IMO).
Maybe GCC-XML would be useful?
 
Old 05-16-2008, 05:43 PM   #11
graemef
Senior Member
 
Registered: Nov 2005
Location: Hanoi
Distribution: Fedora 13, Ubuntu 10.04
Posts: 2,376

Rep: Reputation: 147Reputation: 147
Would ctags help with the parsing requirements?
 
Old 05-17-2008, 01:17 AM   #12
jiml8
Senior Member
 
Registered: Sep 2003
Posts: 3,171

Original Poster
Rep: Reputation: 114Reputation: 114
Quote:
Originally Posted by ntubski View Post
Maybe GCC-XML would be useful?
What kind of output does it give?
 
Old 05-17-2008, 01:19 AM   #13
jiml8
Senior Member
 
Registered: Sep 2003
Posts: 3,171

Original Poster
Rep: Reputation: 114Reputation: 114
Quote:
Originally Posted by graemef View Post
Would ctags help with the parsing requirements?
I don't think so. Looks like ctags just generates an index. This is far simpler than a full-up parser that recognizes size and type as well...though I suppose I should look more closely at ctags before dismissing it.

The basic requirement for parsing was set forth in an earlier post. Does anyone here know if ctags would prove suitable for that? perhaps with relatively minor modification?
 
Old 05-17-2008, 02:10 AM   #14
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 12,130

Rep: Reputation: 986Reputation: 986Reputation: 986Reputation: 986Reputation: 986Reputation: 986Reputation: 986Reputation: 986
Have a read of this.

I liked the concept so much I recently even went and built a Centos system so I could have a play (I got so badly bitten by "rpm hell" a few years back I refuse to have anything to do with any distro using it).
Haven't managed to have a serious play, but this is probably as close as you're gunna get to what you are looking for.

Especially as some-one else is maintaining it for you.
 
Old 05-17-2008, 09:39 AM   #15
ararus
Member
 
Registered: Mar 2008
Location: UK
Distribution: Slackware
Posts: 56

Rep: Reputation: 15
Sounds like a great idea. But surely, we already have a parser, i.e. GCC. I haven't looked at the source, but if it's sufficiently modular, one could surely extract the parser and build it as "libgcc_parser.so"?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
browser plugin for Konqueror web browser (linspire) Tracianddwayne Linux - Newbie 2 01-06-2010 02:08 PM
Firefox error: Error launching browser window:no XBL binding for browser JockVSJock Slackware 6 06-12-2006 10:02 PM
How to Change Default Browser to a new browser banner Linux - General 1 05-13-2005 11:09 PM
A Question on Browser Sessions, and Browser Windows: opioid Programming 1 09-23-2004 02:53 PM
gaim browser / default browser question hildog Linux - Newbie 2 10-03-2003 09:17 AM


All times are GMT -5. The time now is 02:30 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration