Look at Xorg, the GUI system for most of Linux/BSD. Im not overly sure how its drivers work, but thats your problem anyways (perhaps look at the source code?). For 3D, the DRI system in Linux is used, it defines the entire 3D rendering system, from in-kernel necessities, 3D video card drivers, and X glue code. Im not sure how the driver model works in current X servers, but if you look at current driver code you might be able to ask better questions.
|