Read pixel in Linux
 

What's the best (meaning easiest way, and the fastest way) to read a pixel when running an x server? Under windows, I was able to drive a gui, watch the screen to react to events (much like the game botters do) using pixelchecksums and getpixelcolors. In my case, I have to watch and automate a java app, that I don't have the source to as part of my job. It's a crippled third party vendor app, and we're forced to use it -- you know the rest of the story with those :-). At any rate, under windows "reading" the screen was easy, but under X11, I don't find the same thing. I considered using everything from python to java. My windows code was actually C++, and later autohotkey. I've looked at xmacro, andriod, expect, etc, but none really give you that low level access I'm after (they use the XTEST extension which states it's not to be used to automate guis). After alot of searching this led to using Xlib directly. Is this the best way? And are there other wrappers for Xlib besides the orginal C? I know of python-xlib wrapper as well, but that's it.

I believe Xlib will do the job, yes. You need to be careful of the window coordinates though. If you literally want a pixel from the screen (an absolute position relative to the (say) 1024x768 background) you will be looking at the "root" window; if you want a pixel from an application window, and you know which window!, then you may want to give window-relative coordinates, which have the 0,0 origin inside the Window Manager's title bar.

If you're grabbing large areas of the screen, don't do it a pixel at a time (too much comms with the server). Xlib has functions to grab an area I think.

All the common C/C++ GUI toolkits wrap Xlib, AFAIK - eg. Qt, GTK. The KDE toolkit builds on Qt.

I had a brief look to see whether Qt provides an API to read a pixel, but I could only find a method in the class that handles graphical images. http://doc.trolltech.com/3.3/classes.html

HTH,
-nick

Back when I was doing such hacks, I found the frame buffer very easy to use. One call (IIRC, fb_open under Solaris), and you get a struct with a pointer to the video memory, and other useful info.
No messing with X and toolkits here -- but you may need to do that if you care where a particular window lies on your screen.

A completely different approach: Provide an instrumented environment (Drawable etc.) to your java app. I'm not sure how easy it will be to architect, but if you can intercept, e.g., calls to draw and have them do what you need (instead of painting pixels), that might be more efficient than polling pixels off the screen.

Thanks! Can you enlighten me a little more with the framebuffer approach? And could someone provide a link with an up to date API that can do it with XLIB? I too looked for it in GTK and came up short. Yes, I am looking to grab a small chunk of pixels in a few locations. I'll let everyone know if I find anything - glad I'm on the right track with using XLib

Here's an Xlib link:
http://www.ac3.edu.au/SGI_Developer/...b_PG/sgi_html/

Sounds like XGetImage will help.

I think the frame buffer approach and Xlib are quite different. I don't know much about using a frame buffer, but as I understand it, this accesses the hardware "directly" on the machine, whereas the X protocol is designed to operate a display (or many displays) that may be physically remote from where the program is. That makes X slower, but potentially more flexible. Though this isn't likely to be a big issue - most Linux X programs assume run on the same machine as the display.

guitara,
I googled a bit, and it seems a frame buffer in Linux is much more complicated than in Solaris.

In Solaris, you get a pointer p and you can read or write the pixel (x,y) as p[x+linesize*y]. Real easy.

In Linux, things seem geared toward efficiency; people are concerned with leveraging hardware acceleration, picosecond timing of the CRT signal, sound, graphical input devices, image formats, ...

Sorry, you may need to do your own research. I bet there's a simple API somewhere, but I didn't find it right away.

thanks - that's along the lines of what i was looking for. A toolkit for this would be nice, but as such it's a nice reference. I also found this - which may or may not work out. It's a X virtual frame buffer. FUN FUN!
http://www.xfree86.org/4.0.1/Xvfb.1.html