I need to use a Linux box to operate a second box, a desktop. It needs to operate in two modes. In both modes (this already works) the DVi output of the second box is displayed on the Linux console with annotations.
Mode 1) user uses mouse & keyboard to point click and type into displayed window. Input is captured—only at the level of x,y + button-press-release for the mouse, and key press-release for the keyboard, with millisecond timings—and both recorded to file and forwarded over a USB-USB cable to second box.
Mode 2) previously recorded data is played back either straight, or with dithering of x,y, timings, and possibly alterations to which button is clicked, which key is pressed.
Had been sending data over ethernet and emulating USB directly on the second box, but this only works for a couple subject second boxes (linux, apple).
I can purpose build boxes for this, but the goal is to be able to use anything as a second box and it will just see an ordinary keyboard / mouse (hopefully on a single cable). Well, the goal might be to use a Windows second box without having to touch any windows software
So what I want here is a pointer to the right drivers/packages/APIs/library or whatev?