Input event processing in X

The terms Core Pointer, VCP, extension devices, etc. are thrown around much these days. But they either mean very little or just very little to most. Especially because since X server 1.7 and to a lesser part 1.6, the definitions have changed a lot. So here's a bit of a high-level overview of what happens with input events in the server.

X input drivers

The first interesting point is probably that drivers these days aren't really drivers any more. The most common drivers, evdev and synaptics don't have hardware-specifics anymore, they rely on the (Linux) kernel to provide generic events. Thus, if a device isn't handled by evdev that's hardly ever an evdev bug - it usually means that the device isn't handled by the kernel (yet). Same goes for synaptics, if a new touchpad doesn't work it usually isn't supported by the kernel. The situation is different on Solaris and the BSDs, where the mouse/kbd drivers as well as synaptics have hardware-specific protocol parsers.

But on Linux, X input drivers are more sets of features than actual hardware drivers. evdev will happily manage your touchpad but you'll miss out on two-finger scrolling and other synaptics features.

X input devices

Any physical input device that's exported by the kernel as a /dev/input/eventX device file is usually added to the X server as an input device. For example on my box here I have the following devices:

:: whot@quokka:~> xinput --list --short
⎡ Virtual core pointer                     id=2 [master pointer  (3)]
⎜   ↳ Virtual core XTEST pointer               id=4 [slave  pointer  (2)]
⎜   ↳ Microsoft Microsoft® Digital Media Keyboard id=8 [slave  pointer  (2)]
⎜   ↳ TPPS/2 IBM TrackPoint                    id=13 [slave  pointer  (2)]
⎜   ↳ Macintosh mouse button emulation         id=14 [slave  pointer  (2)]
⎜   ↳ SynPS/2 Synaptics TouchPad               id=15 [slave  pointer  (2)]
⎜   ↳ Microsoft Microsoft 5-Button Mouse with IntelliEye(TM) id=16 [slave  pointer  (2)]
⎜   ↳ Wacom Intuos4 6x9 eraser                 id=17 [slave  pointer  (2)]
⎜   ↳ Wacom Intuos4 6x9 cursor                 id=18 [slave  pointer  (2)]
⎜   ↳ Wacom Intuos4 6x9 pad                    id=19 [slave  pointer  (2)]
⎜   ↳ Wacom Intuos4 6x9                        id=20 [slave  pointer  (2)]
⎣ Virtual core keyboard                    id=3 [master keyboard (2)]
    ↳ Virtual core XTEST keyboard              id=5 [slave  keyboard (3)]
    ↳ ThinkPad Extra Buttons                   id=6 [slave  keyboard (3)]
    ↳ AT Translated Set 2 keyboard             id=7 [slave  keyboard (3)]
    ↳ Microsoft Microsoft® Digital Media Keyboard id=9 [slave  keyboard (3)]
    ↳ Sleep Button                             id=10 [slave  keyboard (3)]
    ↳ Video Bus                                id=11 [slave  keyboard (3)]
    ↳ Power Button                             id=12 [slave  keyboard (3)]

At the bottom of the list we have devices like the Power Button and the Video Bus - they look like keyboards to us so they get added (one can usually match up /proc/bus/input/devices with the X device list). Note that it is the configuration deciding which devices get added, not the server itself. The default configurations we ship in the distros simply add all /dev/input/eventX files, hence the exhaustive list. It is possible to blacklist devices as well.

The actual input devices in the list above are obvious as well, the touchpad, the trackpoint, a wacom tablet and a mouse and two keyboards. From the X server's point of view, the touchpad is identical to a mouse btw., the fact that other drivers are used doesn't matter to the server.
This is my everday laptop and I have 18 devices, that's quite a bit. Our current maximum is 40 but that's just an arbitrary define that can be increased to 128. Once that level is reached, things get more complicated but I'll skip on the details here. For now, we're safe.

ID numbering starts at 2 for internal reasons, and in our implementation devices 2, 3, 4 and 5 are hardcoded and always present. More on that later. 0 and 1 are reserved for XI2, if you've programmed with it you may remember the XIAllDevices and XIAllMasterDevices defines. They are treated like devices internally.

What you can also see in this list is that the server differs between pointers and keyboards. This is an unfortunate but required distinction and we have code all over the place to handle pointer events from keyboards and the other way round. There are also plans to work around this distinction in evdev, but that requires some API work in the server.

Slave devices

What really matters in the server is not the physical devices but the so-called slave devices. This is a term that got introduced by the MPX work pre server 1.7.

• A physical device usually corresponds to one or more slave devices. There are two exceptions here: the wacom X11 driver creates multiple X devices from a single device file. The other exception are physical devices that are split into multiple kernel devices. For example, all touch devices that need the kernel's multi-device quirk show up as several /dev/input/eventX devices and thus are added as multiple X devices. Nonetheless, for most devices, the physical device == slave devices is valid.


• A slave device may not be a physical device at all (e.g. "Virtual core XTEST pointer", "Virtual core XTEST keyboard").


• The server may at any point in time have zero or more slave devices. Our current implementation has two hardcoded ones (the XTEST pointer and XTEST keyboard) but they are not required by the protocol.


• In the default configuration, all slave devices are hotplugged. Even when they're already present at server startup, there's no difference in the code path.


• A slave device may be either attached or floating. More on that later.


• Any slave device, once enabled, may generate events, but only X Input Extension events, both for versions 1.x and XI2. More on that later.

The last point is interesting for clients - if your client uses the core input API (e.g. XSelectInput(3)), then slave devices are invisible to the client. Currently, virtually all clients use core input, a situation that hopefully changes in the future. GTK for example will support XI2 in its next version.

Master devices

For indirect input devices like a mouse, the virtual representation of the device is as important as the device. We tend to think of "moving the cursor" as equivalent as "moving the mouse", those to concepts are quite ingrained. (Note that in this article, the term "cursor" applies to the concept, not to the shape). The same goes for the keyboard focus, but both cursor and focus are just abstract virtual representations. We represent them as devices as well, they are so-called master devices.

In the device list above, you can see two master devices - one pointer, one keyboard. Other facts about master devices are:

• Master devices always come as pairs, one pointer and one keyboard. Internally, the devices know the respective other device in the pair.


• Our implementation has two master devices hardcoded, the "virtual core pointer" and the "virtual core keyboard". More on that later.


• Master devices do not generate events by themselves, they rely on slave devices to generate events for them (more on that later).


• A master device may have zero or more slave devices attached.


• You may have more than one pair of master devices - this is the essence of MPX.


• Master devices may be created or destroyed by a client at runtime. Likewise, the attachment of slave devices can be manipulated at runtime.


• Master devices may send core events and/or X Input Extension events (1.x and XI2). Only the first pair of master devices is visible to X Input 1.x clients. This is a protocol detail that I won't elaborate on here.

Now the terminology of attached and floating on slave devices may be a bit clearer, a floating slave is simply not attached to a master device. That affects event delivery, more on that later.

Again, the last point is of interest to the clients again - core events always and only come from a master device. These are the devices a client grabs when it grabs the pointer or the keyboard. These are the devices whose state is returned when the client asks for the modifier key state. And because core clients assume only one pointer and only one keyboard, the server has some elaborate tricks to make a client believe there's only one pair, even when we have multiple master devices present.

Event generation

Event generation is the first half of the event processing. When the kernel device file has new information for us, the server receives a SIGIO and calls into the driver. The input driver reads this information off, processes it and "posts" it up as key, button or motion data. The server then converts the data into a internal event format and puts it onto the event queue. Once that's done, the signal handler returns and processing continues as usual.

Depending on the driver, zero or more events may be posted for each kernel event. For example, the evdev mouse wheel emulation converts motion data into several button press and release events. Same with the synaptics driver that converts position data into button events for edge scrolling.

Event delivery

During event delivery, the events are taken out of the event queue and processed one-by-one in the order they were received. The event delivery itself is staged into slave delivery and master delivery.

As said above, an event may only be generated by a slave device. Once that happens, the event is processed for this slave device and delivered to any listening clients - if there are any. Remember, these clients would have to be XI 1.x or XI2 aware to get events. Regardless of the event delivery, the same event is then processed again but this time for the respective master device. Obviously, if the slave device is floating this stage is skipped. Looking at the device list above, whenever I press a button on the "SynPS/2 Synaptics TouchPad", this event is also delivered through the "Virtual core pointer".

When the event passes through the master device, it updates the master's internal state and is then delivered to the client. Delivery happens either as core event, as XI 1.x event or as XI2 event. The same event may be delivered to multiple clients (for example if two or more clients registered the same event mask on the same window), but once delivered the event cannot be delivered as a different event category.

Master devices are the union of the respective slave devices. Thus, if you have two mice attached to the same master device and press the first button on both, the button won't be released on the master until all mice have released that button. Likewise, if you press the left button on mouse A and the right button on mouse B, the master device appears to have both buttons down.

What about the CorePointer/CoreKeyboard?

Back before server 1.4, the CorePointer option was a common configuration option. The XI 1.x protocol specification requires that one pointer and one device be deemed the "core" devices, thus only able to generate core events and not XI 1.x events. All other devices may generate XI 1.x events but not core events. Configuring a device as CorePointer simply said: this device controls the cursor.
Daniel Stone's hotplugging work in 1.4 introduced the "Virtual core pointer" and the "Virtual core keyboard". Instead of a physical device being a core pointer, these hardcoded virtual devices became the default core pointer and keyboard, allowing all physical devices to be used as XI extension devices. Note that only XI extension devices provide for more than two axes, so e.g. pressure is impossible with the core protocol. All MPX did was to allow for more than one pair of these hardcoded virtual devices (surprisingly, that's not quite that trivial).

In the old system, there was the option of "SendCoreEvents". This option worked around extension devices not being allowed to generate core events. For any device that "SendCoreEvents", the core pointer would generate a core event whenever the original device generated an event. You may notice that this sounds quite similar to the staged event delivery and in fact it is (the implementation today is very different though). These days the SendCoreEvent option is only useful if you want to turn it off. And if that's the case, the device is simply set floating at server startup - because as said above, floating devices do not pass their events to the master device and thus do not send core events.

Finally, before 1.7 drivers could manipulate the "SendCoreEvents" setting in the device. This is not possible anymore, whether a device sends core events is only decided based on whether it is attached or not.

XTEST devices

The last two devices that may need explanation are the XTEST devices. They are hardcoded for each master device and serve as input devices for the XTEST extension. This extension allows clients to trigger pointer or keyboard event just as if a physical device has generated them. Whenever a client calls XTestFakeButtonEvent(3), it's the XTEST pointer that presses a virtual button down, generating an event on the master device.

Handling multiple master devices

I've explained this in the past but I want to touch on this again to have everything in one article to link to. The ClientPointer is a new principle introduced with MPX.

Many protocol requests ask for information or action on the "pointer" or on the "keyboard". With XI 1.x, this meant the core pointer and the core keyboard, respectively. With XI2/MPX in the server, there may be more than one core pointer or keyboard at any time. So when a client asks "where is the cursor", we need to reply with one of them. The ClientPointer is simply a reference to a master pointer device that each client holds. Whenever device information is requested but no device is specified, the server will take the ClientPointer (or the paired master keyboard). The general rule is - unless you reassign the ClientPointer (XSetClientPointer(3)), all applications will get information from the first pointer and keyboard device. There are a few tricks in the server though to dynamically reassign ClientPointers during grabs. This greatly improve usability on legacy desktops and is one of the reasons why you can - on a standard desktop - use two applications simultaneously with only minor glitches.

A simplified summary

• Each physical device is added as a so-called "slave device" to the server.


• Slave devices are attached to so-called "master devices".


• A master pointer is equivalent to a cursor, a master keyboard is equivalent to a keyboard focus.


• When a slave device generates an event, this event is sent through the master device as well.


• A master device is the union of all attached slave devices.


• Legacy clients only receive events from master devices.


• There is no concept of a "core pointer" or "core keyboard" anymore, the master devices have taken over this role.