Multitouch in X - Pointer emulation

This post is part of a series on multi-touch support in the X.Org X server.

1. Multitouch in X - Getting events
2. Multitouch in X - Pointer emulation
3. Multitouch in X - Touch grab handling
4. Multitouch in X - Multitouch-touchpads

In this post, I'll outline how pointer emulation on touch events works. This post assumes basic knowledge of the XI2 Xlib interfaces.

Why pointer emulation?

One of the base requirements of adding multitouch support to the X server was that traditional, non-multitouch applications can still be used. Multitouch should be a transparent addition, available where needed, not required where not supported.

So we do pointer emulation for multitouch events, and it's actually specified in the protocol how we do it. Mainly so it's reliable and predictable for clients.

What is pointer emulation in X

Pointer emulation simply means that for specific touch sequences, we generate pointer events. The conditions for emulation are that the the touch sequence is eligible for pointer emulation (details below) and that no other client has a touch selection on that window/grab.

The second condition is important: if your client selects for both touch and pointer events on a window, you will never see the emulated pointer events. If you are an XI 2.2 client and you select for pointer but not touch events, you will see pointer events. These events are marked with the XIPointerEmulated so that you know they come from an emulated source.

Emulation on direct-touch devices

For direct-touch devices, we emulate pointer events for a touch sequence provided the touch is the first touch on the device, i.e. no other touch sequences were active for this device when the touch started. The touch sequence is emulated until it ends, even if other touches start and end while that sequence is active.

Emulation on dependent-touch devices

Dependent touch devices do not emulate pointer events. Rather, we send the normal mouse movements from the device as regular pointer events.

Button events and button state

Pointer emulation triggers motion events and, more importantly, button events. The button number for touches is hardcoded to 1 (any more specific handling such as long-click for right buttons should be handled by touch-aware clients instead), so the detail field of an emulated button event is 1 (unless the button is logically mapped).

The button state field on emulated pointer events adjusts for pointer emulation as it would for regular button events. The button state is thus (usually) 0x0 for the emulated ButtonPress and 0x100 for the MotionNotify and ButtonRelease events.

Likewise, any request that returns the button state will have the appropriate state set, even if no emulated event actually got sent.

Grab handling works as for regular pointer events, though the interactions between touch grabs and emulated pointer grabs are somewhat complex. I'll get to that in a later post.

The confusing bit

There is one behaviour about the pointer emulation that may be confusing, even though the specs may seem logical and the behaviour is within the specs.

If you put one finger down, it will emulate pointer events. If you then put another finger down, the first finger will continue to emulate pointer events. If you now lift the first finger (keeping the second down) and put the first finger down again, that finger will not generate events. This is noticable mainly in bi-manual or multi-user interaction.

The reason this doesn't work is simple: to the X server, putting the first finger down just looks like another touchpoint appearing when there is already one present. The server does not know that this is the same finger again, it doesn't know that your intention was to emulate again with that finger. Most of the semantics for such interaction is in your head alone and hard to guess. Guessing it wrong can be quite bad, since that new touchpoint may have been part of a two-finger gesture with the second finger and whoops - instead of scrolling you just closed a window, pasted your password somewhere or killed a kitten. So we err on the side of caution, because, well, think of the kittens.