Multitouch in X - Touch grab handling

This post is part of a series on multi-touch support in the X.Org X server.

1. Multitouch in X - Getting events
2. Multitouch in X - Pointer emulation
3. Multitouch in X - Touch grab handling
4. Multitouch in X - Multitouch-touchpads

In this post, I'll outline how grabs on touch events work. This post assumes basic knowledge of the XI2 Xlib interfaces.

Passive grabs

The libXi interface has one new passive grab call: XIGrabTouchBegin, which works pretty much like the existing passive grab APIs. As with event selection, you must set all three event masks XI_TouchBegin, XI_TouchUpdate and XI_TouchEnd or a BadValue error occurs. Once a passive grab activates in response to a touch, the client must choose to either accept or reject a touch. Details on that below.

Grabs activate on a TouchBegin event and due to the nature of multitouch, multiple touch grabs may be active at any time - some of them for different clients.

Active grabs

Active grabs do not have a new call, they are handled through the event masks of the existing XIGrabDevice(3) call. If a client has an active touch grab on the device, it is automatically the owner of the touch sequence (ownership is described below). If a client has an active pointer or keyboard grab on the device, it is the owner of the touch sequence for pointer emulated touch events only. Other touch events are unaffected by the grab and are processed normally.

Acceptance and rejection

Pointer grabs provide exclusive access to the device, but to some degree a client can opt to replay the event it received on the next client. We expect that touch sequences will often trigger gesture recognition, and a client may realise after a few events that it doesn't actually want that touch sequence. So we expanded the replay semantics. clients with a touch grab must choose to either accept or reject a touch.

Accepting a touch signals to the server that the touch sequence is meant for this client and no-one else. The server then exclusively delivers to that client until the terminating TouchEnd.

Rejecting a touch sequence signals that the touch sequence is not meant for this client. Once a client rejects a touch sequence, the server sends the TouchEnd event to that client (if the touch is still active) and replays the full touch sequence [1] on the next grab or window. We use the term owner of a touch sequence to talk about the current recipient.

The order of events for two clients Cg and Cw, with Cg having a grab and Cw having a regular touch event selection on a window, is thus:

TouchBegin to Cg    → 
TouchUpdate to Cg   → 
TouchUpdate to Cg   → 
                    ← Cg rejects touch
                    ← Cw becomes new owner
TouchEnd+ to Cg     →
TouchBegin* to Cw   → 
TouchUpdate* to Cw  → 
TouchUpdate* to Cw  → 
#### physical touch ends #### 
TouchEnd to Cw      →

Events with + mark an event created by the server, * mark events replayed by the server

For nested grabs, this sequence simply repeats for each client until either a grabbing client accepts the touch or the client with the event selection becomes the owner.

In the above case, the touch ended after Cg rejected the touch. If the touch ends before the current owner accepted or rejected it, the owner gets the TouchEnd event and the touch is left handing until the owner accepts or rejects it. If accepted, that's it. If rejected, the new owner gets the full sequence in one go, including the TouchEnd event. The sequence is thus:

TouchBegin to Cg    → 
TouchUpdate to Cg   → 
TouchUpdate to Cg   → 
#### physical touch ends #### 
TouchEnd to Cg      →
                    ← Cg rejects touch
                    ← Cw becomes new owner
TouchBegin* to Cw   → 
TouchUpdate* to Cw  → 
TouchUpdate* to Cw  → 
TouchEnd* to Cw     →

Touch ownership handling

One additional event type that XI 2.2 introduces is the XI_TouchOwnership event. Clients selecting for this event signal that they need to receive touch events before they're the owner of the touch sequence. This event type can be selected on both grabs and event selections.

First up: there are specific use-cases where you need this. If you don't fall into them, you're better off just skipping on ownership events, they make everything more complicated. And whether you need ownership events depends not only on you, but also the stack you're running under. On normal touch event selection, touch events are only delivered to the current owner of the touch. With multiple grabs, the delivery is sequential and delivery of touch events may be delayed.

Clients selecting for touch ownership events get the events as they occur, even if they are not the current owner. The XI_TouchOwnership event is delivered if and when they become the current owner. The last part is important: if you select for ownership events, you may receive touch events but you may not become the owner of that sequence. So while you can start reacting to that sequence, anything your app does must be undo-able in case the e.g. window manager claims the touch sequence.

If we look at the same sequence as above with two clients selecting for ownership, the sequence looks like this:

TouchBegin to Cg     → 
TouchBegin to Cw     → 
TouchOwnership to Cg →
TouchUpdate to Cg    → 
TouchUpdate to Cw    → 
TouchUpdate to Cg    → 
TouchUpdate to Cw    → 
                     ← Cg rejects touch
                     ← Cw becomes new owner
TouchEnd+ to Cg      →
TouchOwnership to Cw →
#### physical touch ends #### 
TouchEnd to Cw      →

Note: TouchOwnership events do not correspond to any physical event, they are always generated by the server

If a touch ends before the owner accepts, the current owner gets the TouchEnd, all others get a TouchUpdate event instead. That TouchUpdate has a flag XITouchPendingEnd set, signalling that no more actual events will arrive from this touch but the touch is still waiting for owner acceptance.

TouchBegin to Cg     → 
TouchBegin to Cw     → 
TouchOwnership to Cg →
TouchUpdate to Cg    → 
TouchUpdate to Cw    → 
TouchUpdate to Cg    → 
TouchUpdate to Cw    → 
#### physical touch ends #### 
TouchEnd to Cg       →
TouchUpdate to Cw    →  (XITouchPendingEnd flag set)
                     ← Cg rejects touch
                     ← Cw becomes new owner
TouchOwnership to Cw →
TouchEnd to Cw       →

In both cases, we dealt with a rejecting owner. For an accepting owner, the sequences look like this:

TouchBegin to Cg     → 
TouchBegin to Cw     → 
TouchOwnership to Cg →
TouchUpdate to Cg    → 
TouchUpdate to Cw    → 
TouchUpdate to Cg    → 
TouchUpdate to Cw    → 
                     ← Cg accepts touch
TouchEnd+ to Cw      →
TouchUpdate to Cg    → 
#### physical touch ends #### 
TouchEnd to Cg      →

or

TouchBegin to Cg     → 
TouchBegin to Cw     → 
TouchOwnership to Cg →
TouchUpdate to Cg    → 
TouchUpdate to Cw    → 
TouchUpdate to Cg    → 
TouchUpdate to Cw    → 
#### physical touch ends #### 
TouchEnd to Cg       →
TouchUpdate to Cw    →  (XITouchPendingEnd flag set)
                     ← Cg accepts touch
TouchEnd* to Cw      →

In the case of multiple grabs, the same strategy applies in order of grab activation. Ownership events may be selected by some clients but not others. In that case, each client is treated as requested, so the event sequence the server deals with may actually look like this:

TouchBegin to C1     → 
TouchBegin to C3     → 
TouchOwnership to C1 →
TouchUpdate to C1    → 
TouchUpdate to C3    → 
TouchUpdate to C1    → 
TouchUpdate to C3    → 
                     ← C1 rejects touch
                     ← C2 becomes new owner
TouchEnd+ to C1      →
TouchBegin* to C2    → 
TouchUpdate* to C2   → 
TouchUpdate* to C2   → 
                     ← C2 rejects touch
                     ← C3 becomes new owner
TouchEnd+ to C2      →
TouchOwnership to C3 →
#### physical touch ends #### 
TouchEnd to C3       →

[1] obviously we need to store these events so "full sequence" really means all events until the buffer was full