Multitouch in X - Pointer emulation

This post is part of a series on multi-touch support in the X.Org X server.

1. Multitouch in X - Getting events
2. Multitouch in X - Pointer emulation
3. Multitouch in X - Touch grab handling
4. Multitouch in X - Multitouch-touchpads

In this post, I'll outline how pointer emulation on touch events works. This post assumes basic knowledge of the XI2 Xlib interfaces.

Why pointer emulation?

One of the base requirements of adding multitouch support to the X server was that traditional, non-multitouch applications can still be used. Multitouch should be a transparent addition, available where needed, not required where not supported.

So we do pointer emulation for multitouch events, and it's actually specified in the protocol how we do it. Mainly so it's reliable and predictable for clients.

What is pointer emulation in X

Pointer emulation simply means that for specific touch sequences, we generate pointer events. The conditions for emulation are that the the touch sequence is eligible for pointer emulation (details below) and that no other client has a touch selection on that window/grab.

The second condition is important: if your client selects for both touch and pointer events on a window, you will never see the emulated pointer events. If you are an XI 2.2 client and you select for pointer but not touch events, you will see pointer events. These events are marked with the XIPointerEmulated so that you know they come from an emulated source.

Emulation on direct-touch devices

For direct-touch devices, we emulate pointer events for a touch sequence provided the touch is the first touch on the device, i.e. no other touch sequences were active for this device when the touch started. The touch sequence is emulated until it ends, even if other touches start and end while that sequence is active.

Emulation on dependent-touch devices

Dependent touch devices do not emulate pointer events. Rather, we send the normal mouse movements from the device as regular pointer events.

Button events and button state

Pointer emulation triggers motion events and, more importantly, button events. The button number for touches is hardcoded to 1 (any more specific handling such as long-click for right buttons should be handled by touch-aware clients instead), so the detail field of an emulated button event is 1 (unless the button is logically mapped).

The button state field on emulated pointer events adjusts for pointer emulation as it would for regular button events. The button state is thus (usually) 0x0 for the emulated ButtonPress and 0x100 for the MotionNotify and ButtonRelease events.

Likewise, any request that returns the button state will have the appropriate state set, even if no emulated event actually got sent.

Grab handling works as for regular pointer events, though the interactions between touch grabs and emulated pointer grabs are somewhat complex. I'll get to that in a later post.

The confusing bit

There is one behaviour about the pointer emulation that may be confusing, even though the specs may seem logical and the behaviour is within the specs.

If you put one finger down, it will emulate pointer events. If you then put another finger down, the first finger will continue to emulate pointer events. If you now lift the first finger (keeping the second down) and put the first finger down again, that finger will not generate events. This is noticable mainly in bi-manual or multi-user interaction.

The reason this doesn't work is simple: to the X server, putting the first finger down just looks like another touchpoint appearing when there is already one present. The server does not know that this is the same finger again, it doesn't know that your intention was to emulate again with that finger. Most of the semantics for such interaction is in your head alone and hard to guess. Guessing it wrong can be quite bad, since that new touchpoint may have been part of a two-finger gesture with the second finger and whoops - instead of scrolling you just closed a window, pasted your password somewhere or killed a kitten. So we err on the side of caution, because, well, think of the kittens.

Multitouch in X - Getting events

This post is part of a series on multi-touch support in the X.Org X server.

1. Multitouch in X - Getting events
2. Multitouch in X - Pointer emulation
3. Multitouch in X - Touch grab handling
4. Multitouch in X - Multitouch-touchpads

I recommend re-reading Thoughts on Linux multitouch from last year for some higher-level comments.
In this post, I'll outline how to identify touch devices and register for touch events.

This post assumes basic knowledge of the XI2 Xlib interfaces. Code examples should not be scrutinised for language-correctness.

New event types

XI 2.2 defines four new event types: XI_TouchBegin, XI_TouchUpdate, XI_TouchEnd are the standard events that most applications will be using. The fourth event, XI_TouchOwnership is mainly for handling specific situations where reaction speed is at a premium and gesture processing when grabs are active. I won't be covering those in this post.

Identifying touch devices

To use multitouch functionality from a client application, the client must announce support for the X Input Extension version 2.2 through the XIQueryVersion(3) request.

int major = 2, minor = 2;
XIQueryVersion(dpy, &major, &minor);
if (major * 1000 + minor < 2002)
    printf("Server does not support XI 2.2\n");

Once announced, an XIQueryDevice(3) call may return a new class type, the XITouchClass. If this class is present on a device, the device supports multitouch.The class struct itself is defined like this:

typedef struct
{
    int         type;
    int         sourceid;
    int         mode;
    int         num_touches;
} XITouchClassInfo;

The num_touches field specifies the number of simultaneous touches supported by the device. If the number is 0, we simply don't know (likely) or the device supports an unlimited number of touches (less likely). Regardless of the value expect that some devices lie, so it's best to treat this value as a guide only.

The mode field specifies the type of touch devices. We currently define two types and the server behaviour differs depending on the type:

• XIDirectTouch for direct-input touch devices (e.g. your average touchscreen or tablet).  For this type of device, the touch events will be delivered to the windows at the of the touch point. Again, similar to what you would expect from a tablet interface - you press top left and the application top-left responds.
• XIDependentTouch for a indirect input devices with multi-touch functionality. Touchpads are the prime example here. Touch events on such devices will be sent to the window underneath the cursor and clients are expected to interpret the touchpoints as (semantically) relative to the cursor position. For example, if your cursor is inside a Firefox window and you touch with two fingers on the top-left corner of the touchpad, Firefox will get those events. It can then decide on how to interpret those touchpoints.

A device that has a TouchClass may send touch events, but these events use the same axes as pointer events. Having said that, a touch device may still send pointer events as well - if the physical device generates both.
Your code to identify touch devices could roughly look like this:

XIDeviceInfo *info;
int nevices;

info = XIQueryDevice(display, XIAllDevices, &ndevices);

for (i = 0; i < ndevices; i++)
{
    XIDeviceInfo *dev = &info[i];
    printf("Device name %d\n", dev->name);
    for (j = 0; j < dev->num_classes; j++)
    {
        XIAnyClassInfo *class = dev->classes[j];
        XITouchClassInfo *t = (XITouchClassInfo*)class;

        if (class->type != XITouchClass)
            continue;

        printf("%s touch device, supporting %d touches.\n",
               (t->mode == XIDirectTouch) ?  "direct" : "dependent",
               t->num_touches);
    }
}

Selecting for touch events

Selecting for touch events on a window is mostly identical to pointer events. A client creates an event mask and submits it with XISelectEvents(3). One exception applies: a client must always select for all three touch events [1], XI_TouchBegin, XI_TouchUpdate, XI_TouchEnd. Selecting for one or two only will result in a BadValue error.

As for button events, only one client may select for touch events on any given window and the event delivery attempts traverse from the bottom-most window in the window tree up to the root window. Where a matching event selection is found, the event is delivered and the traversal stops.

Handling touch events

The three event types [1] are XIDeviceEvents like pointer and keyboard events. So from a client's point of view, in essence all we added was new event types.

The detail field of touch events specifies the touch ID, a unique ID for this particular touch for the lifetime of the touch sequence. Each touch sequence consists of a TouchBegin event, zero or more TouchUpdate events and one TouchEnd event. Since multiple touch sequences may be ongoing at any time, keeping track of the ID is important. The server guarantees that the touch ID is unique per device and that it will not be re-used [2]. Note that while touch IDs increase, they increase by an implementation-defined amount. Don't rely on the next touch ID to be the current ID + 1.

The button state in a touch event is the state of the physical buttons only. A TouchUpdate or TouchEnd event will thus usually have a zero button state. [3]

That's pretty much it, otherwise the handling of touch events is identical to pointer or keyboard events. Touch event handling should be straightforward and the significant deviations from the current protocol are in the grab handling, something I'll handle in a future post.

[1] I know, it's four. Good that you're paying attention.
[2] Technically ID collision may occur. For that to happen, you'd need to hold at least one touch down while triggering enough touches to exhaust a 32 bit ID range. And hope that after the wraparound you will get the same ID. There are better ways to spend your weekend.
[3] pointer emulation changes this, but I'll get to that some other time.

Multitouch patches posted

After pulling way too many 12+ hour days, I've finally polished the patchset for native multitouch support in the X.Org server into a reasonable state. The full set of patches is now on the list. And I'm still expecting this to get merged for 1.12 (and thus in time for Fedora 17).

The code is available from the multitouch branches of the following repositories:

  git://people.freedesktop.org/~whot/xserver
  git://people.freedesktop.org/~whot/inputproto
  git://people.freedesktop.org/~whot/xf86-input-evdev
  git://people.freedesktop.org/~whot/libXi

Here's a screencast running Fedora 16 with the modified X server and a little multitouch event debugging application.

Below is a short summary of what multitouch in X actually means, but one thing is important: being the windowing system, X provides multitouch support. That does not mean that every X application now supports multitouch, it merely means that they can now use multitouch if they want to. That also includes gestures, they need application support.

A car analogy: X provides a new road, the applications still have to opt to drive on it.

Multitouch events

XI 2.2 adds three main event types: XI_TouchBegin, XI_TouchUpdate and XI_TouchEnd. These three make up a touch sequence. X clients must subscribe to all three events at once and will then receive the events as they come in from the device (more or less, grabs can interfere here). Each touch event has a unique touch ID so clients can track the touches over time.

We support two device types: XIDirectDevice includes tablets and touchscreens where the events are delivered to the position the touch occurs at. XIDependentDevice includes multitouch-capable touchpads. Such devices still control a normal pointer by default, but for multi-finger gestures are possible. For such devices, the touchpoints are delivered to the window underneath the pointer.

That is pretty much the gist of it. I'll post more information over time as the release gets closer, so stay tuned.

Pointer emulation

Multitouch can be a compelling interaction method but as said above, X only provides support for multitouch. It will take a while for applications to pick it up (Carlos Garnacho is working on GTK3) and some never will. Since we still need to interact with those applications, we provide backwards-compatible pointer emulation. Again, the details are in the protocol but the gist of it is that for the first touchpoint we emulate pointer events.

That's the really nasty bit, because you now have to sync up the grab event semantics of the core, XI 1.x and XI2 protocols and wrap it all around the new grab semantics. So that if you have a multitouch app running under a window manager without multitouch support everything still works as expected.
That framework is now in place too though I expect it to still have bugs, especially in the hairier corner cases.

But other than that, it should work just as intended. I can interact with my GNOME3 desktop quite well and I get multitouch events to my test applications.

[edit Dec 20: typo fix]

A short update on multitouch

For the last couple of weeks I've been pretty much working full-time on getting multitouch/XI 2.2 ready for the merge (well, I was on holidays for a bit too). So first of all - sorry if I've been ignoring bugs or emails, I'm working to a few deadlines here. Anyway, here's a bit of a status update.

Right now, it looks like touch event delivery is working, including nested grabs.Chase Douglas started on the pointer emulation while I was away and we're now at the point where emulation works, except that pointer grabs on top of multitouch clients aren't handled yet. I'm still rather optimistic to get this into 1.12, though it's getting a bit unwieldly. Carlos Garnacho has already sent me some patches, so he's testing the lot against the GTK branches.

However, since touch support cannot simply be bolted on top and needs to be integrated properly, this has triggered some extra rewrites here and there. I'm currently some 200 commits ahead of master sync-point. I'm planning to get this number down to something sane before merging but meanwhile, sorry, I'll have to keep ignoring you until this is done.

Improving code readability through temporary variables

We don't always have the luxury of using library interfaces that are sensibly designed and enforce readability self-explanatory (see this presentation). A fictional function may look like this:

extern void foo(struct foo *f, 
                Bool check_device, 
                int max_devices, 
                Bool emulate);

But a calls often end up like this:

foo(mystruct, TRUE, 0, FALSE);

Or, even worse, the function call could be:

foo(mystruct, !dev->invalid, 0, list_is_first_entry(dev->list));

The above is virtually unreadable and to understand it one needs to look at the function definition and the caller at the same time. The simple use of a temporary variable can do wonders here:

Bool check_device = !dev->invalid;
Bool emulate_pointer = list_is_first_entry(dev->list));

foo(mystruct, check_device, 0, emulate_pointer);

It adds a few lines of code (though I suspect the compiler will mostly reduce the output anyway) but it improves readability. Especially in the cases where the source field name is vastly different to the implied effect. In the second example, it's immediately obvious that pointer emulation should happen for the first entry.

The bugzilla attention span

Some bugs are less important than others, and there's always a certain background noise of bugs that aren't complete deal-breakers. And there are always more of those to fix than I have time for.

Every once in a while, I sweep through a list of open bugs and try to address as many as possible. Given the time constraints, I have a limited attention span for each bug. Anything that wastes time reduces the chance of a bug getting fixed. The bugs that tend to get fixed (or at least considered) are the ones with attachments: an xorg.conf, Xorg.log and whichever xorg.conf.d snippets were manually added. Always helpful is an evtest log of the device that's a problem.

If it takes me 10 comments to get useful log from a bug reporter because they insist on philosophical discussions, ego-stroking, blame games, etc., chances are I've used up the allocated time for that bug before doing anything productive. And I've had it happen often enough that reporters eventually attached useful information but I just never went back to that bug. Such is life.

So, in the interest of getting a bug fixed: attach the logs and stay on-topic. If something is interesting or important enough to have a philosophical discussion about it, then we can always start one later.

Mediawiki for personal notes

For years, I've had the problem of how to store my notes but finally found something that is usable.

I've tried a few variations over time, but none suited my use-case sufficiently. Textfiles are too limited and become unmanageable quick. Org-Mode requires Emacs (Emacs and I disagree on how much pain I'm willing to accept in my hands), Zim wasn't available on RHEL and didn't scale well after a while. TomBoy and Gnote require GNOME which doesn't work on the Mac*, aside from me always running into synchronization issues using it on three machines. I toyed with ideas like Evernote but keeping notes in the cloud makes me slightly nervous and doesn't work for confidential stuff anyway. Other wikis I tried over the last few years were dokuwiki, ikiwiki and tiddlywiki.

Many moons ago, Aaron Stafford showed me how he used MediaWiki. At first that didn't seem viable since it didn't cover a few things I wanted: portability, git-based backups that I can take to work and back and use on any machine I care about. Anyway, I've used this for well over a year now and it works for me.

The ingredients:

• one usb stick


• truecrupt (realcrypt in Fedora's rpmfusion)


• mediawiki

(Note: I'm not giving step-by-step instructions because if you're setting this up, you should understand what you're doing instead of just copying commands from a random website)

Preparing the USB stick is easy enough. Format it with FAT32 (if you're not suffering from having a Mac, you can use a better file system). Create a massive file with truecrypt, encrypt it. You can encrypt the entire stick but having the in-file option allows multiple encrypted files on the same stick.

The filesystem for the encrypted file should be FAT32 again. Then install mediawiki in a directory, point apache to it and run through the mediawiki install process. Set the mediawiki up to use an sqlite database. Finally, set up a cron job that essentially runs git add * && git commit -am "Autocommit $date" every hour. Backup is simply done by running git push remote.

I recommend writing a few scripts that automount the stick once you plug it in. Once that's done and on all machines you care about, you just need to plug the stick in, start httpd and off you go. Of course, if you trust your hosting provider enough you could also set it up somewhere on the web and you can skip the USB stick madness.

Now, is this setup perfect? No, by far not. Issues that I see with it:

• no text-based backend. Now that my X server is decidedly more stable that doesn't matter that much anymore


• interface decidedly Web 1.0 (TiddlyWiki is much nicer here)


• no real database merging, essentially requiring a single install instead of several synchronising ones


• the mediawiki syntax is random at best, and chaotic at worst. As you get used to it becomes less of a problem


• search hardly works. Not sure if that's a sqlite issue or a broken setup

Lessons learned

Forgetting the USB stick at work means you can't take notes at home for that evening or weekend. I now have a repeating event in my calendar to remind me to take it home.

Databases don't merge and git won't help with a binary file. For a while, I kept two wikis, one for work notes, one for private stuff. But then you have something that overlaps both (e.g. a computer setup that you use at work and at home). Eventually I ended up dumping the smaller database and importing it into the other one before I had too much overlap.

Categories are awesome. They're essentially tags, add [[Category:Foobar]] to any page and the matching Category page lists all pages in that category (plus, that page can also be a page with other info). Categories can be nested.

Everything must be written down. This is something of a golden rule for any note-taking attempt. Unless you take notes, your notes won't be useful.

I still use a normal pencil and paper notebook. Especially when debugging I use pen and paper and then transfer the final result over to the mediawiki.

Install Lazarus. It has saved me a few times.

Redirects help finding pages. Whenever I search for a page with a certain title and it's not there, I add a redirect from that title to the real page. So next time, I'm sorted.

I'm now re-learning how to navigate pages instead of just searching through them. This has one big benefit: on the path to a page you may encounter another page you've forgotten about and that you can now deal with again.

Keep a "Log" (or "Diary") page, with short comments of what you did on each day. I've set this to my home page now and it consists of entries like "Looked at [[Fedora 16 Blocker Bugs]]". It's a quick way to jot down what you're doing, pointing to the pages that contain the actual information.

As said above, I've been using this setup for quite a while now. It's not perfect but software hardly ever is. The final ingredient though I only found last week: the LACIE MosKeyTo, a USB stick that sticks out by only a few mm. Before that I was always worried about breaking off the stick when I carried the laptop around.

So if you're looking for note-taking software, I can't say mediawiki is perfect. But it is the most usable one I've found for me.



* My Mac is essentially a netbook and I don't really install anything on it. It keeps me from working too much at home.

Natural scrolling in the synaptics driver

[updated Apr 16 2012 for smooth scrolling]
The latest version of Mac OSX (Lion) introduced a feature referred to natural scrolling. Since I've been getting a few requests to add the same feature to the synaptics driver. At it's base natural scrolling is an inversion of the scroll direction*. We've had the ability to do this for years, though more by accident than intent.
The synaptics driver (until X server 1.12's smooth scrolling) uses buttons 4/5/6/7 for up/down/left/right scrolling. If you want up to scroll down, just swap them and you're good to go.

xinput set-button-map <device name> 1 2 3 5 4

Now down is up and up is down. The device name can be obtained as usual with xinput list. This will work for servers up to including 1.11.
As of synaptics 1.6RC3 (1.5.99.903), the options VertScrollDelta and HorizScrollDelta can now be set to a negative value. For servers supporting smooth scrolling (X server 1.12+), this effectively inverts the scroll direction.
* I realise that natural scrolling in OSX may have a few other details to make it do whatever it does. Synaptics supports scroll inversion and scrolling inertia.

What's new in XI 2.1 - smooth scrolling

This post is the last part of a series about new changes in XI 2.1. See also:

• What's new in XI 2.1 - versioning


• What's new in XI 2.1 - XI2 defines


• What's new in XI 2.1 - raw events

The core protocol allows for up to 255 buttons but only two axes: x and y. When scroll wheels became ubiquitous, the solution was to send button events for the four scroll wheel directions. Even today, a logical button click 4 results in scroll down, 5, 6, and 7 result in scroll up, left, and right, respectively.

However, scrolling is more complex than that. Velocity information and subpixel scrolling are handy to have on scroll axes. With XI 2.1, servers may now mark some axes as axes providing scrolling information. These axes may represent real axes on the device (e.g. a mouse wheel) or be virtual axes (e.g. the edge scrolling on synaptics devices). For each scrolling axis, XIQueryDevice(3) provides an XIScrollClassInfo.

XIDeviceInfo *info;

int ndevices;



info = XIQueryDevice(display, XIAllDevices, &ndevices);

for (i = 0; i < ndevices; i++) {

     XIScrollClassInfo *scroll;

     for (j = 0; j < info[i].num_classes; j++) {

          scroll = info[i].classes[j];

          if (scroll->type != XIScrollClass)

            continue; /* Not interested in others */





          printf("Valuator %d is a valuator for  ", scroll->number);

          switch(scroll->scroll_type) {

              case XIScrollTypeVertical: printf("vertical"); break;

              case XIScrollTypeHorizontal: printf("horizontal"); break;

          }

          printf(" scrolling\n");

          printf("Increment of %d is one scroll event\n", scroll->increment);

          if (scroll->flags & XIScrollFlagsNoEmulation)

              printf("No emulated button events for this axis\n");

          if (scroll->flags & XIScrollFlagsPreferred)

              printf("No emulated button events for this axis\n");

   }

}



XIFreeDeviceInfo(info);

The above code example runs through each class on each device, printing only the scrolling information bit. Note that for each XIScrollClassInfo, a XIValuatorClassInfo with the same axis number must be present. That valuator info then contains the actual details of the device.

Let's say a XIScrollClassInfo is available for valuator 3. This simply means that any coordinates on valuator 3 should be interpreted as scrolling events. The increment defines what delta the driver considers to be one scroll event. For an increment of +5, each delta of 5 should be regarded as one scroll unit down. For an increment of -3, each delta of 3 should be regarded as one scroll unit up (i.e. inverted).

Note that a driver may change this at run-time. As always, a client should listen to XIDeviceChangedEvents.

Scroll button emulation

If a driver provides smooth-scrolling valuators, the server provides two-way emulation of scroll events to remain backwards-compatible to existing X clients. For any button 4-7 event from the device, the server emulates a scroll event with the right increment on the respective axis. (Note that drivers are discouraged to use button events for scrolling if they support smooth scrolling). For any scroll event on a scroll axis axis, the server emulates the matching button 4-7 events. If the scroll event has an abs(value) less than the increment on this axis, the server only emulates once the cumulative value of several events hits a multiple of the increment (positive or negative).

For clients, this means they get two events for each hardware scrolling event. The one that was emulated by the server has the XIPointerEmulated flag set and can be safely ignored by the client. The same flag is set for XIRawEvents as well.

A device may have multiple scoll axes of the same type. If so, one of them is marked as XIScrollFlagsPreferred and any legacy button events will be emulated on that axis. This bit is informative only, I don't think clients need to care about it. Plus, we're aiming for drivers to do smooth scrolling only, so hopefully we don't need to do button → scroll valuator emulation.

Potential side-effects

• Pre-2.1 clients may now get valuator information from a scroll axis that they don't identify as scroll axis. For 1.x clients, this is not much of an issue. 1.x clients never knew what an axis represented anyway and needed a UI for users to select what an axis meant. 2.0 clients have axis labels and should act on those. So I think this is mostly a non-issue.
  


• Clients that implement smooth-scrolling may act on information less than one increment. Permanent up/down movement of values less than one increment may then cause smooth scrolling events but no legacy events. I don't think this is an issue, users usually trigger the scrolling axis until they see the effect on-screen. The worst-case scenario here is that scrolling in one client requires more finger movement than in another client.
  


• Clients cannot yet change the increment. I don't know if this will be an issue in the real world given that the actual interpretation is in the client anyway.
  

What's new in XI 2.1 - raw events

This post is part of a series about new changes in XI 2.1. See also:

• What's new in XI 2.1 - versioning


• What's new in XI 2.1 - XI2 defines

Raw events were introduced in XI 2.0 to allow clients to track direct device data, before the server gets its hands on it. The prime target for these was games that needed relative mouse events.

However, there was one flaw with raw events: they were delivered either to the root window or to the grabbing client only. Which made them less-than-useful, since grabs are virtually everywhere: active grabs by clients, passive grabs whenever a popup or dropdown menu appears, implicit grabs whenever the button is down.

This behaviour changes in XI 2.1. If a client announces support for XI 2.1, it will get raw events delivered to the root window at all times.

The second change to raw event behaviour is that the events now have the sourceid set (Bug 34420). The libXi interface catered for this but we never sent the value down the wire. This is fixed now in XI 2.1.

What's new in XI 2.1 - XI2 defines

This post is part of a series about new changes in XI 2.1. See also:

• What's new in XI 2.1 - versioning

A while ago, I spent some time staring at gdb wondering why a passive grab didn't activate. Turns out it wasn't a bug in the server after all, it was a bug in the program. The modifiers were set like this:

XIGrabModifiers modifiers[1];

modifiers[0].modifiers = AnyModifier;

AnyModifier is a define that comes from the core protocol:

#define AnyModifier             (1<<15)  /* used in GrabButton, GrabKey */

but XI2 provides its own define:

#define XIAnyModifier           (1U << 31)

Passive grabs only activate if the modifiers match the current modifier state. An XI2 grab that uses AnyModifier won't activate as expected - AnyModifier looks like a normal modifier bitmask to the server.

I wanted to reply to the reporter with "don't mix core and XI2 defines" but there's one problem: you couldn't currently write an XI2 application without using some core defines (e.g. GrabModeAsync). I've looked through the protocol spec and added those defines that previously forced clients to mix input-related constants. New defines now available are:

• XIPropModeReplace, XIPropModePrepend, and XIPropModeAppend


• XIAnyPropertyType


• XIGrabModeSync and XIGrabModeAsync


• XIGrabSuccess, XIAlreadyGrabbed, XIGrabInvalidTime, XIGrabNotViewable, and
  XIGrabFrozen


• XIOwnerEvents, XINoOwnerEvents

These will be available to any application compiling against XI 2.1 headers. All the above are identical to the core defines. Clients are still expected to use CurrentTime and None from the core protocol headers since neither of those two is input-related. For applications building against 2.1, the guideline to using XI2 can only be summarised as: Only use a define Foo if there is no equivalent XIFoo version.

The libXi man pages have also been updated where applicable to point out the new defines.
Many thanks to Timur Kristóf for uncovering this.

XIOwnerEvents and XINoOwnerEvents are new additions to improve readability. They simply map to True/False.

What's new in XI 2.1 - versioning

This post is the start of a mini-series describing the new features added to the X Input Extension version 2.1, or short XI 2.1.

Also, I need to point out that XI 2.1 is not yet released, some minor changes may still be added. Nonetheless, this series should help you understand the new additions and if you see any issues - by all means let me know so we can amend before it is set in stone.

In this post, I'll talk about versioning.

First, for all those looking for big headlines: multitouch in X has been postponed to XI 2.2. XI 2.1 is a small incremental change with two bugfixes and one new feature. These are uncontroversial and more importantly, they are ready now. So XI 2.1 is released as a small change, with bigger changes coming for XI 2.2. We currently still plan to get XI 2.2 into server 1.12 so this is really just a numbers game and ticking off work that is complete.

The small list of changes for XI 2.1:

• New raw events behaviour


• Smooth scrolling


• XI2 defines

I'll describe each of those in a separate post.

Announcing support for XI 2.1

Any client that supports XI2 must already announce so with XIQueryVersion(3). To unlock XI 2.1 behaviour, a client must announce support for XI 2.1 or later.

int major = 2;

int minor = 1;

int rc;



rc = XIQueryVersion(display, &major, &minor);

if (rc == Success)

    printf("Server supports XI %d.%d\n", major, minor);

else if (rc == BadRequest)

    printf("Server does not support XI2.\n");

else

    printf("Internal error\n");

The server is compatible with XI 2.0 and XI 2.1. Thus, a client that does not announce XI 2.1 support will get XI 2.0 behaviour.

Using XI2 version defines

Because I fell into this trap myself, a word of warning about a potential bug. If you are currently using the XI_2_Major and XI_2_Minor defines for versioning, recompiling against new inputproto headers will automatically bump your client up to the latest version, including possible unexpected behaviours. Don't do this, XI2 is versioned through numbers, not defines. Have a look at this xinput commit to get a better idea.

Making git reset less dangerous

If you've ever fat-fingered a git reset HEAD~2 into a git reset HEAD~23 then this may be of interest to you. If you have, add this to your .gitconfig:

[alias]

	r = !sh -c \"git log -1 --format=oneline\" && git reset

Using git r instead of git reset now prints the current HEAD before resetting.

$ git r --hard HEAD~23

9d09ffc3ba1a65fc7feefd21abd5adacf3274628 dix: NewCurrentScreen must work on pointers where possible

HEAD is now at 5083b56 Revert "mi: move switching screen on dequeue to separate helper function"

Whoops. Fat-fingered. A git r --hard 9d09ffc3ba1a65fc7feefd21abd5adacf3274628 will undo the above and get you back to your previous HEAD.

Note: git fsck --lost-found will also help you to find the commit again, but it'll take more time and effort.

Update: as mjg59 pointed out to me, git reflog records git resets, thus making it easy to find the previous HEAD too. Wish I'd known that command sooner ;)

Starting a terminal in GNOME3

After GNOME3 came out, one of the common criticisms I heard/read about was that a click on the the terminal icon doesn't start a new terminal - it brings the current one to the foreground.

I wouldn't have noticed, I can't remember the last time I clicked on that icon. GNOME has for years supported setting a keyboard short-cut to fire up a terminal. GNOME3 still supports that short-cut. Go to System Settings → Keyboard → Shortcuts, click on "Launch Terminal". Assign a shortcut (e.g. Ctrl+Alt+T). And that's it. Now you can fill your screen faster with new terminals than you could ever click it.

On that note, I recommend using "Terminator" as your terminal emulator, assign shortcuts to "maximize window" and "maximize window horizontally" and with a few shortcuts you essentially have tiling window manager for terminals, inside GNOME.

Disabling ssh host key checking

Note: this post describes how to disable a security feature. Unless you know why you're doing this, better don't do it.

The testboxes I have need to boot a variety of different operating systems and OS versions, usally from USB disks. Each OS has different ssh host keys, so all-too-frequently I got this error when trying to ssh in:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that the RSA host key has just been changed.
[...]

Deleting the old key from $HOME/.ssh/known_hosts and re-trying fixes the issue - until the next reboot into a different system.

Now, while I'm sure there's a way to share host keys between installs some of those systems are very short-lived only. So I needed something for my main box (where I ssh from).

A while ago I found the magic recipe. Pop this into your $HOME/.ssh/config:

Host 192.168.0.100
        UserKnownHostsFile /dev/null
        StrictHostKeyChecking no

And what that does is nicely explained in ssh_config(5):

UserKnownHostsFile

Specifies a file to use for the user host key database instead of ~/.ssh/known_hosts.

StrictHostKeyChecking

[...]
If this flag is set to “no”, ssh will automatically add new
host keys to the user known hosts files. If this flag is set
to “ask”, new host keys will be added to the user known host
files only after the user has confirmed that is what they
really want to do, and ssh will refuse to connect to hosts
whose host key has changed. The host keys of known hosts will
be verified automatically in all cases. The argument must be
“yes”, “no”, or “ask”. The default is “ask”.

The combination of /dev/null and "no" to key checks means that ssh will automatically add new hosts. But the host key will be saved in /dev/null, so next time it's like ssh connects to a new unknown host.

Problem solved, though in your own interest you should keep the Host definition as narrow as possible. Host key checking exists for a reason.

A man page for xkeyboard-config

Command-line configuration of XKB keyboard maps usually involves setxkbmap:

setxkbmap -layout "foo" -variant "bar" -options "gobble:dygook"

The problem is though that you need to know your layout, your variant, and your options. And they're not always straightforward. What command would you use for a US querty and German Dvorak layout where the Meta key is on the left Windows key and Ctrl is mapped to the Caps Lock key? If you knew the answer [1] without looking it up first you're scaring me. Stay away from children please.

A few days ago Carl Worth asked a few XKB-configuration questions on IRC and amongst other things this led to yet another complaint about the lack of a man page. Which I answered with the usual "feel free to write one". Except that a few hours later I suddenly had the epiphany: we already have all this data - GNOME uses it just fine for its own keyboard configuration panel. A little later some XSL had been beaten into submission enough for me to proudly say that we now have a man page that lists every single option that you can pipe into setxkbmap. xkeyboard-config(7) for president!

This will be available in the next version, scheduled for September 27. Until then, see the git commit here. Fedora users: xkeyboard-config-2.3-2.fc16 and xkeyboard-config-2.3-2.fc15.


[1] setxkbmap -layout "us,de" -variant ",dvorak" \
-option "altwin:left_meta_win,ctrl:nocaps" Obvious now, isn't it?

Using environment git trees

Despite the confusing title, this is not tree-hugger territory ;)

I have two main machines and several test boxes. More often than not, I end up installing a new machine and get new home directory. This is annoying, because I've gotten quite used to my zsh setup, my vim customisations and all the other little tweaks my systems have accumulated over the years.

Somewhen around last year, this really started pissing me off so I set up a git tree for all those configurations, but I have now split that git tree into two trees: one called environment and one called shellenv.

Here's the current list of files in shellenv:

:: whot@barra:~/shellenv (master)> git ls-files
.gitconfig
.screenrc
.ssh/authorized_keys2
.tigrc
.zsh/10-exports
.zsh/20-aliases
.zsh/30-chpwd
.zsh/30-compctl
.zsh/30-keybindings
.zsh/40-xinput-compctl
.zsh/90-prompt
.zsh/func.tmp
.zsh/func/_fedpkg
.zshrc
setup.sh

The list of files in environment includes my mutt setup, my $HOME/scripts directory with all scripts to automate various stuff, my irssi setups, etc. Both repositories have a setup.sh:

#!/bin/bash

pwd=$PWD
filename=`basename $0`
excludes=". .. .git $filename"

for file in `ls -a`; do
    skip=0
    for exclude in $excludes; do
        if test $file = $exclude; then
            skip=1
            break
        fi
    done

    if test $skip -eq 1; then
        continue
    fi

    if ! test -e "$HOME/$file"; then
        echo ln -s "$pwd/$file" "$HOME/$file"
        ln -s "$pwd/$file" "$HOME/$file"
    elif test -d "$file"; then
        echo "$file already exists"
    else
        echo "$file already exists"
    fi
done

All this does is to symlink the target file with the current file (provided if the target does not exist). The script used in my environment tree is a tad more complicated since it takes hostnames into account to populate the various /etc directories as well but you get the gist.

The result of all that? I can install a new test machine, run git clone and ./setup.sh and the machine has my shell environment. I get a new actual machine (or I update any of my local configuration) and I can run git clone or git pull and all my main boxes have the identical setup. Do try this at home.


Note: I know there are a few bugs in the symlink scripts. I always run them the same way though, they work for me and it's not really worth my time right now to try to make them perfect

Linking kernel and X devices

XI2 has made it possible to access any device the X server knows about and get events from those devices. Alas, one piece was missing: short of parsing the Xorg.log file it was so far impossible to link the system devices with kernel device. You had to guess that /dev/input/event3 was your "AT Translated Set 2 keyboard" with the X device ID of 6.

Why would one need this? Some properties of a device are not exported in X. These included for example anything in sysfs.

I've now pushed patches to evdev, synaptics and the wacom driver to export the device node as a simple string property. From this device node, a client can navigate to the matching sysfs dir, get other events out of the kernel or do whatever they like to do.

The property name is "Device Node" and it is a standard char* that contains the device path. The path is set by the driver, so even if you use e.g. auto-dev on synaptics, the value will be filled in correctly. On the downside, if the driver doesn't support it, the property will not be available on a device. If you write clients that require this information, ensure that you can cope with a missing property and remember that several X devices may share the same device node (e.g. wacom).

These patches do not require a new server, though we have added a #define XI_PROP_DEVICE_NODE to the server to make it easier for others to use this property without misspelling it.

Another property that was pushed in the same series was the "Device Product ID" property (#define XI_PROP_PRODUCT_ID once the new server is out). This property contains two unsigned 32-bit values: vendor ID and product ID. Having this information available makes it simpler for X clients to apply configuration based on specific devices. This can come in handy if you e.g. need different acceleration for a set of mice or you simply want to remap buttons on a specific device.
Again, the two values are set by the driver and all the above rules apply.

I may end up writing those patches for mouse and keyboard too but feel free to help me out with it.

The obligatory not-quite-a screenshot:

:: whot@barra:~> xinput list-props "SynPS/2 Synaptics TouchPad" 
Device 'SynPS/2 Synaptics TouchPad':
[...]
 Device Product ID (252): 2, 7
 Device Node (253): "/dev/input/event4"

Update June 7 2011: As Peter pointed out in the comments, the device node is always local to the host the server runs on. It is up to the client to figure out if it runs on the same host.

Fedora 15 and offlineimap users beware

This is just a word of warning: if you have upgraded to F15 and you use offlineimap, observe very carefully what it does and ensure you have a backup of your email.

This post has updates. See at the end of the post

I noticed yesterday that I couldn't search for some email that I knew existed. On closer inspection I found a huge chunk of my archive inbox to be missing. Several months worth of emails, they stopped in March and then continued with the emails from yesterday that I had copied in from my Inbox.

The same happened to my fedora-devel and xorg-devel folders, possibly others. Re-running offlineimap to sync these pulled down hundreds of emails, so I let it run overnight. This morning I discovered that sometimes offlineimap simply states something like "Deleted 1018 messages" and they're gone again. This is cumulative, it seems to delete only newer ones (I was at September 2010, but after one more run I'm now back to July 2010 being the newest messages in in my xorg-devel folder).

The good news is that the emails are still on the server. Also, so far only mailboxes with huge numbers of emails (>10000) seem to be affected, smaller mailboxes appear to work fine.

I have not filed a bug yet because I don't know what the actual issue is. Could be offlineimap, could be something else. But in the meantime, beware.
Update May 30: Filed as Bug 708898

Update May 30
I downgraded to offlineimap 6.2.0 which is in F14 (offlineimap-6.2.0-2.fc14.noarch) and managed to eventually resync my mailboxes. Then I went back to F15's offlineimap-6.3.3-1.fc15.noarch. Of course it didn't trigger on my test mailbox, but it did then do the same thing again on my xorg-devel inbox.

Syncing lists-xorg: IMAP -> Maildir
 Deleting 8181 messages (33540, 335
[...] 
in Maildir[lists-xorg]

F15's offlineimap-6.2.0-3.fc15.noarch does not show this behaviour and seems to resync the locally deleted emails. Of course that'll take a while, but I'll try to get logs this time (my first couple of tries to get logs from offlineimap crashed it).

GNOME 3.0 middle mouse button emulation

This is essentially a GNOME 3.0-specific update to New evdev middle mouse button emulation defaults. In recent evdev versions, middle mouse button emulation is disabled by default and can be enabled with Option "Emulate3Buttons" "on" in the xorg.conf file or an xorg.conf.d snippet.

GNOME 3.0 has a gsettings key to also trigger this behaviour on a per-user basis (see bug 633863). The default value is off (false). Settings applied from the desktop environment overwrite xorg.conf(.d) settings, so to get middle button emulation in GNOME simply toggle the gsettings key and don't worry about xorg.conf(.d). The command to toggle the key is:

$> gsettings set org.gnome.settings-daemon.peripherals.mouse \
     middle-button-enabled true

Since this gsettings are persistent, setting the key once is sufficient. It will be applied after login, for each device and as new devices are hot-plugged.

Custom input device configuration in GNOME

Thanks largely to Bastien, Gnome Bug 635486 just got committed, in time for GNOME3.

It adds is the ability for custom configuration of input devices from within GNOME, especially for settings that are not handled by the GNOME UI tools. It's principle is quite simple. A new gsettings key hotplug-command in the org.gnome.settings-daemon.peripherals.input-devices schema points to an executable. This executable is called at gnome-settings-daemon startup, whenever a device has been added and whenever a device has been removed. Something like this:

        $HOME/executable -t added -i 12 My device name

So the device with the ID 12 and the name "My device name" just got added. Other types are "removed" and "present" for those device that are already present when g-s-d starts up. It's quite simple to configure any device to your liking now. Have a look at the input-device-example.sh script shipped with gnome-settings-daemon. I'll give you two examples of what you could add to the script and the rest should be easy enough to figure out:

# Map stylus button 1 to button 8
if [ "$device" = "Wacom Intuos4 6x9 stylus" ]; then
        xsetwacom set "$device" Button 1 8
fi

# map tapping to LMR instead of default LRM
if [ "$device" = "SynPS/2 Synaptics TouchPad" ]; then
        xinput set-prop $id "Synaptics Tap Action" 0 0 0 0 1 2 3
fi

How to DOS a developer

So you think your favourite FOSS project is moving too fast? Easy, just DOS the developers. Here's a few tips on how to achieve this:

On IRC

• If you have a problem, just state that you have a problem. Let the developer ask for what exactly it is. Then answer every question with the minimum amount of information possible and let the developer keep asking.


• Wait 5 minutes before answering but keep asking why the developer doesn't answer immediately. That way you can maximise the context switching costs of the developer, ensuring he or she can't get anything else done while waiting for you.


• If the developer asks for version numbers, be vague. Things like "whatever was in $DISTRO three days ago" is best because unless the developer is also the distro maintainer, you've answered the question without providing any information.


• Don't stay on IRC. Just go offline whenever your computer goes to sleep and re-ask the question every time you reconnect. This way you ensure that those not monitoring the channel will try to answer the questions whenever you're currently offline. Never send email to the list, because others could answer it in their own time.

Via email

• Send bug reports to the developers directly. That way you force them to reply with at least "please file a bug in $BUGZILLA". That takes 2 min of their time, not counting context switches. Make sure you reply with more than one "thank you" email so they need to deal with more email.


• Randomly remove the CC from mailing lists during discussions. That way you make sure the developer has to answer twice if they didn't notice the CC was missing.


• When taking discussions back onto the list, misquote (or purposely misinterpret) the results of the private discussion. Force the developer to justify themselves in public.


• Be verbose. As verbose you can be. If you can say something in one sentence or three paragraphs (with three overlapping or identical examples), choose the latter. Searching for signal in lots of noise is a favourite pasttime of many developers.


• If you change the subject of a discussion, leave the email subject as-is. This way you ensure that the email cannot easily be found later. Wait for a few weeks, then refer to this discussion, but not in the same thread.


• Pastebin everything instead of including it in emails. Links to websites, backtraces, log files, patches, etc. Make sure the pastebin has a short expiry date.


• If you must include something as attachment, make sure it's at least in a zipfile so no-one can look at it without a few mouse-clicks first.

Via code

• When commenting on patches, reply with vague statements on how the patch doesn't work. This way, the patch is made to look bad and the developer now has to spend time making sure your vague statements weren't just unfounded.


• When testing code, make sure you don't explain how a bug happend. Just say "after a while it crashed", that's enough information for anyone to figure out what's going wrong.


• Read The Daily WTF and gain inspiration for your next open source project. If you ever get other people helping you, you maximise their pain.


• Don't document anything. Just use arbitrary hardcoded values, with no explanation.


• Never, ever, explain why you're doing something. Code is not meant to be self-explanatory.


• Don't follow anything I said in on commit messages


• Do everything as complicated as possible but don't explain why. Obfuscating that you're flipping the sign of a variable three to four times is a good example, anyone reading the code is sure to spend hours on it


• Use numbers as both bools and numbers. C is great for that, you can return 2 and then use it as Boolean and as actual numeric value lateron. Hide any such usage as well as you can.


• When sending patches, mix code changes with arbitrary whitespace changes. Fun minutes can be spend looking at two identical lines trying to find the difference that's missing.


• When sending updated versions of a patch, don't tell anyone what has changed. That way, you force your reviewers to review the whole patch, every time you change a single character.


• When sending patches, make sure that it only fixes the problem on your specific setup. That way you also can tell the developers off for not merging an important bugfix that clearly works when you tested it. And you force them to reply to you.

Those are just the immediate things that come to my mind. If the project is already low on manpower, any of these will have a high impact. If it has enough developers, you may need to use several of them at once to tie up the most resources. Of course all this requires developers that still think that communicating with users is worthwile and that other people generally tell the truth. If they have given up on humanity, they may just ignore you and keep improving the project. In that case, you can at least badmouth everyone on tech "news" sites forums and comments.

Oh, in case you're wondering why I don't get anything done lately, see above. I realise that probably none of it is intentional, though the net effect is the same.

evtest is now on freedesktop.org

evtest, that little program that displays kernel input events and information about input devices now has a home on freedesktop.org. I hope this helps to reduce the number of different upstreams and distro-specific patches that float around.

Code: http://cgit.freedesktop.org/evtest/

Bugzilla: https://bugzilla.freedesktop.org/enter_bug.cgi?product=evtest

Relative and absolute devices in X

One of the recurring questions about to input event handling is how relative and absolute devices are handled in X. Surprisingly, the answer is "that's complicated" and it took me a while to wrap my head around this.

Back in the days of yonder, devices had no mode. X (the coordinate, not the window system) was x, y was usually y unless bugs intervened and we all had mice, computers the size of fridges and fridges the size of, well, fridges. Such was the lore, blah blah (those of us who are old enough to actually remember those days, which excludes myself for example, may remember them fondly).

In 1994, the new kid on the block (not one of the singing ones, they disbanded that same year¹, presumably for other reasons. Though that'd be one thing I'd love to be credited with...), the X Input Extension protocol added support for device mode and the notion of relative and absolute devices was born (within the Xiverse, anyway). Fast-forward half a lifetime of something that lives for 30 years and XI2 added to that per-valuator modes. i.e. a device may have an absolute X axis and a relative Y axis.

But: the device mode in X describes the behaviour of a device only. On an absolute device, the pointer is expected to jump to the position the device tells it. On a relative device, the pointer is expected to move by the coordinates provided, based on the current pointer position (and accounting for pointer acceleration).

However, there are no requirements for how to send coordinates and absolute devices may send relative data and vice versa. So inside the server, the mode of a device (or a valuator) has no meaning. It's merely a flag sent to the client.

Which makes mode processing a bit tricky because the only time we know if an event was relative or absolute is when we receive the actual event (events include a absolute/relative flag). An absolute device may calculate deltas based on the previous position and then submit the new position as absolute coordinate - and thus feel like a relative device to the user. For whatever reason you'd want to do this, I don't think any driver does it atm (though I have vague recollections of having done this in the past). But it just adds to the insanity that is input in X.

Either way, we cannot rely on the device mode for anything in the server, and the actual device mode is just a hint to the client.

¹I had to wikipedia this. I wonder what that does to my reputation when the admins see the logs.

Reviewing patches backwards

Though this may be obvious to some, I only learned a short while ago that it's much easier to review patches bottom to top. Why? Because much code is written bottom to top.

In C, one uses static functions for code that doesn't need to be exported. Because we're lazy and try to group code together, many statics don't have a separate declaration. So you end up writing with code like this:

static void cure_world_peace(void) {
    ...
}

static void save_cancer(void) {
    ...
}

static void solve_all_problems(void) {
   cure_world_peace();
   save_cancer();
}

void foo(void) {
   ...
   solve_all_problems();
   ...
}

When you're reviewing top to bottom, you see the new feature first but lack the context. This gets more and more confusing the more functions you see because you have to keep in mind what they're doing. Reviewing bottom to top means you'll see the context first and you can tick it off mentally. Only then will get to the static functions and see the details.

And coincidentally, this is also how I write code - smack the hooks where they belong and then implement the details.

The deficiency argument

There are many good reasons for bugs not getting addressed. One of them is the design argument ("this system isn't designed for this feature") or the dependency argument ("this bug is in this library we link against"). Both are valid explanations for why a software project does not address a bug directly. And of course the ever-present reasons "lack of time" and "not enough manpower". However, quite often the defence for a bug in some software is along the lines of "Windows can't do this either".

If you explain a deficiency in your software through the deficiencies of competing software, you still have a problem.

I don't care whether other implementations are buggy when I'm using some software. I care about what I'm using because there's a compelling reason why I chose this one. And if it doesn't do the job, it needs to be fixed.

Always strive for your software to be excellent and look to other software for inspiration, not excuses.