libinput and button debouncing

A few days ago, I pushed code for button debouncing into libinput, scheduled for libinput 1.9. What is button debouncing you ask? Well, I'm glad you asked, because otherwise typing this blog post would've been a waste of time :)

Over in Utopia, when you press the button on a device, you get a press event from the hardware. When you release said button, you get a release event from the hardware. Together, they form the button click interaction we have come to learn and love over the last couple of decades. Life is generally merry and the sunshine to rainbow to lollipop ratio is good. Meanwhile, over here in the real world, buttons can be quite dodgy, don't always work like they're supposed to, lollipops are unhealthy and boy, have you seen that sunburn the sunshine gave me? One way how buttons may not work is that they can lose contact for a fraction of a second and send release events even though the button is being held down. The device usually detects that the button is still being down in the next hardware cycle (~8ms on most devices) and thus sends another button press.

For us, there are not a lot of hints that this is bad hardware besides the timestamps. It's not possible for a user to really release and press a button within 8ms, so we can take this as a signal for dodgy hardware. But at least that's someting. All we need to do is ignore the release event (and subsequent button event) and only release when the button is released properly. This requires timeouts and delays of the event, something we generally want to avoid unless absolutely necessary. So the behaviour libinput has now is enabled but inactive button debouncing on all devices. We monitor button release and button press timestamps, but otherwise leave the events as-is, so no delays are introduced. Only if a device sends release/press combos with unfeasably short timeouts, activate button debouncing. Once active, we filter all button release events and instead set a timer. Once the timer expires, we send the button release event. But if at any time before then another button press is detected, the scheduled release is discarded, the button press is filtered and no event is sent. Thus, we paper over the release/press combo the hardware gives us and to anyone using libinput, it will look like the button was held down for the whole time.

There's one downside with this approach - the very first button debounce to happen on a device will still trigger an erroneous button release event. It remains to be seen whether this is a problem in real-world scenarios. That's the cost of having it as an auto-enabling feature rather than an explicit configuration option.

If you do have a mouse that suffers from button bouncing, I recommend you try libinput's master branch and file any issues if the debouncing doesn't work as it should. Might as well get any issues fixed before we have a release.

libinput and pressure-based palm detection

I (finally!) merged a patchset to detect palms based on pressure into libinput. This should remove a lot of issues that our users have seen with accidental pointer movement. Palm detection in libinput previously used two approaches: disable-while-typing and an edge-based approach. The former simply ignores touchpad events while keyboard events are detected, the latter ignores touches that happen in the edge zones of the touchpad where real interaction is unlikely. Both approaches have the obvious disadvantages: they're timeout- and location-dependent, causing erroneous pointer movements. But their big advantage is that they work even on old touchpads where a lot of other information is unreliable. Touchpads are getting better, so it's time to make use of that.

The new feature is relatively simple: libinput looks at per-touch pressure and if that pressure hits a given threshold, the touch is regarded as palm. Once a palm, that touch will be ignored until touch up. The threshold is intended to be high enough that it cannot easily be hit. At least on the touchpads I have available for testing, I have to go through quite some effort to trigger palm detection with my finger.

Pressure on touchpads is unfortunately hardware-dependent and we can expect most laptops to have different pressure thresholds. For our users this means that the feature won't immediately work perfectly, it will require a lot of hwdb entries. libinput now ships a libinput measure touchpad-pressure tool to experiment with the various pressure thresholds. This makes it easy to figure out the right pressure threshold and submit a bug report (or patch) for libinput to get the pressure threshold updated. The documentation for this tool is available as part of libinput's online documentation.

TLDR: if libinput seems to misdetect touches as palms, figure out the right threshold with libinput measure touchpad-pressure and file a bug report so we can merge this into our hwdb.

libinput 1.8 switching to git-like helper tool naming "libinput sometool"

I just released the first release candidate for libinput 1.8. Aside from the build system switch to meson one of the more visible things is that the helper tools have switched from a "libinput-some-tool" to the "libinput some-tool" approach (note the space). This is similar to what git does so it won't take a lot of adjustment for most developers. The actual tools are now hiding in /usr/libexec/libinput. This gives us a lot more flexibility in writing testing and debugging tools and shipping them to the users without cluttering up the default PATH.

There are two potential breakages here, one is that the two existing tools libinput-debug-events and libinput-list-devices have been renamed too. We currently ship compatibility wrappers for those but expect those wrappers to go away with future releases. The second breakage is of lesser impact: typing "man libinput" used to bring up the man page for the xf86-input-libinput driver. Now it brings up the man page for the libinput tool which then points to the man pages of the various features. That's probably a good thing, it puts the documentation a bit closer to the user. For the driver, you now have to type "man 4 libinput" though.

xf86-input-wacom 0.34 workaround for pressure range bugs

Back in 2003, the pressure range value for Wacom pen tablets was set to 2048. That's a #define in the driver, but it shouldn't matter because we also advertise this range as part of the device description in the X Input protocol. Clients should be using that advertised min/max range and scale appropriately, in the same way as they should be doing this for the x/y axis ranges.

Fast-forward to 2017 and we changed the pressure range. New Wacom devices now use ~8000 levels, but we opted to just #define it in the driver to 65536 and be done with it. We now scale all models into that range, with varying granularity based on the physical hardware. It shouldn't matter because it's not tied to a reliable physical property anyway and the only thing that matters is the percentage of the max value (which is why libinput just gives you a [0, 1] range. Hindsight is a bliss.).

Slow-forward to 2017-and-a-bit and we received complaints that pressure handling is now broken. Turns out that some applications hardcoded the old 2048 range and now get confused, because virtually any pressure will now hit that maximum. Since those applications are largely proprietary ones and cannot be updated easily, we needed a workaround to this. Jason Gerecke from Wacom got busy and we now have a "Pressure2K" option available in the driver. If set, this option will scale everything into the 2048 range to make sure those applications still work. To get this to work, the following xorg.conf.d snippet is recommended:

Section "InputClass"
    Identifier "Wacom pressure compatibility"
    MatchDriver "wacom"
    Option "Pressure2K" "true"
EndSection

Put it in a file that sorts higher than the wacom driver itself (e.g. /etc/X11/xorg.conf.d/99-wacom-pressure2k.conf) and restart X. Check the Xorg.log/journal for a "Using 2K pressure levels" message, then verify it works by running xinput list "device name". xinput should show a range of 0 to 2048 on the third valuator.

$>  xinput list "Wacom Intuos4 6x9 Pen stylus"
Wacom Intuos4 6x9 Pen stylus                    id=25   [slave  pointer  (2)]
        Reporting 8 classes:
                Class originated from: 25. Type: XIButtonClass
                Buttons supported: 9
                Button labels: None None None None None None None None None
                Button state:
                Class originated from: 25. Type: XIKeyClass
                Keycodes supported: 248
                Class originated from: 25. Type: XIValuatorClass
                Detail for Valuator 0:
                  Label: Abs X
                  Range: 0.000000 - 44704.000000
                  Resolution: 200000 units/m
                  Mode: absolute
                  Current value: 22340.000000
                Class originated from: 25. Type: XIValuatorClass
                Detail for Valuator 1:
                  Label: Abs Y
                  Range: 0.000000 - 27940.000000
                  Resolution: 200000 units/m
                  Mode: absolute
                  Current value: 13970.000000
                Class originated from: 25. Type: XIValuatorClass
                Detail for Valuator 2:
                  Label: Abs Pressure
                  Range: 0.000000 - 2048.000000
                  Resolution: 1 units/m
                  Mode: absolute
                  Current value: 0.000000
                Class originated from: 25. Type: XIValuatorClass
                Detail for Valuator 3:
                  Label: Abs Tilt X
                  Range: -64.000000 - 63.000000
                  Resolution: 57 units/m
                  Mode: absolute
                  Current value: 0.000000
                Class originated from: 25. Type: XIValuatorClass
                Detail for Valuator 4:
                  Label: Abs Tilt Y
                  Range: -64.000000 - 63.000000
                  Resolution: 57 units/m
                  Mode: absolute
                  Current value: 0.000000
                Class originated from: 25. Type: XIValuatorClass
                Detail for Valuator 5:
                  Label: Abs Wheel
                  Range: -900.000000 - 899.000000
                  Resolution: 1 units/m
                  Mode: absolute
                  Current value: 0.000000

This is an application bug, but this workaround will make sure new versions of the driver can be used until those applications have been fixed. The option will be available in the soon-to-be-released xf86-input-wacom 0.35. The upstream commit is d958ab79d21b57141415650daac88f9369a1c861.

Edit 02/06/2017: wacom devices have 8k pressure levels now.

xinput list shows a "xwayland-pointer" device but not my real devices and what to do about it

TLDR: If you see devices like "xwayland-pointer" show up in your xinput list output, then you are running under a Wayland compositor and debugging/configuration with xinput will not work.

For many years, the xinput tool has been a useful tool to debug configuration issues (it's not a configuration UI btw). It works by listing the various devices detected by the X server. So a typical output from xinput list under X could look like this:

:: whot@jelly:~> xinput list
⎡ Virtual core pointer                          id=2    [master pointer  (3)]
⎜   ↳ Virtual core XTEST pointer                id=4    [slave  pointer  (2)]
⎜   ↳ SynPS/2 Synaptics TouchPad                id=22   [slave  pointer  (2)]
⎜   ↳ TPPS/2 IBM TrackPoint                     id=23   [slave  pointer  (2)]
⎜   ↳ ELAN Touchscreen                          id=20   [slave  pointer  (2)]
⎣ Virtual core keyboard                         id=3    [master keyboard (2)]
    ↳ Virtual core XTEST keyboard               id=5    [slave  keyboard (3)]
    ↳ Power Button                              id=6    [slave  keyboard (3)]
    ↳ Video Bus                                 id=7    [slave  keyboard (3)]
    ↳ Lid Switch                                id=8    [slave  keyboard (3)]
    ↳ Sleep Button                              id=9    [slave  keyboard (3)]
    ↳ ThinkPad Extra Buttons                    id=24   [slave  keyboard (3)]

Alas, xinput is scheduled to go the way of the dodo. More and more systems are running a Wayland session instead of an X session, and xinput just doesn't work there. Here's an example output from xinput list under a Wayland session:

$ xinput list
⎡ Virtual core pointer                     id=2 [master pointer  (3)]
⎜   ↳ Virtual core XTEST pointer               id=4 [slave  pointer  (2)]
⎜   ↳ xwayland-pointer:13                      id=6 [slave  pointer  (2)]
⎜   ↳ xwayland-relative-pointer:13             id=7 [slave  pointer  (2)]
⎣ Virtual core keyboard                    id=3 [master keyboard (2)]
    ↳ Virtual core XTEST keyboard              id=5 [slave  keyboard (3)]
    ↳ xwayland-keyboard:13                     id=8 [slave  keyboard (3)]

As you can see, none of the physical devices are available, the only ones visible are the virtual devices created by XWayland. On a Wayland session, the X server doesn't have access to the physical devices. Instead, it talks via the Wayland protocol to the compositor. This image from the Wayland documentation shows the architecture:

In the above graphic, devices are known to the Wayland compositor (1), but not to the X server. The Wayland protocol doesn't expose physical devices, it merely provides a 'pointer' device, a 'keyboard' device and, where available, a touch and tablet tool/pad devices (2). XWayland wraps these into virtual devices and provides them via the X protocol (3), but they don't represent the physical devices.

This usually doesn't matter, but when it comes to debugging or configuring devices with xinput we run into a few issues. First, configuration via xinput usually means changing driver-specific properties but in the XWayland case there is no driver involved - it's all handled by libinput inside the compositor. Second, debugging via xinput only shows what the wayland protocol sends to XWayland and what XWayland then passes on to the client. For low-level issues with devices, this is all but useless.

The takeaway here is that if you see devices like "xwayland-pointer" show up in your xinput list output, then you are running under a Wayland compositor and debugging with xinput will not work. If you're trying to configure a device, use the compositor's configuration system (e.g. gsettings). If you are debugging a device, use libinput-debug-events. Or compare the behaviour between the Wayland session and the X session to narrow down where the failure point is.

inputfd - a protocol for direct access to input devices in wayland

This is a higher-level explanation of the inputfd protocol RFC I sent to the list on March 31. Note that this is a first draft, this protocol may never see the light of the day or may be significantly altered before it lands.

First, what is it? inputfd is a protocol for a Wayland compositor to pass a file descriptor (fd) for an input device directly to the client. The client can then read events off this fd and process them, without any additional buffering in between. In the ideal case, the compositor doesn't care (or even notice) that events are flowing between the kernel and the client. Because the compositor sets up the fd, the client does not need any special privileges.

Why is this needed? There are a few input devices that will not be handled by libinput. libinput is the stack for those devices that have a direct interaction with the desktop - mice, keyboards, touchpads, ... But plenty of devices do not want or require desktop interactions. Joysticks/gamepads are the prime example here, but 3D mice or VR input devices also come to mind. These devices don't control the cursor on the desktop, so the compositor doesn't care about the device's events. Note that the current draft only caters for joysticks, 3D mice etc. are TBD.

Why not handle these devices in libinput? Joysticks in particular are so varied that a library like libinput would have to increase the API surface area massively to cater for every possibility. And in the end it is likely that libinput would merely buffer events and pass them on almost as-is anyway. From a maintenance POV - I don't want to deal with that. And from a technical POV - there's little point to have libinput in between anyway. Furthermore, it's already the case that gaming devices are opened directly by the application rather than going through X. So just connecting the clients with the device directly has a advantages.

What does the compositor do? The compositor has two jobs: device filtering and focus management. In the current draft, a client can say "give me all gaming devices" but it's the compositor that decides which device is classified as such (see below for details). Depending on seat assignment or other policies, a client may only see a fraction of the devices currently connected.

Focus management is relatively trivial: the compositor decides that a client has focus now (e.g. by clicking into the window) and hands that client an fd. On focus out (e.g. alt-tab) the fd is revoked and the client cannot read any more events. Rinse, wash, repeat as the focus changes. Note that the protocol does not define how the focus is decided, that task is up to the compositor. Having focus management in the compositor gives us a simple benefit: the client can't hog the device or read events when it's out of focus. This makes it easy to have multiple clients running that need access to the same device without messing up state in a backgrounded client. The security benefit is minimal, few people enter their password with a joystick. But it's still a warm fuzzy feeling to know that a client cannot read events when it's not supposed to.

What devices are applicable devices? This is one of the TBD of the protocol. The compositor needs to know whether a device is a gaming device or not, but the default udev tag ID_INPUT_JOYSTICK is too crude and provides false positives (e.g. some Wacom tablets have that tag set). The conversion currently goes towards having a database that can define this better, but this point is far from decided on.

There are a couple of other points that are still being discussed. If you have knowledge in game (framework) development, do join the discussion, we need input. Personally I am far from a game developer, so I cannot fathom all the weird corner cases we may have to deal with here. Flying blind while designing a protocol is fun, but not so much when you then have to maintain it for the next 10 years. So some navigational input would be helpful.

And just to make sure you join the right discussion, I've disabled comments here :)

A simple house-moving tip: use tape to mark empty cupboards

When you've emptied a cupboard, put masking tape across it, ideally in a colour that's highly visible. This way you immediately see know which ones are finished and which ones still need attention. You won't keep opening the cupboard a million times to check and after the move it takes merely seconds to undo.