It's templates all the way down

Benjamin Tissoires and I have been busy anthophila and working on the freedesktop CI templates. This post is primarily of interest if you're working on GitLab, specifically if your repo is hosted on gitlab.freedesktop.org. If either of those applies, prepare to be distracted from the current pandemic, otherwise maybe just prepare to be entertained. I'll do my best to be less miserable than the news.

We all know that CI/CD really helps with finding bugs early. If you don't know that yet, insert a jedi handwave before the previous sentence and now you do. GitLab is the git forge now used by freedesktop.org and it comes with a built-in CI system. I'm leaving out the difficult bits such as actually setting the thing up because this is obviously all handled by Heinzelmännchen and just readily available, hooray. I'm also going to assume that you roughly know how to write GitLab CI jobs or, failing that, at least know how to read YAML without screaming. So for this post, we start with the basic problem that your .gitlab-ci.yml is getting unwieldy, repetitive or generally just kinda sucks to maintain. Which is roughly where libinput and libevdev were a while back which caused Benjamin to start the ci-templates.

Now, what do we want? (other than a COVID-19 cure) Reproducible tests, possibly on different distributions, with the same base system across tests. For my repos the goal was basically "test on the common distributions to catch certain bugs early". [1] For Mesa, the requirement is closer to "have a fixed set of images that 'never' change so tests are reproducible". Both goals have much in common.

Your first venture into CI will look like this:

myjob:
  image: fedora:31
  before_script:
    - dnf update -y
    - dnf install -y onepackage twopackage threepackage floor
  script:
    - meson builddir && ninja -C builddir test

So, in short: take a Fedora 31 docker image, update it [2], install the required packages and then run the actual test part - meson and ninja. Easy.

This works fine but it takes approximately forever because dnf update is slow and you're potentially pulling down gigs of packages on every test run. Which is fun, but less so when you have 10 different jobs and they all do that. So let's call this step 1 and pretend we're more advanced than that. Step 2 is where you start building an image you re-use, steps 3 to N are the bits where you learn more than you want to know about docker, podman, skopeo and how many typos you can put into a YAML file. So, ad break, and we jump right to the part where enlightenment is just around the corner or wherever enlightenment lurks these days.

Using the CI Templates

Here's the .gitlab-ci.yml to build a Fedora 31 images with ci-templates and run the test on that image:

include:
  - project: 'freedesktop/ci-templates'
    ref: 123456deadbeef
    file: '/templates/fedora.yml'

variables:
   # project name of the upstream repo
   FDO_UPSTREAM_REPO: someproject/name

stages:
  - prep
  - test

myimage:
  extends: .fdo.container-build@fedora
  stage: prep
  variables:
    FDO_DISTRIBUTION_VERSION: '31'
    FDO_DISTRIBUTION_PACKAGES: 'onepackage twopackage threepackage floor'
    FDO_DISTRIBUTION_TAG: '2020-03-20.0'

myjob:
  extends: .fdo.distribution-image@fedora
  stage: test
  script:
    - meson builddir && ninja -C builddir test
  variables:
    FDO_DISTRIBUTION_VERSION: '31'
    FDO_DISTRIBUTION_TAG: '2020-03-20.0'

Now, you guessed correctly that the .fdo and FDO_ prefixes are used by the templates. There is a bunch of stuff hidden here. Basically, this will:

• check if the image exists in your personal project's registry and use that, but if not
• check if the image exists in the given upstream project's registry and use that, but if not
• create a Fedora 31 image with the given packages installed and pushes it with the tag to the registry
• use that image (whether newly created or pre-existing) and run the tests on it

There are a few more details too, but that's roughly the summary of it. For existing tags, the the myimage job effectively becomes a noop and the myjob job will re-use the image. The image will be in your registry so you can podman run it locally to reproduce a bug.

To build a new image, simply change the tag. Either because you want newer packages or you need extra (or less packages). And the nice thing here: you will build a new image as part of your merge request and run the CI against that new image. But upstream and every other MR will keep using the old image - right up until your MR is merged at which point every (future) MR will use that new updated image.

Want to build a Debian Stretch image? Replace Fedora and 31 with debian and stretch. Same for Ubuntu, Centos, Alpine and Arch though for those two you don't need a version number.

Templating the templates

"But, but, Peter, I want to test on eleventy different distribution like you do" I hear you say. Well, fear not, for this is where the ci-fairy comes in. How about we *gasp* generate the .gitlab-ci.yml file from a base configuration? That can't possibly be a bad idea, so let's do that! First, we save our configuration into the .gitlab-ci/config.yml:

distributions:
  - name: fedora
    tag: 12345
    version: 30
  - name: ubuntu
    tag: abcde
    version: '19.10'
  # and so on, and so forth

packages:
  - curl
  - wget
  - gcc

There is no specific requirement on the structure of the config file, ci-fairy simply loads it and passes it to Jinja2. Your template could thus look like this .gitlab-ci/ci.template file:

include:
{% for d in distributions %}
   - project: 'freedesktop/ci-templates'
     ref: 123456deadbeef
     file: '/templates/{{d.name}}.yml'
{% endfor %}

stages:
  - prep
  - test

{% for d in distributions %}

.{{d.name}}.{{d.version}}:
  variables:
    FDO_DISTRIBUTION_VERSION: '{{d.version}}'
    FDO_DISTRIBUTION_TAG: '{{d.tag}}'

myimage.{{d.name}}.{{d.version}}:
  extends:
    - .fdo.container-build@{{d.name}}
    - .{{d.name}}.{{d.version}}
  stage: prep
  variables:
    FDO_DISTRIBUTION_PACKAGES: "{{' '.join(packages)}}"

myjob.{{d.name}}.{{d.version}}:
  extends:
    - .fdo.distribution-image@{{d.name}}
    - .{{d.name}}.{{d.version}}
  stage: test
  script:
    - meson builddir && ninja -C builddir
{% endfor %}

And to locally generate our .gitlab-ci.yml, all we need to do is

$ pip3 install git+http://gitlab.freedesktop.org/freedesktop/ci-templates
$ cd path/to/project
$ ci-fairy generate-template
$ ci-fairy lint  # checks the resulting YAML for syntax errors
$ git commit .gitlab-ci.yml

And, for reference, the file we generated here looks like this:

include:
   - project: 'freedesktop/ci-templates'
     ref: 123456deadbeef
     file: '/templates/fedora.yml'
   - project: 'freedesktop/ci-templates'
     ref: 123456deadbeef
     file: '/templates/ubuntu.yml'

stages:
  - prep
  - test

.fedora.30:
  variables:
    FDO_DISTRIBUTION_VERSION: '30'
    FDO_DISTRIBUTION_TAG: '12345'

myimage.fedora.30:
  extends:
    - .fdo.container-build@fedora
    - .fedora.30
  stage: prep
  variables:
    FDO_DISTRIBUTION_PACKAGES: "curl wget gcc"

myjob.fedora.30:
  extends:
    - .fdo.distribution-image@fedora
    - .fedora.30
  stage: test
  script:
    - meson builddir && ninja -C builddir

.ubuntu.19.10:
  variables:
    FDO_DISTRIBUTION_VERSION: '19.10'
    FDO_DISTRIBUTION_TAG: 'abcde'

myimage.ubuntu.19.10:
  extends:
    - .fdo.container-build@ubuntu
    - .ubuntu.19.10
  stage: prep
  variables:
    FDO_DISTRIBUTION_PACKAGES: "curl wget gcc"

myjob.ubuntu.19.10:
  extends:
    - .fdo.distribution-image@ubuntu
    - .ubuntu.19.10
  stage: test
  script:
    - meson builddir && ninja -C builddir

Aside from the templating a new thing here is the e.g. .fedora.30 template what we extend from. This is an easy way to avoid having to set things like the distribution version and the tag multiple times. And a few things of note: the tag is job-specific (not distribution-specific). So you could have two Fedora 30 images with two different tags. This is also just an example I typed out, a real-world .gitlab-ci.yml will look more complex and different. So only rely on the above to get an idea of what's possible.

A word for non-gitlab.freedesktop.org users: You can use the remote: include directive to use the templates from elsewhere. ci-fairy isn't tied to freedesktop.org either but you'll have to provide more flags to get what you want instead of relying on the default behaviours.

The documentation for CI Templates has more, go and peruse my pretties.

[1] For months the CI was basically just a build test because I couldn't run the test suite in a container
[2] Updating isn't always required but sooner or later you run into a dependency issue if you don't

A tale of missing touches

libinput 1.15.1 had a new feature: it matched the expected touch count with the one actually seen as opposed to the one advertised by the kernel. That is good news for ALPS devices whose kernel driver lies about their capabilities because these days who doesn't. However, in some cases that feature had the side-effect of reducing the touch count to zero - meaning libinput would ignore any touch. This caused a slight UX degradation.

After a bit of debugging and/or cursing, the issue was identified as a libevdev issue, specifically - the way libevdev replays events after a SYN_DROPPED event. And after several days of fixing things, adding stuff to the CI and adding meson support for libevdev so the CI can actually run a few useful things, it's time for a blog post to brain-dump and possibly entertain the occasional reader such as you are. Congratulations, I guess.

The Linux kernel's evdev protocol is a serial protocol where all events have a type, a code and a value. Events are grouped by EV_SYN.SYN_REPORT events, so the event type is EV_SYN (0), the event code is SYN_REPORT (also 0). The value is usually (but not always), you guessed it, zero. A SYN_REPORT signals that the current event sequence (also called a "frame") is to be interpreted as one hardware event [0]. In the simplest case, two hardware events from a mouse could look like this:

EV_REL   REL_X        1
EV_SYN   SYN_REPORT   0
EV_REL   REL_X        1
EV_REL   REL_Y        1
EV_SYN   SYN_REPORT   0

While we have five evdev events here, those represent one hardware event with an x movement of 1 and a second hardware event with a diagonal movement by 1/1. Glorious, we all understand evdev now (if not, read this and immediately afterwards this, although that second post will be rather reinforced by this post).

Life as software developer would be quite trivial but our universe hates us and we need an extra event code called SYN_DROPPED. This event is used by the kernel when events from the device come in faster than you're reading them. This shouldn't happen given that most input devices scan out at the casual rate of every 7ms or slower and we're not exactly running on carrier pigeons here. But your compositor has been a busy bee rendering all these browser windows containing kitten videos and thus completely neglected to check whether you've moved the finger on the touchpad recently. So the kernel sort-of clears the current event buffer and positions a shiny steaming SYN_DROPPED in there to notify the compositor of its wrongdoing. [1]

Now, we could assume that every evdev client (libinput, every Xorg driver, ...) knows how to handle SYN_DROPPED events correctly but we're self-aware enough that we don't. So SYN_DROPPED handling is wrapped via libevdev, in a way that lets the clients use almost exactly the same processing paths they use for normal events. libevdev gives you a notification that a SYN_DROPPED occured, then you fetch the events one-by-one until libevdev tells you have the complete current state of the device, and back to kittens you go. In pseudo-code, your input stack's event loop works like this:

while (user_wants_kittens):
   event = libevdev_get_event()

   if event is a SYN_DROPPED:
     while (libevdev_is_still_synchronizing):
        event = libevdev_get_event()
        process_event(event)
   else:
        process_event(event)

Now, this works great for keys where you get the required events to release or press new keys. This works great for relative axes because meh, who cares [2]. This works great for absolute axes because you just get the current state of the device and done. This works great for touch because, no wait, that bit is awful.

You see, the multi-touch protocol is ... special. It uses the absolute axes, but it also multiplexes over those axes via the slot protocol. A normal two-touch event looks like this:

EV_ABS ABS_MT_SLOT         0
EV_ABS ABS_MT_POSITION_X   123
EV_ABS ABS_MT_SLOT         1
EV_ABS ABS_MT_POSITION_X   456
EV_ABS ABS_MT_POSITION_Y   789
EV_ABS ABS_X               123
EV_SYN SYN_REPORT          0

The first two evdev events are slot 0 (first touch [3]), the second two evdev events are slot 1 (second touch [3]). Both touches update their X position but the second touch also updates its Y position. But for single-touch emulation we also get the normal absolute axis event [3]. Which is equivalent to the first touch [3] and can be ignored if you're handling the MT axes [3] (I'm getting a lot of mileage out of that footnote). And because things aren't confusing enough: events within an evdev frame are position-independent except the ABS_MT axes which need to be processed in sequence. So that ABS_X events could be anywhere within that frame, but the ABS_MT axes need to be grouped by slot.

About that single-touch emulation... We also have a single-touch multi-touch protocol via EV_KEY. For devices that can only track N fingers but can detect N+M fingers, we have a set of BTN_TOOL defines. Two fingers down sets BTN_TOOL_DOUBLETAP, three fingers down sets BTN_TOOL_TRIPLETAP, etc. Those are just a bitfield though, so no position data is available. And it tops out at BTN_TOOL_QUINTTAP but then again, that's a good maximum backed by a lot of statistical samples from users hands. On many devices, we have to combine that single-touch MT protocol with the real MT protocol. Synaptics touchpads on PS/2 only support 2 finger positions but detect up 5 touches otherwise [4]. And remember the ALPS devices? They say they have 4 slots but may only send data for two or three, so we have to detect this at runtime and switch to the BTN_TOOL bits for some touches.

So anyway, now that we unfortunately all understand the MT protocol(s), let's look at that libevdev bug. libevdev checks the slot states after SYN_DROPPED to detect whether any touch has stopped or started during SYN_DROPPED. It also detects whether a touch has changed, i.e. the user lifted the finger(s) and put the finger(s) down again while SYN_DROPPED was happening. For those touches it generates the events to stop the original touch, then events to start the new touch. This needs to be done over two event frames, i.e. with a SYN_REPORT in between [5]. But the implementation ended up splitting those changes - any touch that changed was terminated in the first event frame, any touch that outright stopped was terminated in the second event frame. That in itself wasn't the problem yet, the problem was that libevdev didn't emulate the single-touch multi-touch protocol with those emulated frames. So we ended up with event frames where slots would terminate but the single-touch protocol didn't update until a frame later.

This doesn't matter for most users. Both protocols were still correct-enough in their own bubble, only once you start mixing protocols was where it all started getting wonky. libinput does this because it has to, too many devices out there only track two fingers. So if you want three-finger tapping and pinch gestures, you need to handle both protocols. Despite this we didn't notice until we added the quirk for ALPS devices. Because now libinput sometimes noticed that after a SYN_DROPPED there were no fingers on the touchpad (because they all stopped/changed) but the BTN_TOOL bits were still on so clearly we have a touchpad that cannot track all fingers it detects - in this case zero. [6]

So to recap: libinput's auto-adjustment of the touch count for buggy touchpad devices failed thanks to libevdev's buggy workaround of the device sync. The device sync we need because we can't rely on userspace handling touches correctly across SYN_DROPPED. An event which only gets triggered because the compositor is too buggy to read input events in time. I don't know how to describe it exactly, but what I can see all the way down are definitely not turtles.

And the sad thing about it: if we didn't try to correct for the firmware and accepted that gestures are just broken on ALPS devices because the kernel driver is lying to us, none of the above would have mattered. Likewise, the old xorg synaptics driver won't be affected by this because it doesn't handle multitouch properly anyway, so it doesn't need to care about these discrepancies. Or, in other words and much like real life: the better you try to be, the worse it all gets.

And as the take-home lesson: do upgrade to libinput 1.15.2 and do upgrade to libevdev 1.9.0 when it's out. Your kittens won't care but at least that way it won't make me feel like I've done all this work in vain.

[0] Unless the SYN_REPORT value is nonzero but let's not confuse everyone more than necessary
[1] A SYN_DROPPED is per userspace client, so a debugging tool reading from the same event node may not see that event unless it too is busy with feline renderings.
[2] yes, you'll get pointer jumps because event data is missing but since you've been staring at those bloody cats anyway, you probably didn't even notice
[3] usually, but not always
[4] on those devices, identifying a 3-finger pinch gesture only works if you put the fingers down in the correct order
[5] historical reasons: in theory a touch could change directly but most userspace can't handle it and it's too much effort to add now
[6] libinput 1.15.2 leaves you with 1 finger in that case and that's good enough until libevdev is released

User-specific XKB configuration - part 1

The xkeyboard-config project is the repository for all XKB descriptions, or "keyboard layouts" as the layman would say. But languages are weird and thus xkeyboard-config contains an obscene amount of different layouts. And of course there are additional layouts that are more experimental than common [1].

The fault, as usual, lies with us (the pronoun, not the layout). XKB is weird and its flexible to the point of driving even bananas bananas but due to some historic accidents it's largely non-editable. All XKB files are installed in system folders and we all know the 11th commandment was "thou shalt not edit things in /usr/share". But, luckily, that is all about to change. Or rather: it has changed as of libxkbcommon 0.10.0, released Jan 20 2020.

xkeyboard-config provides two types of files. The ones that actually set up your keyboard layout and the ones that allow you to keep sane while doing so, despite your best efforts to the contrary.

KcCGST

Let's look at the first set of files. XKB uses "keycodes, compat, geometry, symbols, types", conveniently if unpronouncably called KcCGST. Keycodes map your "physical" scancode to an internal code-name. For example, your key with the digit 1 on it is AE01 (alphabetic key, row E from bottom, key number 1 from left). Then you map those keycodes into symbols (1 and !). This happens based on the key's type which defines the combination of modifiers to produce the symbols [2]. Compat is largely magic weird stuff (locking modifiers, pointer control) and geometry would let you draw a pretty picture of your keyboard if it was defined for your keyboard which it won't be.

To see the full keyboard layout simply run xkbcomp -xkb $DISPLAY - and marvel. xkeyboard-config keeps all these parts so your X server or Wayland compositor can load it at runtime depending on your layout.

RMLVO

But when it comes to actual layout selection we like our users enough that we don't make them handle KcCGST but rather provide them with RMLVO instead - "rules, models, layouts, variants, options". You select layout "us" and something converts this into the right components to actually load. Run setxkbmap -layout us -print to see this happening.

"layouts" is what you'd usually associate with a country (except politics is still a thing, so more weirdness here) and "variants" are variations of those. Layout "us" gives you QWERTY and "fr" gives you AZERTY but the "us(dvorak)" variant gives you whatever heresy dvorak applies to those physical keys. And of course, things don't stop there - options are tack-on thingies that do stuff. Like remapping caps lock to compose so you're less capable of shouting at me. Come to think of it, it should really be enabled by default for that reason. You can combine multiple options largely at-will. "models" are largely obsolete (except where they aren't) thanks to the Linux kernel evdev interface which makes all keyboard look the same. But they used to be a thing and maybe one day they'll make a comeback like bell bottom jeans. Disliked by everyone but some weirdos insist on using them.

Rules

Finally, we have rules and thus come to the core of the matter of this post. Rules are magic files that tell the various tools how to go from RMLVO to KcCGST. It's a weird format but it's quite understandable, just open /usr/share/X11/xkb/rules/evdev and have a looksie. It'll make you the popular kid at the next frat party.

Many many moons ago before the Y2K bug was even in its larvae stage, the idea was that you could configure all of those because every UNIX tool had to be more flexible than your yoga teacher. I'm unsure to what extent this was actually ever the case but around 2007-ish the old keyboard driver got deprecated and the evdev driver made it's grand entrance. And one side-effect of that was that things broke. evdev uses different keycodes, so all those users that copy-pasted unnecessary XKB configuration into their xorg.conf now had broken keys because they were applying the wrong rules. After whacking enough moles that we got in trouble with the RSPCA we started hardcoding the "evdev" ruleset everywhere. The xorg.conf option "XKBRules" became a noop and thus stopped breaking users' setups.

Except that it also stopped users from deploying their own rules files - something that probably didn't really matter anyway. This had some unintended side-effects though. First, to have a working custom XKB layout you basically had to get it merged upstream. Yes, you could edit the files locally but they'd just be overwritten next time you update the packages. Second, getting rid of hardcoded things is hard so we're stuck with the evdev ruleset for the forseeable future. This was the situation until, well, now.

User-specific rules and layouts

The new libxkbcommon release changes two things: it prepends $XDG_CONFIG_HOME/xkb/ to the lookup path for XKB rules (and other files). So any file in that path will be picked before the system paths. This makes it possible to have KcCGST files in your home directory and actually use them. This was somewhat possible before by passing the right flags to the various tools but now it's on by default - at least where libxkbcommon is used (Wayland).

Secondly, rules files now support an include statement. This means you can set your own rules and include system rules. Because everything is hardcoded to evdev this effectively means your new rule file will be $XDG_CONFIG_HOME/xkb/rules/evdev and have at least one line: ! include %S/rules/evdev. If you do just that, you get the evdev ruleset from the system installation path. And any lines you add before or after that line will be loaded. Have a look at the git commit for the details but the summary is that you'll have a rules file that looks like this:

$ cat $XDG_CONFIG_DIR/xkb/rules/evdev
! option                =       symbols
  custom:foo            =       +custom(foo)
  custom:bar            =       +custom(baz)

! include %S/evdev

This file will define the option->symbol mappings as above and then include the system-provided evdev rules file, i.e. it'll behave like before with those two added. To get those to do something, you need to have the actual symbols files:

$ cat $XDG_CONFIG_HOME/xkb/symbols/custom
partial alphanumeric_keys
xkb_symbols "foo" {
    key <TLDE> {        [      VoidSymbol ]       };
};

partial alphanumeric_keys
xkb_symbols "baz" {
    key <AB01> {        [      k, K ]       };
};

And voila, you can now use the XKB option "custom:foo" and/or "custom:bar" to remap your tilde or Z key. The rest is left to the reader as an exercise in creativity.

Remaining work

The libxkbcommon change was only the first part of the full feature. The remaining parts is to have libxkbcommon actually resolve XDG_CONFIG_HOME when running in gnome-shell which doesn't work right now thanks to secure_getenv() always returning NULL. That's an issue with gnome-shell in particular thanks to the rt-scheduler feature, enabled by default on Fedora already.

The second part, and harder, is to make the new options appear in the various graphical configuration tools. xkeyboard-config ships an XML file [3] that lists every possible combination with some human description for it. This XML file is used by the various tools directly but none of those tools support XML's XI:Include statements. So we'd either have to update all those tools to extend the parsing accordingly or, most likely a smarter long-term solution, write a wrapper library that provides a stable API to get at the same info. That way we can update the include paths under the hood without having to update every tool. Of course this requires every tool to update to the new library first, so, well, chicken, egg, usual problem. Anyway, we'll get there eventually.

[1] For example, I suspect a meetup of icelandic dvorak users doesn't qualify for a group discount.
[2] Each key has "levels" with one symbol each and modifiers that switch between those levels. Most keys have two levels - normal and shift. But there's a key type for EIGHT_LEVEL_ALPHABETIC_LEVEL_FIVE_LOCK and you can cry or laugh and both reactions are appropriate
[3] ask your grandparents about that, it's basically JSON for old people

libinput's timer offset errors

Let's say you have a friend (this wouldn't happen to you, of course, just that friend) who is staring at their system logs and wonder why it is full of messages similar to this:

 libinput error: client bug: timer event5 debounce short: offset negative (-7ms)

And the question is of course - what is going on here and why hasn't this been fixed yet. Now, the libinput documentation explains this already but it's always worthwhile to fire out a blog post into the void in the hope someone reads it.

libinput uses a specific model to communicate with the Wayland compositor (or the X server). There is a single epoll file descriptor and that fd will trigger whenever something happens that's of interest to libinput. When that fd triggers, the compositor is expected to call libinput_dispatch() which is the main "do stuff" function of libinput.

The actual trigger doesn't matter, it could be an event from a device but it could be something else. The caller doesn't have to care. All that matters is that there is code like this:

if (libinput_fd_triggered_in_select)
   libinput_dispatch();

And then libinput will do the right thing. Whether you also want events from libinput is almost orthogonal to this.

libinput uses timerfd internally so any timeouts also trigger the epoll fd. Timeouts are scheduled based on the event's time stamp, so if you get an event with timestamp T, a timeout of 180ms will be scheduled for time T + 180ms. So the process looks something like this:

T(0): kernel button event
T(0): libinput_dispatch(): schedule timeout for T(0+180)
...
T(180): epoll fd triggers
T(180): libinput_dispatch(): process timeout
...

This works generally fine. Even with some delays we don't generally need to worry about the timeouts and they still trigger as expected. But some of the timeouts are "short", as in 8ms short. And this is where these warnings may trigger.

Let's say your compositor is busy doing some rendering. The epoll fd triggers with a button event but the compositor is too busy to handle it immediately. Instead, it finishes whatever it's doing and only then calls libinput_dispatch():

T(0): kernel button event
...
T(12): libinput_dispatch(): schedule timeout for T(0+8)

libinput error: client bug: timer event5 debounce short: offset negative (-4ms)

libinput will still use the event's timestamp instead of the wall clock time so the scheduled timeout is no longer in the future. And that is when the error message will be printed. This isn't a libinput bug, it's always a bug in the compositor. Especially gnome-shell is still struggling with these instances and while great strives have been made to make it more responsive, it can still happen.

The error message may seem cryptic, but it provides a bunch of useful information: event5 is your event node, "debounce short" is the timer name so we know where we got stuck. And 4ms gives us an indication how much we got delayed.

And for the record: the other end of this issue - delayed libinput_dispatch() after a timeout should have triggered is handled quietly by libinput. For example, if you have a physical event queued and a timeout expiry, we will process the earlier one first to make sure the sequences are handled correctly.

libinput and tablet pad keys

Upcoming in libinput 1.15 is a small feature to support Wacom tablets a tiny bit better. If you look at the higher-end devices in Wacom's range, e.g. the Cintiq 27QHD you'll notice that at the top right of the device are three hardware-buttons with icons. Those buttons are intended to open the config panel, the on-screen display or the virtual keyboard. They've been around for a few years and supported in the kernel for a few releases. But in userspace, they events from those keys were ignored, casted out in the wild before eventually running out of electrons and succumbing to misery. Well, that's all changing now with a new interface being added to libinput to forward those events.

Step back a second and let's look at the tablet interfaces. We have one for tablet tools (styli) and one for tablet pads. In the latter, we have events for rings, strips and buttons. The latter are simply numerically ordered, so button 1 is simply button 1 with no special meaning. Anything more specific needs to be handled by the compositor/client side which is responsible for assigning e.g. keyboard shortcuts to those buttons.

The special keys however are different, they have a specific function indicated by the icon on the key itself. So libinput 1.15 adds a new event type for tablet pad keys. The events look quite similar to the button events but they have a linux/input-event-codes.h specific button code that indicates what they are. So the compositor can start the OSD, or control panel, or whatever directly without any further configuration required.

This interface hasn't been merged yet, it's waiting for the linux kernel 5.4 release which has a few kernel-level fixes for those keys.

libinput and button scrolling locks

For a few years now, libinput has provided button scrolling. Holding a designated button down and moving the device up/down or left/right creates the matching scroll events. We enable this behaviour by default on some devices (e.g. trackpoints) but it's available on mice and some other devices. Users can change the button that triggers it, e.g. assign it to the right button. There are of course a couple of special corner cases to make sure you can still click that button normally but as I said, all this has been available for quite some time now.

New in libinput 1.15 is the button lock feature. The button lock removes the need to hold the button down while scrolling. When the button lock is enabled, a single button click (i.e. press and release) of that button holds that button logically down for scrolling and any subsequent movement by the device is translated to scroll events. A second button click releases that button lock and the device goes back to normal movement. That's basically it, though there are some extra checks to make sure the button can still be used for normal clicking (you will need to double-click for a single logical click now though).

This is primarily an accessibility feature and is likely to find it's way into the GUI tools under the accessibility headers.

libinput's bus factor is 1

A few weeks back, I was at XDC and gave a talk about various current and past input stack developments (well, a subset thereof anyway). One of the slides pointed out libinput's bus factor and I'll use this blog to make this a bit more widely known.

If you don't know what the bus factor is, Wikipedia defines it as:

The "bus factor" is the minimum number of team members that have to suddenly disappear from a project before the project stalls due to lack of knowledgeable or competent personnel.

libinput has a bus factor of 1.

Let's arbitrarily pick the 1.9.0 release (roughly 2 years ago) and look at the numbers: of the ~1200 commits since 1.9.0, just under 990 were done by me. In those 2 years we had 76 contributors in total, but only 24 of which have more than one commit and only 6 contributors have more than 5 commits. The numbers don't really change much even if we go all the way back to 1.0.0 in 2015. These numbers do not include the non-development work: release maintenance for new releases and point releases, reviewing CI failures [1], writing documentation (including the stuff on this blog), testing and bug triage. Right now, this is effectively all done by one person.

This is... less than ideal. At this point libinput is more-or-less the only input stack we have [2] and all major distributions rely on it. It drives mice, touchpads, tablets, keyboards, touchscreens, trackballs, etc. so basically everything except joysticks.

Anyway, I'm largely writing this blog post in the hope that someone gets motivated enough to dive into this. Right now, if you get 50 patches into libinput you get the coveted second-from-the-top spot, with all the fame and fortune that entails (i.e. little to none, but hey, underdogs are big in popular culture). Short of that, any help with building an actual community would be appreciated too.

Either way, lest it be said that no-one saw it coming, let's ring the alarm bells now before it's too late. Ding ding!

[1] Only as of a few days ago can we run the test suite as part of the CI infrastructure, thanks to Benjamin Tissoires. Previously it was run on my laptop and virtually nowhere else.
[2] fyi, xf86-input-evdev: 5 patches in the same timeframe, xf86-input-synaptics: 6 patches (but only 3 actual changes) so let's not pretend those drivers are well-maintained.