Monday, April 18, 2016

libinput and graphics tablet pad support

When we released graphics tablet support in libinput earlier this year, only tablet tools were supported. So while you could use the pen normally, the buttons, rings and strips on the physical tablet itself (the "pad") weren't detected by libinput and did not work. I have now merged the patches for pad support into libinput.

The reason for the delay was simple: we wanted to get it right [1]. Pads have a couple of properties that tools don't have and we always considered pads to be different to pens and initially focused on a more generic interface (the "buttonset" interface) to accommodate for those. After some coding, we have now arrived at a tablet pad-specific interface instead. This post is a high-level overview of the new tablet pad interface and how we intend it do be used.

The basic sign that a pad is present is when a device has the tablet pad capability. Unlike tools, pads don't have proximity events, they are always considered in proximity and it is up to the compositor to handle the focus accordingly. In most cases, this means tying it to the keyboard focus. Usually a pad is available as soon as a tablet is plugged in, but note that the Wacom ExpressKey Remote (EKR) is a separate, wireless device and may be connected after the physical pad. It is up to the compositor to link the EKR with the correct tablet (if there is more than one).

Pads have three sources of events: buttons, rings and strips. Rings and strips are touch-sensitive surfaces and provide absolute values - rings in degrees, strips in normalized [0.0, 1.0] coordinates. Similar to pointer axis sources we provide a source notification. If that source is "finger", then we send a terminating out-of-range event so that the caller can trigger things like kinetic scrolling.

Buttons on a pad are ... different. libinput usually re-uses the Linux kernel's include/input.h event codes for buttons and keys. But for the pad we decided to use plain sequential button numbering, starting at index 0. So rather than a semantic code like BTN_LEFT, you'd simply get a button 0 event. The reasoning behind this is a caveat in the kernel evdev API: event codes have semantic meaning (e.g. BTN_LEFT) but buttons on a tablet pad don't those meanings. There are some generic event ranges (e.g. BTN_0 through to BTN_9) and the Wacom tablets use those but once you have more than 10 buttons you leak into other ranges. The ranges are simply too narrow so we end up with seemingly different buttons even though all buttons are effectively the same. libinput's pad support undoes that split and combines the buttons into a simple sequential range and leaves any semantic mapping of buttons to the caller. Together with libwacom which describes the location of the buttons a caller can get a relatively good idea of how the layout looks like.

Mode switching is a commonly expected feature on tablet. One button is designated as mode switch button and toggles all other buttons between the available modes. On the Intuos Pro series tablets, that button is usually the button inside the ring. Button mapping and thus mode switching is however a feature we leave up to the caller, if you're working on a compositor you will have to implemented mode switching there.

Other than that, pad support is relatively simple and straightforward and should not cause any big troubles.

[1] or at least less wrong than in the past
[2] They're actually linux/input-event-codes.h in recent kernels

Thursday, April 7, 2016

Why libinput doesn't have a lot of config options

Most days, at least one of the bugs I deal with requests something along the lines of "just add $FOO as a config option". In this post, I'll explain why this is usually a bad solution. First, read http://www.islinuxaboutchoice.com/ and keep those arguments in mind. Generally, there are two groups of configuration options - hardware options and user options. Hardware options are those that deal with specific quirks needed on some hardware, but not on other hardware. User options are those that deal with user preferences such as tapping or two-finger vs. edge scrolling.

In the old synaptics driver, we added options whenever something new came up and we tried to make those options generic. This was a big mistake. The driver now has over 70 configuration options resulting in a test matrix with a googolplex of combinations. In other words, it's completely untestable. To make a device work users often have to find the right combination of options from somewhere, write out a root-owned config file and then hope this works. Why do we still think this is acceptable? Even worse: some options are very specific to hardware but still spread in user forum examples like an STD during spring break.

In libinput, we're having none of that. When hardware doesn't work we expect a user to file a bug, we get it fixed upstream for the specific model and thus automatically fix it for all users of that device. We're leaning heavily on udev's hwdb which we have extended to correct devices when the firmware announces wrong information. This has the advantage that there is only one authoritative source of quirks a device needs to work. And we can update this as time goes by without having to worry about stale configuration options. One good example here is the custom acceleration profile that Lenovo X230 touchpads have in libinput. All in all, there is little pushback for the lack of hardware-specific configuration options and most users are fine with it once they accept the initial waiting period to get the patch into their distribution.

User-specific options are more contentious. In our opinion, some features should be configurable and others should not. Where to draw that line is of course quite undefined. For example, tapping on or off was one of the first configuration options available and that was never a cause for arguments either way (except whether the default should be on or off). Other options are more contentious. Clickpad software buttons are always on the bottom edge and their size is hardcoded (synaptics allowed almost free positioning of those buttons). Other features such as changing a two-finger tap to some other button event is not supported at all in libinput. This effectively comes down to cost. You see, whenever you write "it's just 5 lines of code to make this an option", what I think is "once the patch is reviewed and applied, I'll spend two days to write test cases and documentation. I'll need to handle any bug reports related to this, and I'm expected to make sure this option works indefinitely. Any addition of another feature may conflict with this option, so I need to make sure the right combination is possible and test cases are written." So your work ends after writing a 5 line patch, my work as maintainer merely starts. And unless it pays off long-term, the effort is not worth it. Some features make that cut, others don't if they are too much of a niche feature.

All this is of course nothing new and every software project needs to make these decisions. Input isn't even a special case here, it pales in comparison with e.g. the decisions UI designers need to make. However, in FOSS we have a tendency to think that because something is possible, it should be done. Legally, you have freedom to do almost anything with the software, so you can maintain a local fork of libinput with that extra feature applied. If that isn't acceptable, why would it be acceptable to merge the patch and expect others to shoulder the costs?

Wednesday, April 6, 2016

libinput now has a touchpad software middle button

I just pushed a patch to libinput master to enable a middle button on the clickpad software buttons. Until now, our stance was that clickpads only get a left and right software button, split at the 50% mark. The reasoning is simple: touchpads only have markings for left and right buttons (if any!) and the middle button's extents are not easily discoverable if there is no visual or haptic feedback. A middle button event could however be triggered through middle button emulation, i.e. by clicking the touchpad with a finger on the left and right software button area (see the instructions here).

This is nice in theory but, as usual, reality gets in the way. Most interactions with the middle button are quick and short-lived, i.e. clicking the button once to paste. This interaction is what many touchpads are spectacularly bad at. For middle button emulation to be handled correctly, both fingers must be registered before the physical button press. The scanout rate on a touchpad is often too low and on touchpads with extremely light resistance like the T440 it's common to register the physical click before we know that there's a second finger on the touchpad. But even on a T450 and an X220 with much higher clickpad resistance I barely managed to get above 7 out of 10 correctly registered middle button events. That is simply not good enough.

So the patch I just pushed out to master enables a middle software button between the left and the right button. The exact width of the button scales with the touchpad but it's usually around 20-25mm and it's centered on the touchpad so despite the lack of visual or haptic guides it should be reliable to hit. The new behaviour is hard-coded and for now middle button emulation continues to work on touchpads. In the future, I expect I will remove middle button emulation on touchpads or at least disable it by default.

The middle button will be available in libinput 1.3.

Monday, March 14, 2016

why libinput does not have a configuration storage system

A question that pops up with some regularity is whether libinput has a global configuration storage system, and, subsequently, why it doesn't have one. Comparisons are drawn to the X-specific tool xinput that allows to trigger all configuration options (see below though).

As maintainer of libinput, I can state that libinput will not get a configuration storage system. That job is squarely in the realm of the caller (i.e. usually the compositor and/or the desktop environment). Here are a few reasons:

First, you'd get conflicts with the caller. You need to prioritise which configuration has precendence to decide what to do when libinput and the caller disagree on a configuration item. You (the user) also have to figure why a configuration doesn't work when it's clearly enabled in one of those systems. Ok, so you can work around this by putting in a warning somewhere, provided that you make sure that the warning shows up in the right place and users know where to look. And to know when to send the warning, because again, that requires libinput to know which config has priority.

This is amplified by the lack of an autoritative support system. Speaking from X experience, the number of posts on, say, the ubuntu forums that advocate setting configuration options that haven't existed for years is quite sad. Or users advocate config snippets that set everything but the feature they claim it enables. That gets copy-pasted, etc.

Some configuration options can be incompatible with each other. If you have conflicting configuration systems it gets harder because each configuration system cannot make certain options an either/or anymore. We don't have any of these options just yet, but they may come in the future.

Over time, supported features will change. A setting may be exposed in the magic libinput config system and, a few months later, it is now also exposed by, say, GNOME. All the documentation that points to the libinput configuration is now obsolete because GNOME overrides it. Unless you're running version Foo.Bar, or maybe the patched version from $DISTRIBUTION. It gets messy quickly.

How fine-grained do you need the configuration? libinput's config API applies separately for each device. For most desktop environments it is sufficient to have configuration per device type (touchpad vs mouse vs tablet, etc.). But some exceptions may apply and a newer generic touchpad configuration may need to override device-specific older configuration. Because the newer config is designed around the new libinput version, but the older config is something you copied off a forum post from 2 years ago.

Oh, implicit in the request for a configuration system in libinput is usually also: please write and maintain that system for free, and fix any bugs in a timely manner. And I can't shake the feeling that a large number of these requests are only triggered by "my desktop doesn't expose this setting, if we get it in libinput I can force it into any desktop".

Ironically enough, the tool that is usually used as an example for how it could work is xinput. The only reason xinput works is because the xf86-input-libinput driver exposes properties and maps changes in those to the libinput API calls. i.e. everything is handled by libinput's caller, not libinput itself. Just as it should be.

Lest I be accused of only shooting down ideas here is a constructive idea: write a library or DBus-daemon that exposes some configuration storage and makes it easy to query. Then convince the various desktop environments to use that system instead their existing solutions, and bang, you have a generic configuration storage system for libinput.

Tuesday, March 1, 2016

libinput and graphics tablet support

Last week's libinput 1.2 release included the new graphics tablet support. This work, originally started as Lyude's 2014 GSoC project enables the use of drawing tablets through libinput (think Wacom tablets, not iPad/Android tablet).

Wacom tablets provide three input types: pen-like tools, buttons and touch rings/strips on the tablet itself, and touch. libinput's tablet support work focuses on the tool support only, pad buttons are still work in progress. Touch is already supported, either through the touchpad interfaces for external tablets (e.g. Intuos) or touchscreen interfaces for direct-touch tablets (e.g. Cintiq). So the below only talks about how to get events from tools, the pad events will be covered in a separate post when it's ready.

How do things work in the xf86-input-wacom driver, the current standard for tablet support in Linux? The driver checks the kernel device node for capabilities and creates multiple X devices, usually pen, eraser, cursor, and pad. When a pen comes into proximity the driver sends events through the pen device, etc. The pen device has all possible axes available but some (e.g. rotation) won't be used unless you're actually using an Wacom Art Pen. Unless specifically configured, all pens send through the same device, all erasers send through the same device, etc.

The libinput tablet API is a notable departure from this approach. In libinput, each tool is a separate entity generating events. A client doesn't wait for events from the tablet, it waits for events from a tool. The tablet itself is little more than an idle device. This has a couple of effects:

  • A struct libinput_tablet_tool is created on-the-fly as a tool comes into proximity and its this struct that events are tied to.
  • This means we default to per-tool handling. Two pens will always be separate and never multiplexed through one device. [1]
  • The tool can be uniquely identified [1]. It's easy to track a tool across two tablets even though this is quite a niche case.
  • A client can query the tool struct for capabilities, but not the tablet. Hence you cannot know what capabilities are available until a tool is in proximity.
  • The tool-based approach theoretically enables us to have multiple tools in proximity simultaneously, though no current hardware supports this.
Now, the advantages for the professional use-case where artists have multiple tools and/or multiple tablets is quite obvious. But it also changes some things for the average user with a single tool, specifically: the data provided by libinput is now precise and reflects the tool you're using. No more fake rotation axis on your standard pen. But you cannot query that information until the pen has been in proximity at least once. This is a change that clients will have to eventually deal with.

I just pushed the tablet support for the xf86-input-libinput driver and this driver reflects the new approach that libinput takes. When a tablet is detected, we create one device that has no axes and serves as the parent device. Once the first tool comes into proximity, a new device is created with the exact capabilities that the tool provides and the serial number in the name:

$> xinput list
⎡ Virtual core pointer                     id=2 [master pointer  (3)]
[...]
⎜   ↳ Wacom Intuos5 touch M Pen Pen (0x99800b93) id=21 [slave  pointer  (2)]
⎣ Virtual core keyboard                    id=3 [master keyboard (2)]
[...]
    ↳ Wacom Intuos5 touch M Pen                id=11 [slave  keyboard (3)]
Device 11 is effectively mute (in the future this may become the Pad device, not sure yet). Device 21 appeared once I moved the tool into proximity. That tool has all the properties for configuration as they appear on the device. So far this works, but it means static configuration becomes more difficult. If you are still using xinit scripts or other scripts that only run once on login and not once per new device then the options may not apply correctly.

[1] provided the hardware can uniquely identify the pens

Monday, January 25, 2016

libinput and semi-mt touchpads

libinput 1.1.5 has a change in how we deal with semi-mt touchpads, in particular: interpretation of touch points will cease and we will rely on the single touch position and the BTN_TOOL_* flags instead to detect multi-finger interaction. For most of you this will have little effect, even if you have a semi-mt touchpad. As a reminder: semi-mt touchpads are those that can detect the bounding box of two-finger interactions but cannot identify which finger is which. This provides some ambiguity, a pair of touch points at x1/y1 and x2/y2 could be a physical pair of touches at x1/y2 and x2/y1. More importantly, we found issues with semi-mt touchpads that go beyond the ambiguity and reduce the usability of the touchpoints.

Some devices have an extremely low resolution when two-fingers are down (see Bug 91135), the data is little better than garbage. We have had 2-finger scrolling disabled on these touchpads since before libinput 1.0. More recently, Bug 93583 showed that some semi-mt touchpads do not assign the finger positions for some fingers, especially when three fingers are down. This results in touches defaulting to position 0/0 which triggers palm detection or results in scroll jumps, neither of which are helpful. Other semi-mt touchpads assign a straightforward 0/0 as position data and don't update until several events later (see Red Hat Bug 1295073). libinput is not particularly suited to handle this, and even if it did, the touchpad's reaction to a three-finger tap would be noticeably delayed.

In light of these problems, and since these affect all three big semi-mt touchpad manufacturers we decided to drop back and handle semi-mt touchpads as single-finger touchpads with extra finger capability. This means we track only one touchpoint but detect two- and three-finger interactions. Two-finger scrolling is still possible and so is two- and three-finger tapping or the clickfinger behaviour. What isn't possible anymore are pinch gestures and some of the built-in palm detection is deactivated. As mentioned above, this is unlikely to affect you too much, but if you're wondering why gestures don't work on your semi-mt device: the data is garbage.

Friday, January 22, 2016

Is Wayland ready yet?

This question turns up a lot, on the irc channel, mailing lists, forums, your local Stammtisch and at weddings. The correct answer is: this is the wrong question. And I'll explain why in this post. Note that I'll be skipping over a couple of technical bits, if you notice those then you're probably not the person that needs to ask the question in the first place.

On your current Linux desktop, right now, you have at least three processes running: the X server, a window manager/compositor and your web browser. The X server is responsible for rendering things to the screen and handling your input. The window manager is responsible for telling the X server where to render the web browser window. Your web browser is responsible for displaying this post. The X server and the window manager communicate over the X protocol, the X server and the web browser do so too. The browser and the window manager communicate through X properties using the X server as a middle man. That too is done via the X protocol. Note: This is of course a very simplified view.

Wayland is a protocol and it replaces the X protocol. Under Wayland, you only need two processes: a compositor and your web browser. The compositor is effectively equivalent to the X server and window manager merged into one thing, and it communicates with the web browser over the Wayland protocol. For this to work you need the compositor and the web browser to be able to understand the Wayland protocol.

This is why the question "is wayland ready yet" does not make a lot of sense. Wayland is the communication protocol and says very little about the implementation of the two sides that you want to communicate.

Let's assume a scenario where we all decide to switch from English to French because it sounds nicer and English was designed in the 80s when ASCII was king so it doesn't support those funky squiggles that the French like to put on every second character. In this scenario, you wouldn't ask "Is French ready yet?" If no-one around you speaks French yet, then that's not the language not being ready, the implementation (i.e. the humans) aren't ready. Maybe you can use French in a restaurant, but not yet in the supermarket. Maybe one waiter speaks both English and French, but the other one French only. So whether you can use French depends very much on the situation. But everyone agrees that eventually we'll all speak French, even though English will hang around for ages until it finally falls out of use. And those squiggles are so cute!

Wayland is the same. The protocol is stable and has been for a while. But not every compositor and/or toolkit/application speak Wayland yet, so it may not be sufficient for your use-case. So rather than asking "Is Wayland ready yet", you should be asking: "Can I run GNOME/KDE/Enlightenment/etc. under Wayland?" That is the right question to ask, and the answer is generally "It depends what you expect to work flawlessly." This also means "people working on Wayland" is often better stated as "people working on Wayland support in ....".

An exception to the above: Wayland as a protocol defines what you can talk about. As a young protocol (compared to X with 30 years worth of extensions) there are things that should be defined in the protocol but aren't yet. For example, Wacom tablet support is currently missing. Those are the legitimate cases where you can say Wayland isn't ready yet and where people are "working on Wayland". Of course, once the protocol is agreed on, you fall back to the above case: both sides of the equation need to implement the new protocol before you can make use of it.

Update 25/01/15: Matthias' answer to Is GNOME on Wayland ready yet?