xmodmap keyboard deconfiguration

Reading this post on linux.com last weekend made me cringe. Not because it is factually wrong (it isn't), but because what's described here is the reason for a great many bugreports and some rather uninformed hatemail. One thing that startled me in particular is the hint that "to really take charge, you need to dig into the X keyboard extension" when the rest of the article doesn't deal with XKB at all. This prompted me to write this blurb here.

The abstraction cardhouse

In X, keyboard input is abstracted. The keyboard sends a keycode to the client. The client then translates that keycode into a keysym (a glyph, in most cases, more or less), given a keycode-keysym table. Keycodes should be treated as arbitrary by the client. Keysyms on the other hand are constant, the keysym for "♥" is always the same (0x1002665). In the core protocol, that lookup table was reasonably straightforward (for some value of "straightforward"), with several modifiers providing access to different levels in the table. The table being based on the max number of modifiers. For 105 keys and two modifiers, the table was essentially 210 symbols wide (modifiers can be "shift", "alt", etc.).

That was deemed to be insufficient, causing the development of the X Keyboard Extension, lovingly referred to as "XKB" or "that bloody nail in my coffin". I won't go into details, but the lookup tables are a tad more complicated. XKB is integrated around core so that XKB and core applications can coexist nicely. Up to a point that for a long time we kept XKB state and core state separately inside the server instead of generating it on-the-fly. What's unfortunate though is that the XKB protocol spec describes how to get the core lookup table from the XKB data. It also describes how to extrapolate from core to XKB. It doesn't however cover the full circle. So if you go from XKB to core back to XKB, guesswork happens.

Anyway, what you need to take from this is that the general rule of thumb is - any commandline tool uses core, any tool that allows you to select a layout, variant or model uses XKB.
And core doesn't support device specifiers, so it can only ever work on the default device or core device, more on that below.

Pulling out the cards

All the above was mostly good until 2007. That's when input hotplug happened. And 2009, when MPX happened. Now none of the old tools work properly anymore. Not because we've broken them, they still do what they've always done. But the environment has changed and what they do isn't sufficient anymore. So before you think about writing an angry comment that we've broken your configuration: stop using input and output hotplugging, stop using suspend/resume, stop using a desktop environment that manages your keyboard configuration. Then you'll notice everything still works as before and you can xmodmap away.

So why did everything break since?
Back before input hotplug we had one device that was deemed the "core keyboard". Whenever anything configured the device it set the configuration on this core device. Since xset, xmodmap and friends work on core only, the XKB configuration was then modified from that core table. All that was fine, because the core keyboard was hardly ever changed. Now with input-hotplug and MPX, we have the physical devices, attached to what is now called "master devices" but still serve the same purpose as the old "core keyboard". However, we've greatly increased the flexibility of the devices and physical devices are more often than not configured directly now. What we then end up with is a master device that has multiple keyboards configured, all with possible different keyboard configurations. In fact, non-US layout users in Fedora 12 are practically guaranteed to have multiple keyboard configurations. We only apply the user-selected layout to actual keyboard devices, not to devices with keys (Power Button, Video Bus, etc.). These devices come up with the "US" default.

In the case of a user with two physical keyboards, one qwerty, one azerty, the matching master device needs to adjust the keyboard configuration of the device in use. This means that it will switch between the two configurations as the devices are used. That works fine too. But if you now use xmodmap to change the the keyboard config on the master device, this configuration only stays as long as the same physical keyboard device is being used.
Now, the simple fix for this is to apply the new configuration to all devices instead of just the master device. And that's exactly what we do. However, this doesn't work with hotplugging. Once a new device is plugged in, it comes up with the configuration configured in the xorg.conf again. And that config doesn't include whatever xmodmap settings were applied later. So once that device is used, the settings are lost again or at least lost on this device.
Now sprinkle into this the fact that core doesn't really support multiple groups (layouts) on the same keyboard, that keycodes may be different between devices, that with MPX a device that was a slave to an xmodmap configured master device may end up being attached to a master that has a different XKB configuration and you've got the start of an interesting career.

The fix for all this is simple: make the session handle all keyboards, existing and hotplugged ones, and let it update the config accordingly. Whenever you're using a tool like xmodmap it is your responsibility to reapply the config after changing keyboards. Which brings me back to the article above - it curiously didn't mention that. So I strongly recommend that instead of configuring your keyboard with xmodmap you spend your time helping your favourite desktop environment to provide tools to cover your use-case.


PS: surprisingly "deconfiguration" doesn't exist in the dictionary but it seems the best word to describe what xmodmap does.