Thursday, January 8, 2009

4 useful things - try this at home.

tig: a curses interface for git. It gives you a mutt-like interface of the git repository and provides you with the ability to define custom key commands, etc. Really good for reviewing patches or lists of patches before pushing them, and even better for branch work. Example: git checkout mybranch; tig master. Hitting C now will cherry-pick the currently selected patch onto the current branch. I have to admit, that only since I started using tig I got a reasonably good handle on my branches. Thanks to Kristian Høgsberg for showing me tig.



cscope_maps: a cscope pluging for vim. It provides a significant advantage over the ctags support as it doesn't require the use of :cnext anymore. Instead, it lists all tags in a list and you can jump straight to them. It also gives you the "who's using this symbol" cscope search on "Ctrl+Space, s".




taglist: another plugin for vim. After hitting :Tlist, it lists the symbols defined in the current file (and also from previous buffers, though I find this sometimes annoying). Great for getting an overview on a new code base, or searching for symbols. The use of Tlist goes down the more familiar I become with a file, but it's still invaluable after a few years of looking at the same few files.



Nostalgy: a Thunderbird plugin that provides keyboard shortcuts to move emails around. Hit S and type the mailbox, including autocompletion, etc. Sorry, no screenshots (I use TB for work email and I'm too lazy to blur out), but there's screenshots at the URL above anyway.

Aside from the usual stuff these four have become must-haves to me.

edit: fix cscope_maps link, thanks to jonathan.

Tuesday, December 2, 2008

evdev, xorg.conf, hal and other FUD

Sparked by this thread, here's a list of X input facts in random order.


  • The evdev driver is the preferred input driver. If you are not running Linux, then evdev is not available for you and you can keep using the mouse/kbd drivers.


  • If you are running Linux, you can keep using the mouse/kbd driver if you wish, but configuration adjustments need to be made. See below for details.


  • By default, the X server will expect a device list at runtime from HAL. You can turn this behaviour off by using Option "AutoAddDevices" "off" in the ServerLayout, or by disabling HAL at configure time. HAL is not required to use X.


  • If HAL is enabled, then the mouse/kbd drivers are disabled. Otherwise, you will get duplicate devices. If you want to use mouse/kbd, disable HAL.


  • Devices only get added if they have the input.x11_driver option set (check lshal). For mice and keyboards, evdev is the default, drivers such as linuxwacom or synaptics provide their own fdi.


  • Any other options need to be added with the key "input.x11_options.foobar" and must be of type string.


  • Devices specified in the xorg.conf will not be available after unplugging and replugging them, but will become available again after a VT switch or a resume.


  • XKB options are specified by HAL, not by the server.


  • The evdev driver will prevent you from adding the same event device twice. The synaptics driver does too. Evdev will not prevent you from adding /dev/input/mice and /dev/input/event1 at the same time.


  • HAL will prevent you from adding the same UDI.


  • AutoAddDevices specifies whether to add devices through HAL (default on).

  • AutoEnableDevices specifies whether to enable such devices immediately (default on).

  • AllowEmptyInput specifies if a mouse/kbd section if none is present is required (if AEI is on -> no mouse/kbd required, disables mouse/kbd if present in the xorg.conf). AllowEmptyInput defaults to (AutoAddDevices && AutoEnableDevices)


  • Synaptics SHM or other options can be enabled in the FDI file with the key "input.x11_options.SHMConfig" (or likewise). Options have to be type string.


  • Synaptics has improved a lot in regards to autoconfiguration and you probably won't need half of your options anymore.


  • For every default setting that annoys you there's at least one person that loves it. For every default setting that you find appropriate, there's at least one person hating it. Become an active contributor, then you get to decide on what the defaults are.


  • Storing configuration in HAL's fdi file is not the best solution but the best we have right now.


  • The old system worked, because you only ever had one mouse and one keyboard in X. Now you can have multiple of each, independently configured. This is why we switched to HAL for hotplugging and autoconfiguring.


  • Ranting and/or commenting on some blog won't get your issues fixed. Filing a bug may.

Wednesday, November 19, 2008

evdev 2.1 is out

After a (too) long wait, evdev 2.1 is finally out. This release should bring evdev up to speed and provide those features that the mouse driver had, but were previously missing from evdev. Get it while it's fresh.

One of the important things here (especially if you're a distributor): evdev 2.1 does not call EVIOCGRAB on the device anymore. Either patch it in again, or update your server to one that doesn't require it (1.5.3 will do).

Here's a short list of new features that have been added since 2.0:
- axis inversion
- touchscreen support (those that don't report BTN_LEFT)
- run-time calibration for absolute devices
- axis swapping
- mouse wheel emulation
- drag lock

Also included is property support for run-time configuration. We don't have a released server that provides this option yet, but it'll be good for a test-bed until 1.6 comes out.
The properties also caused the long wait for the release, I wanted to wait until the server API had settled down. The next release will be sooner. The proposed release date for X server 1.6 is
Jan 5 2009. Evdev 2.2 will be released by Jan 5 2009 as well.

A great thanks to Adam Jackson, Ander Conselvan de Oliveira, Chris Salch, Dan Nicholson, Daniel Stone, Fernando Carrijo, Julien Cristau, Keith Packard, Michel Dänzer, Simon Munton, and Søren Hauberg for their patches.

Sunday, October 19, 2008

Avoid evdev devices in xorg.conf

Quick addition to my previous blog post:

If you have an xorg.conf that references evdev devices, these will generate duplicate events with the recent changes (they get added twice). Just get rid of the matching entry in your xorg.conf (you most likely don't need it anyway).

Otherwise, add Option "GrabDevice" "True" to the respective input device. This way you restore the old behaviour that forces a kernel grab. But then again, having the device in your xorg.conf devoids you of any hotplugging (for this device) anyway, so what's the point.

There was one thing I missed when testing: if you can CTRL+C the server or VT switch with Alt+cursor keys, then update to xorg-x11-server-1.5.2-6. A one-line fix that is needed if no xorg.conf is present.

Thursday, October 16, 2008

New keyboard configuration handling

Note that the changes I describe here are currently available in Fedora rawhide, but I expect they will be available soon in a similar form both in X.Org git and in other distributions.

We finally closed the last big chapter in the evdev input driver transition - the handling of device grabs (in the kernel). As described previously, evdev used to put a kernel grab on the device so no-one else could open the same device. This avoided duplicate events but the grab hindered other programs that needed access to the device too: lirc, rfkill, HAL and others.
The need for the grab was removed with quite a simple patch. However, this isn't the whole story, and another patch changes server's default behaviour.

So here's the rundown how it may affect you:

In short, evdev and kbd/mouse are mutually exclusive as they produce duplicate events if both are active. In any configuration where both are specified, kbd/mouse are ignored.


If you don't have an xorg.conf
, your devices were hotplugged through HAL. Nothing changes for you.

If you have an xorg.conf with input devices using mouse or kbd as driver, they are now ignored.
Effectively nothing changes though: due to the evdev grab these mouse/kbd devices were previously "mute", i.e. the would never send events anyway. You might as well remove the respective sections from your xorg.conf. However, having the devices in your xorg.conf provided a fallback: if HAL was not working, kbd/mouse would pick up the slack. This fallback is gone.

If you disable AllowEmptyInput, AutoAddDevices or AutoEnableDevices, then you get the traditional X server behaviour:

  1. Devices referenced in the ServerLayout are available in the server, or if none are specified,

  2. The first pointer or keyboard device in the xorg.conf become available in the server, or if none are specified,

  3. The standard configs become available (/dev/console and /dev/input/mice).



Disabling AllowEmptyInput is discouraged and should not be necessary on virtually all standard desktop systems.

Update: If you are using evdev 2.1 and you only disable AllowEmptyInput, the devices will still be added and you will get duplicate events (key repeat handling will then make each key press emit three characters). AEI defaults to (AutoAddDevices && AutoEnableDevices), so if you want to disable HAL, the best option is to set AutoAddDevices to "off".

Friday, October 3, 2008

Device properties have landed

After a few more patches, the property support has now landed in what I'd like to call the final version. Unless there's any more changes in the next few weeks, this will be added as the main component of the X Input Extension v1.5.

So here's a short description of what they are, the protocol additions and the driver-side API.

What are device properties?



Input device properties (IDP) are basically the same as Window Properties (Atoms) in the core X protocol, or RandR output properties. They are numerical identifiers with arbitrary names that can be assigned arbitrary values. The big thing about IDP is that we can now add options to drivers that can be triggered at modified by clients without having to bump the protocol or even the server. Likewise, we can add options to the server easier too.

One of the server's weakest parts was run-time configuration of input devices. Options in the xorg.conf (or nowadays in the HAL fdi file) were easy enough, but once the server was running it was difficult to do anything. Synaptics even exposes shared memory to enable run-time configuration for lack of a better mechanism. IDP should fix this lack of a decent method and hopefully unify the configuration mechanisms. In fact, synaptics, evdev and joystick already provide property support.

Your server already uses a lot of properties for various stuff. The "xlsatoms" command lists all properties currently defined and their names. The "xprop" command lists all window properties and their current value(s).

Driver API


The code to use device properties from a driver (or the server) is basically always the same:

#include<X11/Xatom.h>
#define MYPROP_NAME "Example property"

char prop_value = 8;
/* create atom (if it doesn't exist already anyway) */
Atom prop = MakeAtom(MYPROP_NAME, strlen(MYPROP_NAME), TRUE);
rc = XIChangeDeviceProperty(device, prop,
XA_INTEGER /* type */,
8 /* format */,
PropModeReplace,
1 /* no of items */,
&prop_value /* data */,
TRUE /* send event */):

/* don't allow clients to delete our prop */
XISetDevicePropertyDeletable(device, prop, FALSE);
XIRegisterPropertyHandler(device, SetPropertyHandler,
GetPropertyHandler,
DeletePropertyHandler);


SetPropertyHandler is called whenever a client changes the property. GetPropertyHandler is called whenever a client retrieves the property value, if you have to get data off the device - do it here. Finally DeletePropertyHandler is called when the property is about to be deleted. Note that if you set the property to be non-deletable, the DeletePropertyHandler can only called if a driver or the server wants to delete the property.

In all cases, do stuff, return Success or an error code otherwise. A NULL handler is allowed. Errors are returned immediately to the client, so make sure your property is properly documented that developers can figure out why a BadValue, BadMatch, etc. error occurs.

Here's some example code for a SetPropertyHandler

int SetPropertyHandler(DeviceIntPtr dev, Atom atom,
XIPropertyValuePtr value, BOOL checkonly)
{
if (atom == myprop)
{
if (val->format != 8 || val->type != XA_INTEGER || ...)
return BadMatch;

if (*((char*)val->data) > 10)
return BadValue;

if (!checkonly) /* do something with the value */
{ ...
}
}
return Success; /* You MUST return Success, even if you didn't handle it */
}


Important here: if checkonly is TRUE, you must not change any state. Check only for validity of the atom and the given value. Each SetPropertyHandler is called twice, once to check the value, then again to actually apply the data. The assumption is that if the checkonly run succeeds, the state change cannot fail. If you return an error in the checkonly run, this error is returned to the client and the second run is never executed.

A few rules: keep the handler simple. You cannot assume that you're called in any particular order. You cannot assume that the second run is ever executed. Keep the handler as local as possible, instead of having one big handler it may be better having two separate ones (e.g. one in the DIX, one in the DDX).

That's basically it. A call to XIChangeDeviceProperty() causes an event to be sent to the clients, so this is the API that should be used if the property needs changing from within the server/driver.

Protocol requests and events


As just mentioned above, there's one event DevicePropertyNotify which lists the state (NewValue or Deleted), the property affected and of course the device id. This event is sent whenever a property is changed by a client and whenever a driver/the server changes a property with XIChangeDeviceProperty(..., TRUE).

The four new requests are:

ListDeviceProperties
ChangeDeviceProperty
DeleteDeviceProperty
GetDeviceProperty

All four requests are virtually identical to the ListProperties, ChangeProperty, DeleteProperty, and GetProperty requests of the core protocol - except they allow the specification of a device ID instead of a window ID. Just look up the man page for the core protocol requests for more information.

The xinput tool has all the basic code to list, view and delete properties.

As a final gimmick, evdev, synaptics and the xserver each have a name-properties.h file, (e.g. evdev-properties.h gives you all the defines for the property names) and the command "
pkg-config --cflags xorg-evdev" gives you the necessary include paths.

Friday, September 12, 2008

RMLVO keyboard configuration

This article explains how different keyboard layouts are selected these days. RMLVO stands for rules, models, layouts, variants and options.

  • Rules are simply sets of combinations that result in - hopefully - useful keyboards. Rules combine MLVO to a full keyboard description.
  • Models describe the physical keyboard, both in geometry and in keycodes (see below).
  • Layouts describe the language dependent literals.
  • Variants are variations of or additions to layouts to cover larger or different key ranges.
  • Options are extras added, independent of the keyboard layout, such as special key behaviours and control keys.

RMLVO is the preferred way of selecting keyboard layouts, as it simplifies it for the user by combining meaningful defaults that are then converted into XKB's internal form of keycodes, types, compat, symbols and geometry (Ktcsg). In this article, I will ignore Ktcsg as much as possible, which will leave some questions unanswered. A follow-up article on XKB's internal form will come in the hopefully near future.
The commandline interface to RMLVO configuration is setxkbmap. Here's an example of how setxkbmap converts RMVLO into Ktcsg:


whot@possum:~$> setxkbmap -print -rules evdev -model evdev -layout us \
-variant intl -option compose:caps
xkb_keymap {
xkb_keycodes { include "evdev+aliases(qwerty)" };
xkb_types { include "complete" };
xkb_compat { include "complete" };
xkb_symbols { include "pc+us(intl)+inet(evdev)+level3(ralt_switch_for_alts_toggle)+group(alts_toggle)+group(alt_shift_toggle)+compose(caps)" };
xkb_geometry { include "pc(pc104)" };
};

This basic description is then passed to xkbcomp which compiles the full keyboard description and loads it into the server. For this article, I will only explain how setxkbmap achieves the above description from the commandline options. I will also ignore geometry, which describes the keyboard but does not have any effect on its actual working. All you need to know for now is that geometry describes each physical key size and their relationship to each other, so you could paint a funky graphic of your keyboard.

Let's go through the other stuff:

First an important distinction: We have scancodes, Keycodes and Keysyms. Scancodes are what the physical keyboard provides. They don't change.
Keycodes are standardised 3-4 letter descriptions of key meanings (e.g. ESC for Escape, but also AD01 for "first key in second key row", usually the q-key). Keycodes have no language-dependency. Keysym is the symbolic representation of a letter in a particular language (e.g. 'A').

Key to understanding the inner works of keyboard support in X is accepting that those three need to be mapped against each other. For example, if you hit the Q key, the keyboard may generate a scancode of 24, mapped to AD01, mapped to "q". A keyboard event contains the keycode, but a client, you *never never* think in keycodes or scancodes. You always think in keysyms, and use the scan/keycode as a method of getting there. Why? Modifiers. If you press shift, you still get the same scan/keycode when hitting a key (just test this with xev). The keysym however is different - usually the capital form of the respective letter. A keycode/scancode itself has no meaning, it only gets it in combination with the current keyboard symbol table and the modifier state. And both modifier state and keyboard symbol table may change at any time.

Back to our RMLVO configuration: setxkbmap uses libxkbfile to parse /usr/share/X11/xkb/ (package xkeyboard-config) and combine the files there into a meaningful selection.

setxkbmap -print [args]

applies the given args to the rule set and prints the output to the screen. If you leave out -print, you'll load the new configuration in the server too.

The default ruleset used to be xfree86 or xorg (a symlink to base). For evdev, the ruleset is now "evdev".
Let's look at /usr/share/X11/xkb/rules/xorg. In the first part of the file, we see a number of model definitions, these all contribute to the geometry and the symbols generated. Ignoring most of the top of the rules/xorg file which consists of variable declarations, the interesting part starts at the following line:
! model  = keycodes
i.e. which model maps to which keycodes.
The one that is to be used most commonly these days is evdev, i.e. the line

! model = keycodes
evdev = evdev

tells us that for the model evdev, the keycodes "evdev" are to be used. OLPC also uses evdev, and the xfree86 keycode range is the fallback. Note that if you're using the *rules* evdev, you don't need to specify the *model* evdev, it's implied (and you thought XKB was confusing, eh?).

The full tables of scancode - keycode mapping are in /usr/share/X11/xkb/keycodes/, where the file name is the model specified. In the evdev model, scancode 24 maps to AD01, the key next to TAB. Evdev is a bit special here, as the kernel takes care of the physical differences of the keyboards, providing a uniform scancode range for all keyboards. In the xfree86 keycode table, the scancodes are the actual physical codes.

So now we have our device-dependent scancodes mapped to device-independent keycodes. What do we do with keycodes? They are mapped to the language-specific literals that we eventually want to see on the screen when we hit a key.

This is specified further down the rules/xorg file:
!model layout variant = symbols

This table lists which symbol tables are to be loaded if a specific model, layout and variant has been selected. Layout is basically the language, variants can be e.g. intl or nodeadkeys. There's another table like that, but just mapping model and layout to symbols, without variants.
US keyboards are covered by a fallback in the ML->S mapping table.

!model layout = symbols
* * = pc+%l%(v)

Following this fallback line, the commands

setxkbmap -print -layout us
setxkbmap -print -layout us -variant intl

result in pc+us and pc+us(intl), respectively.

The actual symbol mappings are in /usr/share/X11/xkb/symbols/. Let's have a look at the symbol table for AD01 (the key next to TAB) in the US layout (/usr/share/X11/xkb/symbols/us):

key <AD01> { [ q, Q ] }

It says that the literals q or Q are provided by this keycode. Which one, depends on the Shift level. We can have up to 8 levels per key, each level is triggered by one or more modifiers. Most commonly, Level2 is triggered by Shift and/or CapsLock but that's Ktscg territory so run away as fast as you can.
I didn't discuss options here either because they are just more of the same and only become really important when we discuss Ktcsg.


That's basically it. We know how by going through the various files we can assemble a basic description of how the keyboard should map the physical scancode into a literal keysym. If we can accept that xkbcomp parses the files appropriately to get *full* scancode-keycode mapping, and a *full* keycode-keysym mapping, we now know how a little command-line invocation smartly combines to the correct layout.

As said before, setxkbmap -print shows the current settings, alternatively setxkbmap with parameters changes them

whot@possum:~$> setxkbmap -print -layout us -option "" -variant intl -model evdev
xkb_keymap {
xkb_keycodes { include "evdev+aliases(qwerty)" };
xkb_types { include "complete" };
xkb_compat { include "complete" };
xkb_symbols { include "pc+us(intl)+inet(evdev)" };
xkb_geometry { include "pc(pc104)" };
};


You don't have to specify everything all the time, setxkbmap knows your current setting through the _XKB_RULES_NAMES property (xprop -root | grep XKB). So a command of
setxkbmap -layout de
is enough to switch to german layout.

Let's sum up:

XKB keyboard configuration is provided by xkeyboard-config which provides a set of Rules. Rules state that how certain Models, Layouts and Variants are combined into XKB's internal Ktscg format. Once we have this combination of Ktcsg, we can submit it to xkbcomp, which compiles the full keyboard description and loads it into the server. This however, will be another article.


Thanks to Daniel Stone and Timo Aaltonen for their feedback.