My workflow

After posting about the X.Org supermodule I was asked for more details about how to get everything up and running. So here's my usual workflow. I'm open for improvements in the (moderated) comments.

Prerequisites

My workflow currently includes two patches to git, the second of which is still under review.
Install git from git and apply these patches. They add rebasing support to git submodule update and make the process a lot easier.

Set up a bunch of environment variables that you'll need each time you build:

$> export PKG_CONFIG_PATH=/opt/xorg/lib/pkgconfig
$> export PATH=/opt/xorg/bin:$PATH
$> export LD_LIBRARY_PATH=/opt/xorg/lib
$> export ACLOCAL="aclocal -I /opt/xorg/share/aclocal"

and some more that are just handy:

$> export CFLAGS="-Wall -O0 -ggdb"
$> export CC="ccache gcc"
$> export MAKEFLAGS="-j3"

Building X.Org from git in 5 steps

$> git clone git://people.freedesktop.org/~whot/xorg.git
$> cd xorg
$> git submodule init
$> git submodule update
$> ./util/modular/build.sh -f built.modules /opt/xorg

The supermodule is set up for automatic rebasing and you'll end up with a tree running each module on master. The last command builds everything in the right order and - with the "-f" flag - echos the modules being built into the built.modules file. If it fails (usually due to missing packages) you can resume from the last to-be-built component (the last one in built.modules).

$> ./util/modular/build.sh -f built.modules -r `tail -n 1 built.modules` /opt/xorg

Once that is done, you're left with an X tree in /opt/xorg, most important of which is the binary in /opt/xorg/bin/Xorg.

Working with the tree

I tend to have three or more branches in most repos. The branches that matter are "master", "queue" and "devel". Then I have additional branches for features that result in a patch series (e.g. "xi2").

"master" is always as close to upstream master as I can get. Anything that lands on master will likely be rebased and pushed upstream. Day-to-day bugfixing also happens on master.

"queue" is for patches I sent to the xorg-devel list. The workflow here is usally development on some other branch, then git-format-patch + email, then cherry-picking from the other branch to queue. Patches in "queue" get cherry-picked to master and pushed, and once master is pushed, "queue" is rebased onto master.

"devel" usually happens when I realize that the patch series on master is more than it should be. This is when I branch master into devel, reset master to the previous state and continue on devel. devel is heavily rebased and sometimes doesn't lead anywhere. If it does, intermediate patches are cherry-picked onto master and pushed when they're ready. devel is deleted as soon as I finish with it.

Feature branches (e.g. "xi2") tend to be the same as devel but with a specific feature in mind. Anything that isn't related to it (bugs that I find in code around that feature) is cherry-picked to master and queue, and the feature branch then rebased.

So a single patch may wander from xi2 to devel (when xi2 is branched for some reason) to queue to master before being pushed.

I've been using this workflow for months now, and one of the main reasons why it works fine for me is tig. Tig shows other branches heads and tags in the history list, so by rebasing often I always have a visual marker where the new patches start. Tig also makes cherry-picking easy, so I use it more often than git pull.

The other thing that is incredibly helpful is the zsh git prompt I got from here and modified a bit. My current output reads:

:: whot@dingo:~/xorg/xserver (xi2*+)> 

Where xi2 is the branch name, * shows I have changes not added to the index, + shows I have changes added to the index that will be committed. I cannot recommend this prompt for the kernel though, it takes to long. For the repos I work with it's fine.

A word about backups

Since most of the work is local, it's important to have a backup, especially for devel branches that live longer than a day or two. I added a "backup" remote and force-push the branch I've been working on at the end of each day to this remote. In the worst case, I can just clone from there and resume where I left off.

$> git remote add backup user@host:~/repository.git # only do this once
$> git push -f backup branchname

The big fat X.Org supermodule

I've been using a git supermodule for nearly 2 months now and it works great. So here it is, do with it what you will.

git://people.freedesktop.org/~whot/xorg.git

It includes all the modules I care about to get a server running.

The advantages of git submodule are simply that I have a known working tree I can easily share between my test machines. Whenever I update a component, I can test it and easily revert back to the previous working version if needed.

Advantages for you: if you pull from that tree, you're running a version that at least compiles and runs on my hardware. That doesn't mean it's bug-free of course, but it's a start if you want to get into X server testing.

Disadvantages: I update input stuff often, other stuff when needed. So if you care about the latest and greatest graphics patches, you will need to maintain your own tree. Such is life.

Updating is easy: git pull and git submodule update.

The git submodule interface is missing one important feature to make this workflow better (automatic rebasing), but that'll hopefully be fixed in a future git version.

git-format-patch for a single commit

One thing that always annoyed me was the weirdness of getting a patch from a single commit somewhen back in the history. Turns out I was just ignorant and reading the man page actually helps, git-format-patch accepts a -<n> option, where <n> is the number of patches you need since the commit (inclusive).

So, because I'm a big fan of tig, add this to your $HOME/.tigrc:

bind generic E !git format-patch -1 %(commit)

Start tig, mark the patch you want, hit "E" to get a nice patch file. Doesn't get much simpler than that.

[update Jun 05 2012]
As SEJeff and Michael point out in the comments

git show $sha

shows a single commit. The output of git show is different to git format-patch though (it cannot be applied via git-am).

Zapping the server

Some amount of bad blood was spilled last week (read LWN's writeup for a better summary than I'll provide here).

In short, zapping through Control-Alt-Backspace was disabled and some people are not happy. It is a useful combination and I (need to) use it regularly. Should it be enabled by default? No.

Let's think about this: Assume that the server never supported zapping in the past. Now we add a feature that immediately and without asking terminates your session, shuts down all applications, logs you out, brings down your wireless network in the process, shuts down your VPN and generally makes the computer giggle at you. Of course, we documented this shortcut in the Xorg man page.

I guess we would see the same flame-wars. Potentially even by the same people.

The correct thing to do for a UI feature like this - like with any UI feature - is to consider who will benefit from such a change and make it accessible to those. Experienced users will benefit from zapping. So make it accessible to them. Those who want to use the computer but not have to know about it's internals should not be able to accidentally trigger it. Not the other way round.

Anyway. The correct solution was rather easy, and thankfully, Julien Cristau has written the patch for xkeyboard-config and there's a patch for the X server too. Which, amongst from other things, shows that time spent reading and replying to flame-wars is spent less usefully as time spent writing patches.

The following command enables server zapping.

setxkbmap -option "terminate:ctrl_alt_bksp"

I'll leave it as an exercise to the reader to figure out how to click the matching checkbox in the keyboard configuration tools.

So for the future: If you are unhappy with a particular change, take the issue to the appropriate mailing list and/or persons, in polite tone and based on real arguments (made-up facts only count when you're on TV) and it may just get fixed. It's pretty much a no-brainer, but maybe even such a minimum requirement is too high. In the end, Julien's patch was the reason why the whole issue got sorted out, not the flame-wars.

Rawhide users:
There's a dependency chain in keyboard configuration that made me weep inside a little. Update to fedora-setup-keyboard-0.3-4, xkeyboard-config-1.5-5 and xorg-x11-server-1.6.0-18, then restart HAL and restart X. Zapping will work on a plain X server and in gdm, but only if enabled in the gnome session.

[update]
Option DontZap is restored with this change to it's previous functionality - it disables zapping altogether (if enabled). Even if DontZap is disabled (the default), zapping only works if it is present in the xkb map.

Synaptics 1.1 and what your touchpad can do now

I just stumbled over this and I realized that I never wrote about the synaptics driver. So, here's a list of things that have changed recently with version 1.0 and 1.1.

• Perhaps the most important changes have to to with auto-scaling. Synaptics obtains the touchpad dimensions from the kernel and adjusts speed, acceleration, the edges and more depending on these dimensions. As a result we support a lot more models and touchpads should just work and feel approximately the same on the different touchpad models. More or less, anyway.



• Multi-touch support and two-finger scrolling: the driver detects whether the touchpad can detect multiple fingers. Not all touchpads can do this*, for example my T61 has one that only does single-finger detection. If your touchpad has multi-finger support, the driver enables two-finger scrolling instead of edge scrolling.



• Tapping is disabled by default. I've gotten some flak about this but I maintain that it is the better choice. Tapping is enabled for those touchpads that don't have physical buttons though.



• synclient and syndaemon updates to device properties. Both programs do not require the SHM area anymore and thus also just work without extra configuration. So any option available in the synaptics man page can be modified at runtime.



• Touchpad integration into gnome**. With the help of Matthias Clasen, we took the Ubuntu patches, fixed/updated them, pushed them into rawhide and sent both of them upstream. So gnome-mouse-properties now has a touchpad tab for the basic configuration settings. In rawhide anyway, and hopefully also upstream soon.
  

There's a lot of minor changes, but these are the big things.

* Unfortunately, the driver doesn't export yet whether the device has multifinger support or not. The only way to be sure is to check the Xorg.log and look for the line "(II) SynPS/2 Synaptics TouchPad: buttons: left right middle". If this line also says double and/or triple, your touchpad can detect two finger and/or three fingers.


** I started by fixing gsynaptics in rawhide, but that's now discontinued.

Goodbye VCP/VCK?

:: whot@dingo:~/xorg/app/xinput/src> ./xinput --list --short
Default device                           id=2 [master device]
   ↳ Power Button (FF)                        id=3 [slave  device  (2)]
   ↳ Video Bus                                id=4 [slave  device  (2)]
   ↳ Sleep Button (CM)                        id=5 [slave  device  (2)]
   ↳ Video Bus                                id=6 [slave  device  (2)]
   ↳ Microsoft Microsoft? Digital Media Keyboard id=7 [slave  device  (2)]
   ↳ Microsoft Microsoft? Digital Media Keyboard id=8 [slave  device  (2)]
   ↳ ThinkPad Extra Buttons                   id=9 [slave  device  (2)]
   ↳ Macintosh mouse button emulation         id=11 [slave  device  (2)]
   ↳ Microsoft Microsoft 5-Button Mouse with IntelliEye(TM) id=13 [slave  device  (2)]
   ↳ SynPS/2 Synaptics TouchPad               id=14 [slave  device  (2)]
foobar                                   id=15 [master device]
   ↳ AT Translated Set 2 keyboard             id=10 [slave  device  (15)]
   ↳ TPPS/2 IBM TrackPoint                    id=12 [slave  device  (15)]

So what does this mean?

xinput --list simply lists all the devices connected to the server. In current servers, these were the "Virtual Core Pointer", the "Virtual Core Keyboard" and a bunch of physical devices.

I simply got rid of the distinction between pointers and keyboards, because too many devices are both anyway. The distinction doesn't really make sense. Even worse, with MPX's event routing through the respective master devices it became a bit of a nightmare to do key events correctly. I'll spare you the details.

So anyway. The idea is to merge master pointers and master keyboards into a single master device. This device is both, thus all events from any attached slave device goes through this single master. Nice and clean, and once we need more capabilities (*cough*multitouch*cough*) we can easily tack it onto the MD.

Now, we do have the notion of pointer and keyboards in the core protocol and to break the core protocol requires a presidential pardon that I have yet to obtain. One of the tougher things here are core grabs. With a single master device, all we need to support is an async pointer grab that is overridden by a passive sync keyboard grab (affecting the same device, but other events) that may or may not get modified by an active pointer grab. So while freezing the device for keyboard events you may need to replay pointer events on the device that is frozen, if this device isn't still frozen for pointer events by some other grab. Surprisingly, there's nothing in the Genevan Convetion about this.

Anyhow. AFAICT, this works now, with core grabs passing the X Test Suite and I'm happily running my standard Fedora desktop without noticing anything different.

Some of the remaining issues:
- god, this code is ugly.
- lazy transfer of classes to the MD.
- handling of the focus if the MD loses the key class.
- there is not enough beer in the fridge.
- testing this through the XI2 API to make sure it isn't completely insane.
- spec out XI2 grabs.
- misc. details that will make me regret this attempt.

Oh, as a side-effect this also enables merged modifiers, i.e. pressing shift on one keyboard types capitals on another one (if both are attached to the same master).

[edit: this whole experiment is not upstream, not even on my people.freedesktop until I cleaned it up]

XI2 implementation, take 1

This weekend, I announced the first implementation of XI2 on the xorg-devel list.

You can get the repositories from

git://people.freedesktop.org/~whot/xserver.git
git://people.freedesktop.org/~whot/inputproto.git
git://people.freedesktop.org/~whot/libXi.git
git://people.freedesktop.org/~whot/xinput.git

Check out the xi2 branch on all four repositories.

XI2 is important for two reasons. One, it's the client-side API that enables applications to make use of MPX. For obvious reasons, I have some interest in getting this done.
The other part of XI2 (and why it is called XI2) is that it's a new version of the X Input Extension. One of the goals here is a cleaned up API that is less painful to use than the first, current, version.

Aside from some API cleanup, the new additions are subpixel precision in events, the ability to support devices that change capabilities at runtime and relative device events. And of course all the stuff required to access MPX' functionality.

So, if you're adventurous*, grab the repositories, compile them up and test them. You can write little test apps to play around with the new stuff, and if you find any problems or inconsistencies, please report them back. I'm trying really hard to get this right.
Note that because this is the first version, API changes are to be expected and there will be the odd bug in there.

* One of the main goals of MPX (and XI2) is that it must be invisible until used. So you can grab the repositories and run your everyday desktop off it and there should be no difference.