Thursday, May 10, 2012

Testing X servers from git

Every so-often I get asked the question of how to test the X server (or drivers) from git. The setup I have is quite simple: I have a full tree in /opt/xorg, next to the system-installed binaries in /usr. A symlink and some environment variables allow me to switch between git versions of the server and clients and the system-installed ones.

Installing the tree

Getting that setup is quite easy these days:
mkdir -p xorg/util
git clone git://anongit.freedesktop.org/git/xorg/util/modular xorg/util/modular
cd xorg
./util/modular/build.sh --clone --autoresume built.modules /opt/xorg
That takes a while but if any component fails to build (usually due to missing dependencies) just re-run the last command. The built.modules file contains all successfully built modules and the script will simply continue from the last component. Despite the name, build.sh will also install each component into the specified prefix.

You get everything here, including a shiny new copy of xeyes. Yes, what you always wanted, I know

Note that build.sh is just a shell script, so you can make changes to it. Disable the parts you don't want (fonts, for example) by commenting them out. Or alternatively, generate a list of all modules, remove the ones you don't want or need and build with that set only:

./util/modular/build.sh -L > module_list
vim module_list # you can skip fonts, apps (except xkbcomp) and exotic drivers
./util/modular/build.sh --clone --autoresume built.modules --modfile module_list /opt/xorg

Either way, you end up with /opt/xorg/bin/Xorg, the X server binary. I just move my system-installed and then symlink the new one.

sudo mv /usr/bin/Xorg /usr/bin/Xorg_old
sudo ln -s /opt/xorg/bin/Xorg /usr/bin/Xorg
Next time when gdm starts the server, it'll start the one from git. You can now update modules from git one-by-one as you need to and just run make install in all of them. Alternatively, running the build.sh script again without the --clone parameter will simply git pull in each module.

Setting up the environment

What I then define is a few environment variables. In my .zshrc I have
alias mpx=". $HOME/.exportrc.xorg"
and that file contains
export PKG_CONFIG_PATH=/opt/xorg/lib/pkgconfig:/opt/xorg/share/pkgconfig
export LD_LIBRARY_PATH=/opt/xorg/lib/
export PATH=/opt/xorg/bin:$PATH
export ACLOCAL="aclocal -I /opt/xorg/share/aclocal"
export MANPATH=/opt/xorg/share/man/
So running "mpx" will start git versions of the clients, link clients against git versions of the libraries, or build against git versions of the protocol.

Why this setup?

The biggest advantage of this setup is simple: the system install doesn't get touched at all and if the git X server breaks changing the symlink back to /usr/bin/Xorg_old gives me a working X again. And it's equally quick to test Fedora rpms, just flick the symlink back and restart the server. I have similar trees for gnome, wayland, and a few other large projects.

It also makes it simple to test if a specific bug is a distribution bug or an upstream bug. Install the matching X server branch instead of master and with a bit of symlink flicking you can check if the bug reproduces in both. For example, only a few weeks ago I noticed that xinput got BadAtom errors when run from /usr/bin but not when run from /opt/xorg/bin. Turns out it was a thing fixed in the upstream libXi but not backported to Fedora yet.

The drawback of this setup is that whenever the xorg-x11-server-Xorg module is updated, I need to move and symlink again. That could be automated with a script but so far I've just been too lazy to do it.

[Update 11.05.12: typo and minor fixes, explain build.sh -L]

4 comments:

spbnick said...

Thanks, Peter!

There is a small typo:

git clone://anongit.freedesktop.org/git/xorg/util/modular xorg/util/modular

Should probably be:

git clone git://anongit.freedesktop.org/git/xorg/util/modular xorg/util/modular

spbnick said...

For me build failed with the following:

test -z "/etc/fonts/conf.avail" || /bin/mkdir -p "/etc/fonts/conf.avail"
/usr/bin/install -c -m 644 42-luxi-mono.conf '/etc/fonts/conf.avail'
/usr/bin/install: cannot create regular file `/etc/fonts/conf.avail/42-luxi-mono.conf': Permission denied
make[1]: *** [install-dist_availconfigDATA] Error 1
make[1]: Leaving directory `/home/nick/projects/freedesktop.org/xorg/font/bh-ttf'
make: *** [install-am] Error 2
build.sh: " env LD_LIBRARY_PATH=/opt/xorg/lib make install" failed on font/bh-ttf
build.sh: error processing module/component: "font/bh-ttf"

spbnick said...

I mean, I wouldn't expect a build to actually install files, moreover into system directories. Is there a way to install configuration files into a different directory?

I would try to figure it out myself, but since you are trying to provide complete instructions, maybe you could describe this also.

Thanks:)!

Peter Hutterer said...

spb_nick: right, the fonts are an exception here. They use a directory that comes from fontconfig. you can skip the fonts though, I'll update the post accordingly