Fedora 15 and offlineimap users beware

This is just a word of warning: if you have upgraded to F15 and you use offlineimap, observe very carefully what it does and ensure you have a backup of your email.

This post has updates. See at the end of the post

I noticed yesterday that I couldn't search for some email that I knew existed. On closer inspection I found a huge chunk of my archive inbox to be missing. Several months worth of emails, they stopped in March and then continued with the emails from yesterday that I had copied in from my Inbox.

The same happened to my fedora-devel and xorg-devel folders, possibly others. Re-running offlineimap to sync these pulled down hundreds of emails, so I let it run overnight. This morning I discovered that sometimes offlineimap simply states something like "Deleted 1018 messages" and they're gone again. This is cumulative, it seems to delete only newer ones (I was at September 2010, but after one more run I'm now back to July 2010 being the newest messages in in my xorg-devel folder).

The good news is that the emails are still on the server. Also, so far only mailboxes with huge numbers of emails (>10000) seem to be affected, smaller mailboxes appear to work fine.

I have not filed a bug yet because I don't know what the actual issue is. Could be offlineimap, could be something else. But in the meantime, beware.
Update May 30: Filed as Bug 708898

Update May 30
I downgraded to offlineimap 6.2.0 which is in F14 (offlineimap-6.2.0-2.fc14.noarch) and managed to eventually resync my mailboxes. Then I went back to F15's offlineimap-6.3.3-1.fc15.noarch. Of course it didn't trigger on my test mailbox, but it did then do the same thing again on my xorg-devel inbox.

Syncing lists-xorg: IMAP -> Maildir
 Deleting 8181 messages (33540, 335
[...] 
in Maildir[lists-xorg]

F15's offlineimap-6.2.0-3.fc15.noarch does not show this behaviour and seems to resync the locally deleted emails. Of course that'll take a while, but I'll try to get logs this time (my first couple of tries to get logs from offlineimap crashed it).

GNOME 3.0 middle mouse button emulation

This is essentially a GNOME 3.0-specific update to New evdev middle mouse button emulation defaults. In recent evdev versions, middle mouse button emulation is disabled by default and can be enabled with Option "Emulate3Buttons" "on" in the xorg.conf file or an xorg.conf.d snippet.

GNOME 3.0 has a gsettings key to also trigger this behaviour on a per-user basis (see bug 633863). The default value is off (false). Settings applied from the desktop environment overwrite xorg.conf(.d) settings, so to get middle button emulation in GNOME simply toggle the gsettings key and don't worry about xorg.conf(.d). The command to toggle the key is:

$> gsettings set org.gnome.settings-daemon.peripherals.mouse \
     middle-button-enabled true

Since this gsettings are persistent, setting the key once is sufficient. It will be applied after login, for each device and as new devices are hot-plugged.

Custom input device configuration in GNOME

Thanks largely to Bastien, Gnome Bug 635486 just got committed, in time for GNOME3.

It adds is the ability for custom configuration of input devices from within GNOME, especially for settings that are not handled by the GNOME UI tools. It's principle is quite simple. A new gsettings key hotplug-command in the org.gnome.settings-daemon.peripherals.input-devices schema points to an executable. This executable is called at gnome-settings-daemon startup, whenever a device has been added and whenever a device has been removed. Something like this:

        $HOME/executable -t added -i 12 My device name

So the device with the ID 12 and the name "My device name" just got added. Other types are "removed" and "present" for those device that are already present when g-s-d starts up. It's quite simple to configure any device to your liking now. Have a look at the input-device-example.sh script shipped with gnome-settings-daemon. I'll give you two examples of what you could add to the script and the rest should be easy enough to figure out:

# Map stylus button 1 to button 8
if [ "$device" = "Wacom Intuos4 6x9 stylus" ]; then
        xsetwacom set "$device" Button 1 8
fi

# map tapping to LMR instead of default LRM
if [ "$device" = "SynPS/2 Synaptics TouchPad" ]; then
        xinput set-prop $id "Synaptics Tap Action" 0 0 0 0 1 2 3
fi

How to DOS a developer

So you think your favourite FOSS project is moving too fast? Easy, just DOS the developers. Here's a few tips on how to achieve this:

On IRC

• If you have a problem, just state that you have a problem. Let the developer ask for what exactly it is. Then answer every question with the minimum amount of information possible and let the developer keep asking.


• Wait 5 minutes before answering but keep asking why the developer doesn't answer immediately. That way you can maximise the context switching costs of the developer, ensuring he or she can't get anything else done while waiting for you.


• If the developer asks for version numbers, be vague. Things like "whatever was in $DISTRO three days ago" is best because unless the developer is also the distro maintainer, you've answered the question without providing any information.


• Don't stay on IRC. Just go offline whenever your computer goes to sleep and re-ask the question every time you reconnect. This way you ensure that those not monitoring the channel will try to answer the questions whenever you're currently offline. Never send email to the list, because others could answer it in their own time.

Via email

• Send bug reports to the developers directly. That way you force them to reply with at least "please file a bug in $BUGZILLA". That takes 2 min of their time, not counting context switches. Make sure you reply with more than one "thank you" email so they need to deal with more email.


• Randomly remove the CC from mailing lists during discussions. That way you make sure the developer has to answer twice if they didn't notice the CC was missing.


• When taking discussions back onto the list, misquote (or purposely misinterpret) the results of the private discussion. Force the developer to justify themselves in public.


• Be verbose. As verbose you can be. If you can say something in one sentence or three paragraphs (with three overlapping or identical examples), choose the latter. Searching for signal in lots of noise is a favourite pasttime of many developers.


• If you change the subject of a discussion, leave the email subject as-is. This way you ensure that the email cannot easily be found later. Wait for a few weeks, then refer to this discussion, but not in the same thread.


• Pastebin everything instead of including it in emails. Links to websites, backtraces, log files, patches, etc. Make sure the pastebin has a short expiry date.


• If you must include something as attachment, make sure it's at least in a zipfile so no-one can look at it without a few mouse-clicks first.

Via code

• When commenting on patches, reply with vague statements on how the patch doesn't work. This way, the patch is made to look bad and the developer now has to spend time making sure your vague statements weren't just unfounded.


• When testing code, make sure you don't explain how a bug happend. Just say "after a while it crashed", that's enough information for anyone to figure out what's going wrong.


• Read The Daily WTF and gain inspiration for your next open source project. If you ever get other people helping you, you maximise their pain.


• Don't document anything. Just use arbitrary hardcoded values, with no explanation.


• Never, ever, explain why you're doing something. Code is not meant to be self-explanatory.


• Don't follow anything I said in on commit messages


• Do everything as complicated as possible but don't explain why. Obfuscating that you're flipping the sign of a variable three to four times is a good example, anyone reading the code is sure to spend hours on it


• Use numbers as both bools and numbers. C is great for that, you can return 2 and then use it as Boolean and as actual numeric value lateron. Hide any such usage as well as you can.


• When sending patches, mix code changes with arbitrary whitespace changes. Fun minutes can be spend looking at two identical lines trying to find the difference that's missing.


• When sending updated versions of a patch, don't tell anyone what has changed. That way, you force your reviewers to review the whole patch, every time you change a single character.


• When sending patches, make sure that it only fixes the problem on your specific setup. That way you also can tell the developers off for not merging an important bugfix that clearly works when you tested it. And you force them to reply to you.

Those are just the immediate things that come to my mind. If the project is already low on manpower, any of these will have a high impact. If it has enough developers, you may need to use several of them at once to tie up the most resources. Of course all this requires developers that still think that communicating with users is worthwile and that other people generally tell the truth. If they have given up on humanity, they may just ignore you and keep improving the project. In that case, you can at least badmouth everyone on tech "news" sites forums and comments.

Oh, in case you're wondering why I don't get anything done lately, see above. I realise that probably none of it is intentional, though the net effect is the same.

evtest is now on freedesktop.org

evtest, that little program that displays kernel input events and information about input devices now has a home on freedesktop.org. I hope this helps to reduce the number of different upstreams and distro-specific patches that float around.

Code: http://cgit.freedesktop.org/evtest/

Bugzilla: https://bugzilla.freedesktop.org/enter_bug.cgi?product=evtest

Relative and absolute devices in X

One of the recurring questions about to input event handling is how relative and absolute devices are handled in X. Surprisingly, the answer is "that's complicated" and it took me a while to wrap my head around this.

Back in the days of yonder, devices had no mode. X (the coordinate, not the window system) was x, y was usually y unless bugs intervened and we all had mice, computers the size of fridges and fridges the size of, well, fridges. Such was the lore, blah blah (those of us who are old enough to actually remember those days, which excludes myself for example, may remember them fondly).

In 1994, the new kid on the block (not one of the singing ones, they disbanded that same year¹, presumably for other reasons. Though that'd be one thing I'd love to be credited with...), the X Input Extension protocol added support for device mode and the notion of relative and absolute devices was born (within the Xiverse, anyway). Fast-forward half a lifetime of something that lives for 30 years and XI2 added to that per-valuator modes. i.e. a device may have an absolute X axis and a relative Y axis.

But: the device mode in X describes the behaviour of a device only. On an absolute device, the pointer is expected to jump to the position the device tells it. On a relative device, the pointer is expected to move by the coordinates provided, based on the current pointer position (and accounting for pointer acceleration).

However, there are no requirements for how to send coordinates and absolute devices may send relative data and vice versa. So inside the server, the mode of a device (or a valuator) has no meaning. It's merely a flag sent to the client.

Which makes mode processing a bit tricky because the only time we know if an event was relative or absolute is when we receive the actual event (events include a absolute/relative flag). An absolute device may calculate deltas based on the previous position and then submit the new position as absolute coordinate - and thus feel like a relative device to the user. For whatever reason you'd want to do this, I don't think any driver does it atm (though I have vague recollections of having done this in the past). But it just adds to the insanity that is input in X.

Either way, we cannot rely on the device mode for anything in the server, and the actual device mode is just a hint to the client.

¹I had to wikipedia this. I wonder what that does to my reputation when the admins see the logs.

Reviewing patches backwards

Though this may be obvious to some, I only learned a short while ago that it's much easier to review patches bottom to top. Why? Because much code is written bottom to top.

In C, one uses static functions for code that doesn't need to be exported. Because we're lazy and try to group code together, many statics don't have a separate declaration. So you end up writing with code like this:

static void cure_world_peace(void) {
    ...
}

static void save_cancer(void) {
    ...
}

static void solve_all_problems(void) {
   cure_world_peace();
   save_cancer();
}

void foo(void) {
   ...
   solve_all_problems();
   ...
}

When you're reviewing top to bottom, you see the new feature first but lack the context. This gets more and more confusing the more functions you see because you have to keep in mind what they're doing. Reviewing bottom to top means you'll see the context first and you can tick it off mentally. Only then will get to the static functions and see the details.

And coincidentally, this is also how I write code - smack the hooks where they belong and then implement the details.