Rather by accident, Dave Airlie and I found a minor security issue in the X server last week (read the story here) This issue has been assigned CVE-2013-1940 and is now publicly available. The corresponding bug reports are here: https://bugs.freedesktop.org/63353 and https://bugzilla.redhat.com/950438.
X servers receive notifications from HAL/udev about new input devices, even when you vt-switched to the tty or another server. Input devices added while the server is not the owner of the vt will be added but not enabled, so events from such devices are ignored. On vt-switch back, the device is enabled and the fd is added to the select set used by the server. Future events will trigger a SIGIO and will be processed as expected
evdev holds the fd open between PreInit and enabling the device. If the device is hot-plugged while the server is vt-switched away events accumulate on the fd. evdev calls xf86FlushInput() to discard these events but a bug in that function made it essentially a noop for evdev devices. Thus, once the server is the VT owner again, events from that device are still on the fd and are processed whenever the next event comes along on that device.
Reproducer is fairly simple: open a text editor, vt-switch, hotplug a keyboard, type something on that keyboard, vt-switch back and the events will be replayed on the existing server.
This issue is now fixed upstream and I have released xserver 1.13.4 and xserver 1.14.1. It is of relatively low impact but the fix is easy so I recommend to patch your X servers.