Apparently this installer wants a title

So, Saturday, SIPB ran a hackathon, named Velocihacker following what is apparently becoming a trend. Last time, I ended up spending much of my time floundering over a Qt bug that apparently already got fixed the week before. This one was somewhat more productive. (Not to say the floundering wasn’t interesting or useful in itself. Source-diving a large project like Qt is always an adventure.)

As of this hackathon, I’m now a BarnOwl maintainer. Yay! Nelson helped me through the release process for the barnowl locker. (The versions are, quite excitingly, parallel-installed.)

After that, I went to work on a problem we’ve been having. So, the locker contains builds for various different platforms, but they all need to show up under bin. So, there is some magic in AFS to translate @sys components in paths into your sysname, a string which identifies your platform and architecture. For instance, the SIPB-run Linerva dial-up has a sysname of i386_deb50. Lockers typically symlink bin to arch/@sys and everything magically just works.

Well, not quite. BarnOwl actually uses a wrapper script which launches the executable at BARNOWL_REAL after modifying the environment as appropriate. But other than that, all is good.

Except Athena only puts adjacent Ubuntu and Debian releases in the same sysname. In particular, Debian Lenny and Ubuntu Karmic share sysname suffix deb50. But, they include incompatible versions of libzephyr. Karmic includes zephyr 3 (with soname libzephyr.so.4) while Lenny includes zephyr 2 (soname libzephyr.so.4). So, we spent much of the hackathon messing with the build scripts and wrapper scripts to work around this issue. We eventually decided to incorporate the zephyr soname into the version for the lenny/karmic sysnames and modified the wrapper script.

A bit of work and fiddling later, and BarnOwl runs on both Karmic and Lenny out of the same sysname. It’s a fairly ugly hack, but it works. In the future, I hope to reorganize the BarnOwl locker to be a little cleaner; right now the folder hierarchy isn’t quite what I’d like. It’d also be nice to make this dispatch less of a hack, but I don’t know how to solve this problem in general. My preferred solution to these sorts of problems is to declare that we don’t care and parallel-install conflicting libraries as needed. However, zephyr requires a zhm be running on the machine, so we have dependency on the actual system; it’s no longer clear how to transform a global conflict into a local one. I guess we could try to run two zhms, but that seems ridiculous.

I also fixed an annoying redraw/resize issue, but that will likely wait until the master is ready to take changes for the 1.7 release. 1.6 isn’t out yet. Hopefully I will also have time in the future to finish the massive graphics layer project I’ve been planning for many months now.

A short continuation of the previous post on KDEDIRS.

So, after the KDEDIRS game, I was able to get most of the old programs running. But I had some trouble with the newer ones. KDE is slowly moving all their PIM applications to this framework called Akonadi. As of 4.4, the contacts were maintained by Akonadi.

Even though I don’t use much of KDE PIM, the pieces notoriously all integrate which each other, and KDE would constantly try to launch the Akonadi server, even at login (I have since disabled the offending KRunner plugin). Each time it would fail, complaining that the D-Bus services were not configured, among other problems.

D-Bus is the IPC mechanism behind the modern free desktop. It was inspired by KDE3′s old DCOP system and GNOME’s CORBA implementation, and has since replaced both its predecessors. Now, D-Bus has this concept of services. These services allow D-Bus to automatically launch a service when one attempts to connect to a name.

While the services for the system bus are a hopeless cause for me, I should be able to influence my session bus as I wish. dbus-daemon‘s man page does claim to follow the XDG Base Directory Specification, so everything should Just Work. I set XDG_DATA_DIRS and D-Bus picks up my session files.

But it doesn’t. Inspecting the process’s environment (via /proc/PID/environ) reveals that my changes don’t take effect.

The problem: D-Bus is not launched by my .xsession, where all the magic happens. D-Bus is launched by login manager! (Well, indirectly.) In /etc/X11/Xsession.d/ are files that get sourced by your login session. In particular, /etc/X11/Xsession.d/75dbus_dbus-launch puts dbus-launch into the startup sequence before I ever get to do anything. It is conditional on a STARTDBUS variable, but I am unaware of any way to modify that for my session alone. Removing this script or otherwise messing with it is not fair game; the goal is that I should be able to log back into system KDE safely having not modified any files it cares about.

So, lacking a better way to do this, my KDE 4.4 startup script contains this terrible little hack:

# Kill D-Bus
killall -u "$USER" dbus-daemon

# Launch KDE
exec dbus-launch --exit-with-session startkde

At some point when I have time, I’ll investigate how NixOS manages this. I imagine they patch the display manager or session scripts at some level.

Continuing from the previous installment on lockers.

So, you would think that, with the previous path setup alone, things would Just Work. Of course, there’s a minor issue needing a newer Qt than Ubuntu provides in Karmic, but that’s easy to fix with another locker. In fact, one could even download the LGPL SDK installer for Linux and use the folder as-is as a locker.

And, indeed, this mostly worked. However, I did not simply want to run my own build of a desktop. I wanted my original software to still work, but use the newer libraries. There, I ran into a problem. If I tried to launch yakuake, a terminal that I like to use for things like zephyr, I got this strange error:

I'm sorry Dave. I'm afraid I can't do that.

Well, that’s a bother. To understand what happened, let’s look at how KDE applications locate files.

At the heart of the core kdelibs library is KStandardDirs. (KDE’s API pages are down right now, so I shall direct you to this mirror a developer set up.) When a KDE application wishes to locate a file, it does not hard-code a path or use a compiled-in PREFIX value. Instead, it asks KDECore to find it for them. You provide a resource type (such as data, lib, or config) and a file path. KStandardDirs then goes and locates it for you.

Reading down the docs a bit, we see that the class works by checking a set of registered suffixes for the resource type against a set of roots. (It also does some other magic like appending the application name for some resources.) These roots include the compiled prefix and a colon-separated variable KDEDIRS. This prefix is the prefix kdelibs was compiled with, not the application. As I was using my own KDE, of course it could not find Yakuake’s files. Aha! So I add /usr to KDEDIRS and everything works.

I'm sorry Dave. I'm afraid I still can't do that.

Bah! What’s going on?

Well, if we look at the set of prefixes, the standard suffix for data is share/apps. This fairly KDE-specific namespace in a global install gets stuffed under /usr/share/apps, which is offensive to distributions, so they like to redirect it to /usr/share/kde4/apps. A few other directories get a similar treatment. In Ubuntu’s case, a snippet from /usr/share/pkg-kde-tools/makefiles/1/variables.mk reveals the cause:

# Standard Debian KDE 4 cmake flags
DEB_CMAKE_KDE4_FLAGS += \
        -DCMAKE_BUILD_TYPE=Debian \
        -DKDE4_BUILD_TESTS=false \
        -DKDE_DISTRIBUTION_TEXT="Kubuntu packages" \
        -DCMAKE_SKIP_RPATH=true \
        -DKDE4_USE_ALWAYS_FULL_RPATH=false \
        -DCONFIG_INSTALL_DIR=$(DEB_CONFIG_INSTALL_DIR) \
        -DDATA_INSTALL_DIR=/usr/share/kde4/apps \
        -DHTML_INSTALL_DIR=/usr/share/doc/kde/HTML \
        -DKCFG_INSTALL_DIR=/usr/share/kde4/config.kcfg \
        -DLIB_INSTALL_DIR=/usr/lib \
        -DSYSCONF_INSTALL_DIR=/etc

My kdelibs, however, were compiled directly from upstream sources (in fact, I compiled from the 4.4 branch on a git-svn and hack on it myself). Moreover, these settings fail to set the standard suffixes, only a compiled-in value. (Kubuntu also carries a patch that changes the system-wide FindKDE4Internal.cmake. It may actually register suffixes. I’m not sure.) When using the system kdelibs, these compiled values do their job and everything works fine. However, this makes the system KDE files special in that they are only a priori accessible via the system kdelibs. While I can inform KDE of the system root, the suffix is wrong.

So, I add a little hack. I have yet another locker, kde-kubuntu-fake which contains a fake additional root for each of those directories. This contains merely a symlink farm:

kde-kubuntu-fake
`-- share
    |-- apps -> /usr/share/kde4/apps/
    |-- config -> /usr/share/kde4/config
    |-- config.kcfg -> /usr/share/kde4/config.kcfg
    `-- doc
        `-- HTML -> /usr/share/doc/kde4/HTML

which also gets added to my KDEDIRS. Finally, after all that work, I can launch Yakuake.

So, hopefully this will help convince that random distribution patches like this are sketchy. Admittedly, given the mistake of trying to mush all packages into one single hierarchy under /usr, the namespace poisoning of /usr/share/apps is a little obnoxious, and this is a defensible change. Still, such things do prevent the compatibility between distributions and upstream and make it very hard for a unified free desktop platform to ever emerge from this tangled mess we have now.

C++ is a hideously complicated language with many little-known features and details. One of these features is member pointers, which we shall play a few games with here. The feature itself is largely useless, but quite amusing.

C++ allows pointers to a member of a class. This can be a pointer to a member function (in which case you get a delegate) or a pointer to a data member. Now, for reasons I won’t go into, member function pointers are incredibly complex. Member data pointers, however, are fairly simple; they’re just type-safe offsets with strange syntax (and limited use).

The syntax is as follows:

struct A {
    int a;
    int b;
};

int main() {
    A object;
    int A::* some_field = &A::b;
    object.*some_field = 27;
};

The type declaration states that some_field is a member of class A with type int. This is precisely an offsetof.

The C++ language doesn’t allow you to easily cast these into longs, but you can use the usual offsetof pointer trick to do it.

template<class C, class V>
inline unsigned long memptr_to_int(V C::*ptr) {
    return (unsigned long) &(((C *)0)->*ptr);
}
// ...
int A::*some_field = &A::a;
unsigned long offset = memptr_to_int(some_field);

The compiler will also allow you to change either of the types embedded in the pointer with a reinterpret_cast, for instance:

int A::* foo = &A::field;
char B::* bar = reinterpret_cast<char B::*>(foo);

Using this, we can cast any arbitrary constant integer to a cast:

template <int N> struct offset_struct {
    int pad[N];
    int field;
private:
    offset_struct() {} // forbid actual construction
};

template<class C, class V, int N>
inline V C::* const_to_memptr() {
    return reinterpret_cast<V C::*>(&(offset_struct<N>::field));
}

// ...
int A::*some_field = const_to_memptr<A, int, 5>();

Sadly, I have not yet found a way (short of terrible tricks with unions) to convert an arbitrary integer variable into such an offset. So, if you need offsets which (almost) prevent you from pointing to any undefined fields in a class, C++ data member pointers are what you want.

In practice, this construct sees considerably more use as a template argument, or something similarly inlined. The actual pointers tend not to be created. For instance, in the Boost.Python library, they’re used to create properties out of fields.

Isn’t C++ fun?