On malice and stupidity

In my previous post, I attributed a quotation to one Robert J. Hanlon. This quotation, known as Hanlon’s Razor, deserves a little more attention.

Firstly, I altered the phrase slightly compared its most common form, “Never attribute to malice that which can be adequately explained by stupidity,” substituting incompetence as the final word. I did this simply because I found this form more suitable in the context.

Secondly, the origin of this adage is disputable. A selection of alternatives follows.

  • In his 1980 book Murphy’s Law Book Two: More Reasons why Things Go Wrong!, Arthur Bloch credits Robert J. Hanlon as the creator, citing the above version.
  • Bill Clarke claims to have coined the phrase in 1974, in the story Axioms of a Mad Poet he published that year.
  • In the short story Logic of Empire (1941) by Robert A. Heinlein a similar phrase appears: You have attributed conditions to villainy that simply result from stupidity.
  • Napoleon Bonaparte allegedly uttered the words “Never ascribe to malice that which is adequately explained by incompetence,” although accurate references do not appear to exist.

Perhaps there is some truth to the saying that great minds think alike.

CodeSourcery’s defence

Having covered the spectacular failure of CodeSourcery’s latest ARM compiler a few days ago, I was engaged in a curious debate on IRC with one of their employees. Fiercely denying the problem at first, he eventually offered an explanation: they do not test the compiler output on real hardware; they use QEMU.

QEMU is a CPU emulator supporting a variety of targets. While great for casual development, and for running foreign applications, it is certainly no substitute for real hardware when testing a compiler. Like any piece of software, an emulator is bound to have a few errors, and as it happens, QEMU has known bugs in its handling of the NEON instruction set. Our friend at CodeSourcery should be well aware of these, also being a QEMU developer.

The use of emulators was explained as a necessity due to real hardware not being available. To be fair, CodeSourcery does develop against new hardware before it exists, so some reliance on emulators is unavoidable. This is, however, not the case this time. The Beagleboard was made available to selected developers quite some time ago (I have had one since May, others still longer), and is now being sold by the thousands. CodeSourcery developers, so I am told, were also given an offer of a free board, an offer they chose to refuse.

What does all this mean? Did Murphy decide to inflict maximum bad luck on the hard-working developers, or is there perhaps a larger conspiracy at work? I shall not attempt to speculate in this matter. I will merely repeat this excellent piece of advice given by Robert J. Hanlon: Never attribute to malice that which can be adequately explained by incompetence.

CodeSourcery GCC 2008q3: FAIL

A few days ago, CodeSourcery released their latest version of GCC for ARM, dubbed 2008q3. An announcement email boasts “Improved support for NEON and, in particular, auto-vectorization using NEON.” It is time to put that claim to the test.

FFmpeg has a history of triggering compiler bugs, making it a good test case. Some extra speed would do it good as well.

The new compiler builds FFmpeg without complaint, so everything is looking good so far. To check for any speedup from the improved compiler, I use an Indiana Jones trailer encoded with H.264. Disappointingly, I am unable to get any speed figures. The decoding stops after 160 frames, the immediate cause being an unaligned NEON load in simple loop for copying a few bytes.

Is FFmpeg broken? The same code built with an older compiler release works perfectly, and the parameters passed to the failing function are similar-looking. The answer must lie in the copy loop itself. To verify this hypothesis, I set out to reproduce the error with a minimal test case.

The failure proves remarkably simple to trigger. The test case I arrive at consists of two C source files. The first file is our copy loop:

void copy(char *dst, char *src, int len)
    int i;
    for (i = 0; i < len; i++)
        dst[i] = src[i];

The second file is our main() function, invoking the copy with suitably unaligned arguments:

extern void copy(char *dst, char *src, int len);
char src[20], dst[16];

int main(void)
    char *p = src + !((unsigned)src & 1);
    copy(dst, p, 16);
    return 0;

Compiling this with -mfpu=neon -mfloat-abi=softfp -mcpu=cortex-a8 -O3 flags results in a broken executable. Adding -fno-tree-vectorize makes the error go away.

So much for the improved auto-vectorisation.

Not testing every compiler on FFmpeg is understandable. Not testing even the most trivial of constructs is unforgivable.