Constraints

Each operand of an inline asm block is described by a constraint string encoding the valid representations of the operand in the generated assembler. For example the “r” code denotes a general-purpose register. In addition to the standard constraints, ARM allows a number of special codes, only some of which are documented. The full list, including a brief description, is available in the constraints.md file in the GCC source tree. The following table is an extract from this file consisting of the codes which are meaningful in an inline asm block (a few are only useful in the machine description itself).

Operand codes

Within the text of an inline asm block, operands are referenced as %0, %1 etc. Register operands are printed as rN, memory operands as [rN, #offset], and so forth. In some situations, for example with operands occupying multiple registers, more detailed control of the output may be required, and once again, an undocumented feature comes to our rescue.

Special code letters inserted between the % and the operand number alter the output from the default for each type of operand. The table below lists the more useful ones.

struct bf1_31 {
    unsigned a:1;
    unsigned b:31;
};

void func(struct bf1_31 *p, int n, int a)
{
    int i = 0;
    do {
        if (p[i].a)
            p[i].b += a;
    } while (++i < n);
}

How would we best write this in ARM assembler? This is how I would do it:

func:
        ldr     r3,  [r0], #4
        tst     r3,  #1
        add     r3,  r3,  r2,  lsl #1
        strne   r3,  [r0, #-4]
        subs    r1,  r1,  #1
        bgt     func
        bx      lr

The add instruction is unconditional to avoid a dependency on the comparison. Unrolling the loop would mask the latency of the ldr instruction as well, but that is outside the scope of this experiment.

Now compile this code with gcc -march=armv5te -O3 and watch in horror:

func:
        push    {r4}
        mov     ip, #0
        mov     r4, r2
loop:
        ldrb    r3, [r0]
        add     ip, ip, #1
        tst     r3, #1
        ldrne   r3, [r0]
        andne   r2, r3, #1
        addne   r3, r4, r3, lsr #1
        orrne   r2, r2, r3, lsl #1
        strne   r2, [r0]
        cmp     ip, r1
        add     r0, r0, #4
        blt     loop
        pop     {r4}
        bx      lr

This is nothing short of awful:

• The same value is loaded from memory twice.
• A complicated mask/shift/or operation is used where a simple shifted add would suffice.
• Write-back addressing is not used.
• The loop control counts up and compares instead of counting down.
• Useless mov in the prologue; swapping the roles or r2 and r4 would avoid this.
• Using lr in place of r4 would allow the return to be done with pop {pc}, saving one instruction (ignoring for the moment that no callee-saved registers are needed at all).

Even for this trivial function the gcc-generated code is more than twice the optimal size and slower by approximately the same factor.

The main issue I wanted to illustrate is the poor handling of bit-fields by gcc. When accessing bitfields from memory, gcc issues a separate load for each field even when they are contained in the same aligned memory word. Although each load after the first will most likely hit L1 cache, this is still bad for several reasons:

• Loads have typically two or three cycles result latency compared to one cycle for data processing instructions. Any bit-field can be extracted from a register with two shifts, and on ARM the second of these can generally be achieved using a shifted second operand to a following instruction. The ARMv6T2 instruction set also adds the SBFX and UBFX instructions for extracting any signed or unsigned bit-field in one cycle.
• Most CPUs have more data processing units than load/store units. It is thus more likely for an ALU instruction than a load/store to issue without delay on a superscalar processor.
• Redundant memory accesses can trigger early flushing of store buffers rendering these less efficient.

No gcc bashing is complete without a comparison with another compiler, so without further ado, here is the ARM RVCT output (armcc --cpu 5te -O3):

func:
        mov     r3, #0
        push    {r4, lr}
loop:
        ldr     ip, [r0, r3, lsl #2]
        tst     ip, #1
        addne   ip, ip, r2, lsl #1
        strne   ip, [r0, r3, lsl #2]
        add     r3, r3, #1
        cmp     r3, r1
        blt     loop
        pop     {r4, pc}

This is much better, the core loop using only one instruction more than my version. The loop control is counting up, but at least this register is reused as offset for the memory accesses. More remarkable is the push/pop of two registers that are never used. I had not expected to see this from RVCT.

Even the best compilers are still no match for a human.

Both the 4.3 and 4.4 branches of FSF GCC have had bugfix releases, bringing them to 4.3.4 and 4.4.2, respectively. Neither update contains anything particularly noteworthy.

The CodeSourcery 2009q3 release sees an update to a GCC 4.4 base, a significant change from the 4.3 base used in 2009q1. The update is a mixed blessing. In fact, it is mostly a curse and hardly a blessing at all. On the bright side, the floating-point speed regressions in 2009q1 are gone, 2009q3 being a few per cent faster even than 2007q3. Unfortunately, this improvement is completely overshadowed by a major speed regression on integer code, a whopping 24% in one case. This ties in with the slowdown previously observed with FSF GCC 4.4 compared to 4.3.

ARM RVCT 4.0 is now at Build 697. This update fixes some bugs and introduces others. Notably, it no longer builds FFmpeg correctly. The issue has been reported to ARM.

Texas Instruments, finally, have made a formal release, v4.6.1, of their TMS470 compiler incorporating various fixes allowing it to build a moderately patched FFmpeg. The performance remains somewhere between GCC and RVCT on average.

In light of the above, my recommendations remain unchanged:

• For a free compiler, choose CodeSourcery 2009q1. It beats GCC 4.3.4 by 5-10% in most cases.
• GNU purists are best served by GCC 4.3.4, which is up to 20% faster than 4.4.2 and rarely slower.
• When price is not a concern, ARM RCVT is a good option, outperforming GCC by up to a factor 2.
• In all cases, disable any auto-vectorisation features.

Regardless of which compiler is chosen, I cannot overstress the importance of testing. All compilers are crawling with bugs, and even the most innocent-looking code change can trigger one of them. When using a compiler other than GCC, extra caution is advised considering a lot of code is developed using only GCC and may thus fall prey to bugs unique to said other compiler.

For my test, I selected the following functions:

• __builtin_bswap32: Byte-swap a 32-bit word.
• __builtin_bswap64: Byte-swap a 64-bit word.
• __builtin_clz: Count leading zeros in a word.
• __builtin_ctz: Count trailing zeros in a word.
• __builtin_prefetch: Prefetch data into cache.

To test the quality of these builtins, I wrapped each in a normal function, then compiled the code for these targets:

• ARMv7
• AVR32
• MIPS
• MIPS64
• PowerPC
• PowerPC64
• x86
• x86_64

In all cases I used compiler flags were -O3 -fomit-frame-pointer plus any flags required to select a modern CPU model.

ARM

Both  __builtin_clz and __builtin_prefetch generate the expected CLZ and PLD instructions respectively. The code for __builtin_ctz is reasonable for ARMv6 and earlier:

rsb     r3, r0, #0
and     r0, r3, r0
clz     r0, r0
rsb     r0, r0, #31

For ARMv7 (in fact v6T2), however, using the new bit-reversal instruction would have been better:

rbit    r0, r0
clz     r0, r0

I suspect this is simply a matter of the function not yet having been updated for ARMv7, which is perhaps even excusable given the relatively rare use cases for it.

The byte-reversal functions are where it gets shocking. Rather than use the REV instruction found from ARMv6 on, both of them generate external calls to __bswapsi2 and __bswapdi2 in libgcc, which is plain C code:

SItype
__bswapsi2 (SItype u)
{
  return ((((u) & 0xff000000) >> 24)
          | (((u) & 0x00ff0000) >>  8)
          | (((u) & 0x0000ff00) <<  8)
          | (((u) & 0x000000ff) << 24));
}

DItype
__bswapdi2 (DItype u)
{
   return ((((u) & 0xff00000000000000ull) >> 56)
          | (((u) & 0x00ff000000000000ull) >> 40)
          | (((u) & 0x0000ff0000000000ull) >> 24)
          | (((u) & 0x000000ff00000000ull) >>  8)
          | (((u) & 0x00000000ff000000ull) <<  8)
          | (((u) & 0x0000000000ff0000ull) << 24)
          | (((u) & 0x000000000000ff00ull) << 40)
          | (((u) & 0x00000000000000ffull) << 56));
}

While the 32-bit version compiles to a reasonable-looking shift/mask/or job, the 64-bit one is a real WTF. Brace yourselves:

push    {r4, r5, r6, r7, r8, r9, sl, fp}
mov     r5, #0
mov     r6, #65280      ; 0xff00
sub     sp, sp, #40     ; 0x28
and     r7, r0, r5
and     r8, r1, r6
str     r7, [sp, #8]
str     r8, [sp, #12]
mov     r9, #0
mov     r4, r1
and     r5, r0, r9
mov     sl, #255        ; 0xff
ldr     r9, [sp, #8]
and     r6, r4, sl
mov     ip, #16711680   ; 0xff0000
str     r5, [sp, #16]
str     r6, [sp, #20]
lsl     r2, r0, #24
and     ip, ip, r1
lsr     r7, r4, #24
mov     r1, #0
lsr     r5, r9, #24
mov     sl, #0
mov     r9, #-16777216  ; 0xff000000
and     fp, r0, r9
lsr     r6, ip, #8
orr     r9, r7, r1
and     ip, r4, sl
orr     sl, r1, r2
str     r6, [sp]
str     r9, [sp, #32]
str     sl, [sp, #36]   ; 0x24
add     r8, sp, #32
ldm     r8, {r7, r8}
str     r1, [sp, #4]
ldm     sp, {r9, sl}
orr     r7, r7, r9
orr     r8, r8, sl
str     r7, [sp, #32]
str     r8, [sp, #36]   ; 0x24
mov     r3, r0
mov     r7, #16711680   ; 0xff0000
mov     r8, #0
and     r9, r3, r7
and     sl, r4, r8
ldr     r0, [sp, #16]
str     fp, [sp, #24]
str     ip, [sp, #28]
stm     sp, {r9, sl}
ldr     r7, [sp, #20]
ldr     sl, [sp, #12]
ldr     fp, [sp, #12]
ldr     r8, [sp, #28]
lsr     r0, r0, #8
orr     r7, r0, r7, lsl #24
lsr     r6, sl, #24
orr     r5, r5, fp, lsl #8
lsl     sl, r8, #8
mov     fp, r7
add     r8, sp, #32
ldm     r8, {r7, r8}
orr     r6, r6, r8
ldr     r8, [sp, #20]
ldr     r0, [sp, #24]
orr     r5, r5, r7
lsr     r8, r8, #8
orr     sl, sl, r0, lsr #24
mov     ip, r8
ldr     r0, [sp, #4]
orr     fp, fp, r5
ldr     r5, [sp, #24]
orr     ip, ip, r6
ldr     r6, [sp]
lsl     r9, r5, #8
lsl     r8, r0, #24
orr     fp, fp, r9
lsl     r3, r3, #8
orr     r8, r8, r6, lsr #8
orr     ip, ip, sl
lsl     r7, r6, #24
and     r5, r3, #16711680       ; 0xff0000
orr     r7, r7, fp
orr     r8, r8, ip
orr     r4, r1, r7
orr     r5, r5, r8
mov     r9, r6
mov     r1, r5
mov     r0, r4
add     sp, sp, #40     ; 0x28
pop     {r4, r5, r6, r7, r8, r9, sl, fp}
bx      lr

That’s right, 91 instructions to move 8 bytes around a bit. GCC definitely has a problem with 64-bit numbers. It is perhaps worth noting that the bswap_64 macro in glibc splits the 64-bit value into 32-bit halves which are then reversed independently, thus side-stepping this weakness of gcc.

As a side note, ARM RVCT (armcc) compiles those functions perfectly into one and two REV instructions, respectively.

AVR32

There is not much to report here. The latest gcc version available is 4.2.4, which doesn’t appear to have the bswap functions. The other three are handled nicely, even using a bit-reverse for __builtin_ctz.

MIPS / MIPS64

The situation MIPS is similar to ARM. Both bswap builtins result in external libgcc calls, the rest giving sensible code.

PowerPC

I scarcely believe my eyes, but this one is actually not bad. The PowerPC has no byte-reversal instructions, yet someone seems to have taken the time to teach gcc a good instruction sequence for this operation. The PowerPC does have some powerful rotate-and-mask instructions which come in handy here. First the 32-bit version:

rotlwi  r0,r3,8
rlwimi  r0,r3,24,0,7
rlwimi  r0,r3,24,16,23
mr      r3,r0
blr

The 64-bit byte-reversal simply applies the above code on each half of the value:

rotlwi  r0,r3,8
rlwimi  r0,r3,24,0,7
rlwimi  r0,r3,24,16,23
rotlwi  r3,r4,8
rlwimi  r3,r4,24,0,7
rlwimi  r3,r4,24,16,23
mr      r4,r0
blr

Although I haven’t analysed that code carefully, it looks pretty good.

PowerPC64

Doing 64-bit operations is easier on a 64-bit CPU, right? For you and me perhaps, but not for gcc. Here __builtin_bswap64 gives us the now familiar __bswapdi2 call, and while not as bad as the ARM version, it is not pretty:

rldicr  r0,r3,8,55
rldicr  r10,r3,56,7
rldicr  r0,r0,56,15
rldicl  r11,r3,8,56
rldicr  r9,r3,16,47
or      r11,r10,r11
rldicr  r9,r9,48,23
rldicl  r10,r0,24,40
rldicr  r0,r3,24,39
or      r11,r11,r10
rldicl  r9,r9,40,24
rldicr  r0,r0,40,31
or      r9,r11,r9
rlwinm  r10,r3,0,0,7
rldicl  r0,r0,56,8
or      r0,r9,r0
rldicr  r10,r10,8,55
rlwinm  r11,r3,0,8,15
or      r0,r0,r10
rldicr  r11,r11,24,39
rlwinm  r3,r3,0,16,23
or      r0,r0,r11
rldicr  r3,r3,40,23
or      r3,r0,r3
blr

That is 6 times longer than the (presumably) hand-written 32-bit version.

x86 / x86_64

As one might expect, results on x86 are good. All the tested functions use the available special instructions. One word of caution though: the bit-counting instructions are very slow on some implementations, specifically the Atom, AMD chips, and the notoriously slow Pentium4E.

Conclusion

In conclusion, I would say gcc builtins can be useful to avoid fragile inline assembler. Before using them, however, one should make sure they are not in fact harmful on the required targets. Not even those builtins mapping directly to CPU instructions can be trusted.

The contenders this time were the fastest GCC variant from round one, ARM RVCT, and newcomer TI TMS470. With the same rules as last time, the exact versions and optimisation options were like this:

• CodeSourcery GCC 2009q1 (based on 4.3.3), -mfpu=neon -mfloat-abi=softfp -mcpu=cortex-a8 -std=c99 -fomit-frame-pointer -O3 -fno-math-errno -fno-signed-zeros -fno-tree-vectorize
• ARM RVCT 4.0 Build 591, -mfpu=neon -mfloat-abi=softfp -mcpu=cortex-a8 -std=c99 -fomit-frame-pointer -O3 -fno-math-errno -fno-signed-zeros
• TI TMS470 4.7.0-a9229, --float_support=vfpv3 -mv=7a8 -O3 -mf=5

To keep things fair, I left the vectoriser off also with the TI compiler. The table below lists the decoding times for the sample files, this time normalised against the participating GCC compiler. Remember, smaller numbers are better.  Also keep in mind that this test was done with a development snapshot of TMS470, not an approved release.

Overall, the TI TMS470 compiler comes off slightly worse than GCC. In two cases, however, it was significantly better than GCC, but not as good as RVCT. Incidentally, those were also the ones where RVCT scored the biggest win over GCC.

My conclusions from this test are twofold:

• ARM’s own compiler is very hard to beat. They do seem to know how their chips work.
• GCC is incredibly bad at 64-bit arithmetic on 32-bit machines.

The logical next step is to test these compilers with vectorisation enabled. FFmpeg should offer plenty of opportunities for this feature to shine. Unfortunately, that test will have to wait until the RVCT vectoriser is fixed. The current release does not compile FFmpeg with vectorisation enabled.

The installation script, as expected, copied a number of files into a directory under /opt. More unusually, it also created a small shared library, libxlc101e.so.1, and placed it in /usr/lib. No other files from the installation package were modified, so this must be where the licence is hiding. Without further ado, we proceed to take it apart.

We begin by looking at the symbol table using readelf -s:

Symbol table '.symtab' contains 44 entries:
   Num:    Value  Size Type    Bind   Vis      Ndx Name
     0: 00000000     0 NOTYPE  LOCAL  DEFAULT  UND
     1: 000000b4     0 SECTION LOCAL  DEFAULT    1
     2: 0000017c     0 SECTION LOCAL  DEFAULT    2
     3: 0000036c     0 SECTION LOCAL  DEFAULT    3
     4: 0000055c     0 SECTION LOCAL  DEFAULT    4
     5: 00000580     0 SECTION LOCAL  DEFAULT    5
     6: 000005c4     0 SECTION LOCAL  DEFAULT    6
     7: 00010a3c     0 SECTION LOCAL  DEFAULT    7
     8: 00010a50     0 SECTION LOCAL  DEFAULT    8
     9: 00010ac8     0 SECTION LOCAL  DEFAULT    9
    10: 00010ad8     0 SECTION LOCAL  DEFAULT   10
    11: 00000000     0 SECTION LOCAL  DEFAULT   11
    12: 00000000     0 SECTION LOCAL  DEFAULT   12
    13: 00000000     0 SECTION LOCAL  DEFAULT   13
    14: 00000000     0 SECTION LOCAL  DEFAULT   14
    15: 00000000     0 FILE    LOCAL  DEFAULT  ABS xleval.c
    16: 00010a50     0 OBJECT  LOCAL  HIDDEN   ABS _DYNAMIC
    17: 00010acc     0 OBJECT  LOCAL  HIDDEN   ABS _GLOBAL_OFFSET_TABLE_
    18: 000005f0    24 OBJECT  GLOBAL DEFAULT    6 xlc_extended_eval_lic_dir
    19: 00000660    22 OBJECT  GLOBAL DEFAULT    6 libxlfextendeval_name
    20: 00000738    42 OBJECT  GLOBAL DEFAULT    6 stm_compiler_name
    21: 00000640    16 OBJECT  GLOBAL DEFAULT    6 libupclicense_name
    22: 00000764    32 OBJECT  GLOBAL DEFAULT    6 xlf_compiler_name
    23: 00000580    68 FUNC    GLOBAL DEFAULT    5 _xlgetevalbeta
    24: 00000678    22 OBJECT  GLOBAL DEFAULT    6 libxlcextendeval_name
    25: 00000608    24 OBJECT  GLOBAL DEFAULT    6 xlf_extended_eval_lic_dir
    26: 000006d8    17 OBJECT  GLOBAL DEFAULT    6 xlcmp_name
    27: 000006c0    12 OBJECT  GLOBAL DEFAULT    6 xlc_package_name
    28: 00000650    16 OBJECT  GLOBAL DEFAULT    6 libstmlicense_name
    29: 000005e0    16 OBJECT  GLOBAL DEFAULT    6 xlf_extend_eval_env_var
    30: 000005d0    16 OBJECT  GLOBAL DEFAULT    6 xlc_extend_eval_env_var
    31: 00000620    16 OBJECT  GLOBAL DEFAULT    6 libxlflicense_name
    32: 00010cd0     0 NOTYPE  GLOBAL DEFAULT  ABS __bss_start
    33: 00010a3c    20 OBJECT  GLOBAL DEFAULT    7 _xlevalbeta
    34: 000005c4    10 OBJECT  GLOBAL DEFAULT    6 liblicense_dir
    35: 00000690    22 OBJECT  GLOBAL DEFAULT    6 libupcextendeval_name
    36: 000006ec    30 OBJECT  GLOBAL DEFAULT    6 xlc_compiler_name
    37: 00000630    16 OBJECT  GLOBAL DEFAULT    6 libxlclicense_name
    38: 00010cd0     0 NOTYPE  GLOBAL DEFAULT  ABS _edata
    39: 00010cd0     0 NOTYPE  GLOBAL DEFAULT  ABS _end
    40: 000006cc    12 OBJECT  GLOBAL DEFAULT    6 xlf_package_name
    41: 000006a8    22 OBJECT  GLOBAL DEFAULT    6 libstmextendeval_name
    42: 00010ad8   504 OBJECT  GLOBAL DEFAULT   10 versionString
    43: 0000070c    42 OBJECT  GLOBAL DEFAULT    6 upc_compiler_name

Notice the lone function at position 23, _xlgetevalbeta, which we proceed to disassemble:

00000580 <_xlgetevalbeta>:
 580:   94 21 ff f0     stwu    r1,-16(r1)
 584:   93 c1 00 08     stw     r30,8(r1)
 588:   93 e1 00 0c     stw     r31,12(r1)
 58c:   7c 3f 0b 78     mr      r31,r1
 590:   7d 88 02 a6     mflr    r12
 594:   42 9f 00 05     bcl-    20,4*cr7+so,598 <_xlgetevalbeta+0x18>
 598:   7f c8 02 a6     mflr    r30
 59c:   3f de 00 01     addis   r30,r30,1
 5a0:   3b de 05 34     addi    r30,r30,1332
 5a4:   7d 88 03 a6     mtlr    r12
 5a8:   80 1e ff fc     lwz     r0,-4(r30)
 5ac:   7c 03 03 78     mr      r3,r0
 5b0:   81 61 00 00     lwz     r11,0(r1)
 5b4:   83 cb ff f8     lwz     r30,-8(r11)
 5b8:   83 eb ff fc     lwz     r31,-4(r11)
 5bc:   7d 61 5b 78     mr      r1,r11
 5c0:   4e 80 00 20     blr

This is fairly standard, unoptimised code. After saving a few registers on the stack, it computes the address of the global offset table: 0x598 + 0x10000 + 1332 = 0x10acc, matching _GLOBAL_OFFSET_TABLE_ from the symbol table. Next, a value is loaded from the GOT, forming the return value of the function after the stack has been restored.

To find out what this return value really is, we look at the relocation table (by means of readelf -r):

Relocation section '.rela.dyn' at offset 0x55c contains 3 entries:
 Offset     Info    Type            Sym.Value  Sym. Name + Addend
00010a40  00000016 R_PPC_RELATIVE                               00000784
00010a44  00000016 R_PPC_RELATIVE                               0000097c
00010ac8  00001414 R_PPC_GLOB_DAT    00010a3c   _xlevalbeta + 0

The third entry is the one we are looking for: its offset matches the location read by the code at 0x5a8. This means the _xlgetevalbeta function is returning a pointer to _xlevalbeta, which makes some kind of sense.

Another quick look at the symbol table tells us _xlevalbeta lives at address 0x10a3c and is 20 bytes in size. The section header (provided by readelf -S) helps us find the corresponding location in the file:

Section Headers:
  [Nr] Name              Type            Addr     Off    Size   ES Flg
  [ 0]                   NULL            00000000 000000 000000 00
  [ 1] .hash             HASH            000000b4 0000b4 0000c8 04   A
  [ 2] .dynsym           DYNSYM          0000017c 00017c 0001f0 10   A
  [ 3] .dynstr           STRTAB          0000036c 00036c 0001ee 00   A
  [ 4] .rela.dyn         RELA            0000055c 00055c 000024 0c   A
  [ 5] .text             PROGBITS        00000580 000580 000044 00  AX
  [ 6] .rodata           PROGBITS        000005c4 0005c4 000475 00   A
  [ 7] .data.rel.ro      PROGBITS        00010a3c 000a3c 000014 00  WA
  [ 8] .dynamic          DYNAMIC         00010a50 000a50 000078 08  WA
  [ 9] .got              PROGBITS        00010ac8 000ac8 000010 04  WA
  [10] .data             PROGBITS        00010ad8 000ad8 0001f8 00  WA
  [11] .comment          PROGBITS        00000000 000cd0 000028 00
  [12] .shstrtab         STRTAB          00000000 000cf8 000073 00
  [13] .symtab           SYMTAB          00000000 000fc4 0002c0 10
  [14] .strtab           STRTAB          00000000 001284 000206 00
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings)
  I (info), L (link order), G (group), x (unknown)
  O (extra OS processing required) o (OS specific), p (processor specific)

The address we are looking for is at the start of the .data.rel.ro section, which can be found at offset 0xa3c in the file. It is time for the hexdump tool:

00000a30  69 62 69 74 65 64 2e 00  00 00 00 00 00 00 00 01
00000a40  00 00 00 00 00 00 00 00  00 00 24 05 4a 0c 65 c8

The last four bytes here, 4a 0c 65 c8, are interesting. Taken as a 32-bit big endian value, they are exactly equal to the modification time of the file, or in other words, the time the compiler was installed. This cannot be a coincidence, so using a hex editor, we replace this with the current time, 4a 80 74 31.

Lo and behold, the compiler is working again.

One hopes the engineers at IBM developing the compiler are not the same ones thinking this copy protection method was a good idea. Then again, perhaps they are; it failed miserably at compiling FFmpeg.

These are the compilers I deemed worthy to participate in the test and the optimisation flags I used with each:

• GCC 4.3.3, -mfpu=neon -mfloat-abi=softfp -mcpu=cortex-a8 -std=c99 -fomit-frame-pointer -O3 -fno-math-errno -fno-signed-zeros -fno-tree-vectorize
• GCC 4.4.1, -mfpu=neon -mfloat-abi=softfp -mcpu=cortex-a8 -std=c99 -fomit-frame-pointer -O3 -fno-math-errno -fno-signed-zeros -fno-tree-vectorize
• CodeSourcery GCC 2007q3 (based on 4.2.1), -mfpu=neon -mfloat-abi=softfp -mcpu=cortex-a8 -std=c99 -fomit-frame-pointer -O3 -fno-math-errno -fno-tree-vectorize
• CodeSourcery GCC 2009q1 (based on 4.3.3), -mfpu=neon -mfloat-abi=softfp -mcpu=cortex-a8 -std=c99 -fomit-frame-pointer -O3 -fno-math-errno -fno-signed-zeros -fno-tree-vectorize
• ARM RVCT 4.0 Build 591, -mfpu=neon -mfloat-abi=softfp -mcpu=cortex-a8 -std=c99 -fomit-frame-pointer -O3 -fno-math-errno -fno-signed-zeros

I would have also included the ARM compiler from Texas Instruments, had it been able to compile FFmpeg.

With sample files chosen to exercise various types of code, the result of the test is, sadly, no surprise. The following table lists the runtimes of the different builds relative to the CodeSourcery 2007q3 build. Lower numbers are better.

Looking at the table, I make these observations:

• CodeSourcery 2009q1 produces faster integer code, but slower floating-point code, than 2007q3.
• GCC 4.4.1 produces much slower code than 4.3.3 in several cases, and is never significantly better.
• CodeSourcery GCC generally beats FSF GCC.
• ARM RVCT readily beats every GCC version. The MP3 figure is not a typo.

My recommendation for a free compiler is CodeSourcery 2009q1 unless your code makes heavy use of floating-point, in which case 2007q3 may give better results. If you prefer, for whatever reason, official GNU releases, 4.3.3 should be the version of choice. Avoid GCC 4.4.1; it is far too unpredictable.

Bootnotes

• See also Mike’s test of x86 compilers.
• Thanks to ARM for providing the RVCT compiler.
• Thanks to TI for providing the Beagle board.

A session with oprofile exposes multiplication as the root of the problem. The MPEG audio decoder in FFmpeg includes many operations of the form a += b * c where b and c are 32 bits in size and a is 64-bit. 64-bit maths on a 32-bit CPU is not handled well by GCC, even when good hardware support is available. A couple of examples compiled with GCC 4.3.3 illustrate this.

Suppose you need the high 32 bits from the 64-bit result of multiplying two 32-bit numbers. This is most easily written in C like this:

int mulh(int a, int b)
{
    return ((int64_t)a * (int64_t)b) >> 32;
}

It doesn’t take much thinking to see that the PowerPC mulhw instruction performs exactly this operation. Indeed, GCC knows of this instruction and uses it. But can we be really sure that those low 32 bits are not needed? GCC seems unconvinced:

mulhw   r9,  r4,  r3
mullw   r10, r4,  r3
srawi   r11, r9,  31
srawi   r12, r9,  0
mr      r3,  r12
blr

The second example is slightly more complicated:

int64_t mac(int64_t a, int b, int c, int d)
{
    a += (int64_t)b * (int64_t)c;
    a += (int64_t)b * (int64_t)d;
    return a;
}

This can, of course, be done with four multiplications and four additions. GCC, however, likes to be thorough, and uses twice the number of both instructions, plus some loads, stores and shifts for completeness:

stwu    r1,  -32(r1)
srawi   r0,  r6,  31
mullw   r0,  r0,  r5
srawi   r8,  r7,  31
stw     r29, 20(r1)
srawi   r29, r5,  31
stw     r27, 12(r1)
stw     r28, 16(r1)
mullw   r11, r29, r6
mulhwu  r9,  r6,  r5
add     r0,  r0,  r11
mullw   r10, r6,  r5
add     r9,  r0,  r9
mullw   r29, r29, r7
addc    r28, r10, r4
adde    r27, r9,  r3
mullw   r8,  r8,  r5
mulhwu  r9,  r7,  r5
add     r8,  r8,  r29
lwz     r29, 20(r1)
mullw   r10, r7,  r5
add     r9,  r8,  r9
addc    r12, r28, r10
adde    r11, r27, r9
lwz     r27, 12(r1)
mr      r4,  r12
lwz     r28, 16(r1)
mr      r3,  r11
addi    r1,  r1,  32
blr

Fortunately, this madness is easily fixed with a little inline assembler, more than doubling the speed of the decoder, thus making FFmpeg significantly faster than MAD also on PowerPC.

Thumb-2 performance is claimed to reach 98% of the equivalent ARM code while being only 74% of the size. I decided to put this claim to the test with FFmpeg as the target and compiled the same source revision in ARM and Thumb-2 mode using the RVCT 4.0 compiler. For this test I disabled all hand-written assembler optimisations.

The Thumb-2 executable is 85% of the ARM one in size, which although being a substantial reduction falls somewhat short of the promised 74%. I tested the performance by measuring the time to decode a few sample media files on a Beagle board. Several of the samples actually decoded faster with the Thumb-2 build, with one H.264 video clip decoding 4% faster. Only one test, MP3 audio decoding, was significantly slower (15%) compared to ARM code. The speedup is likely due to reduced I-cache pressure. Thumb-2 and ARM instructions are executed identically after the initial decode stage, so no improvement can result from the change of instruction set alone.

In conclusion, the Thumb-2 performance is better than I had expected. Nevertheless, a 15% slowdown in even one case is reason enough to carefully benchmark the effects before deciding on a switch.

The iPhone is built around an ARM1176 CPU, so the SDK includes an ARM cross-compiler and assembler. Most of the reported errors originate from the Apple assembler which appears to have trouble processing the assembler source files from FFmpeg.

The source files use the GNU assembler syntax, and the Apple assembler is based on an old GNU version, so one might reasonably expect it to work. What I had not realised was just how old a version Apple based their assembler on. The version they chose was 1.38.1, released in January 1991, 18 years ago. Features which have since been added to the GNU assembler, and there are many, have not been merged by Apple. As a result, many special directives and macro features used in FFmpeg are not recognised by the Apple assembler, and modifying the code to work with this assembler would render it unusable with modern GNU versions.

Why not replace the assembler in the SDK with a GNU version, one might ask. The answer is that this is not possible. The Apple system uses an object file format, Mach-O, not supported by the GNU tools. The chances of Apple updating their assembler to support the newer syntax appear slim, so our best hope is for the GNU binutils package to gain support for the Mach-O format. This will need a lot of work, and a working version cannot be expected for yet some time.

While this incompatibility persists, those wishing to run an optimised FFmpeg build on their iPhone will have to rely on patches to make it palatable to the Apple assembler. Supporting the Apple syntax directly in FFmpeg is unfortunately not feasible.

Links

• http://code.google.com/p/ffmpeg4iphone/
• https://lists.ubuntu.com/archives/ubuntu-ca/2008-February/004207.html