From source code to ndk-build using autotools and androgenizer

This post explains how to compile C source code for Android using the Native Development Kit (NDK) by using autotools to set up the building infrastructure and using androgenizer to convert that autotools infrastructure into Android.mk files understood by the ndk-build tool. I’ll first review some autotools concepts very quickly trough examples and iterate over them.

Simple program

(Download the testapp.tgz example or browse it online)

We start from a simple helloworld program with its main() function that depends on a sayhello() function. We want to compile it using autotools in the simplest possible way.

To create an autotools template from scratch, follow the instructions of this presentation, which are abridged here:

  autoscan
  mv configure.scan configure.ac
  # Add this line below the AC_INIT line:
  # AM_INIT_AUTOMAKE
  nano configure.ac
  autoheader
  aclocal
  touch NEWS README AUTHORS ChangeLog
  automake --add-missing --copy
  autoconf

Makefile.am should contain:

  bin_PROGRAMS = testapp
  testapp_SOURCES = testapp.c

Then, to compile, just:

  ./configure
  make

The testapp.tgz file is an example of this. Look at the “autoall” script.

Library + program using libtool

(Download the testlib.tgz example or browse it online)

In this case we have to add a new libtool line to configure.ac, apart from the automake one, under the AC_INIT line:

  AM_INIT_AUTOMAKE
  AC_PROG_LIBTOOL

This is how Makefile.am should look like:

  lib_LTLIBRARIES = libtestlib.la
  libtestlib_la_HEADERS = testlib.h
  libtestlib_la_SOURCES = testlib.c
  libtestlib_ladir = $(includedir)
  libtestlib_la_LDFLAGS = -avoid-version

  bin_PROGRAMS = testapp
  testapp_SOURCES = testapp.c
  testapp_LDADD = .libs/libtestlib.so
  testappdir = $(includedir)

The LDFLAGS are the flags passed to libtool, and are documented here and here in the Autobook reference. In this case, the “-avoid-version” forces generation of libtestlib.so instead of libtestlib.so.0.0.0 (versioned libs aren’t supported in Android).

UPDATE: This “-avoid-version” flag isn’t needed anymore in recent versions of the NDK. They will generate unversioned libraries automatically and won’t recognize the flag (will show an error).

Look at the “autoall” script in the example for all the details. Now it’s fully automatic, so you don’t have to do the insertions by hand in configure.ac.

Androgenized lib + program

(Download the androgenized-testapp.tgz example or browse it online)

All the C code must reside in a directory called “jni”, inside the main directory of the Android project we want to create (in the androgenized-testapp example only the jni is included, technically the other ones aren’t needed):

androgenized-testapp
  src (Java source code)
  libs (compiled binary libs and executables)
  obj (intermediate object code for libs and exes)
  jni
    Application.mk (actually not mandatory)
    Android.mk (generated by androgenizer, drives the compilation when using NDK)
    Makefile.am
    testlib.c
    ...

A new target called Android.mk must be created in Makefile.am:

Android.mk: Makefile.am
        androgenizer -:PROJECT testlib 

        -:REL_TOP $(top_srcdir) -:ABS_TOP $(abs_top_srcdir) 
        -:SHARED testlib 
        -:SOURCES $(libtestlib_la_SOURCES) 
        -:LDFLAGS $(libtestlib_la_LDFLAGS) 

        -:PROJECT testapp 
        -:REL_TOP $(top_srcdir) -:ABS_TOP $(abs_top_srcdir) 
        -:EXECUTABLE testapp 
        -:LDFLAGS -ltestlib 
        -:SOURCES $(testapp_SOURCES) 
> $@

Androgenizer must be downloaded and installed somewhere in our PATH from here. It has a very brief parameter documentation, and the best way to understand it is by seeing usage examples, such as all the androgenizer lines added to the Makefile.am in the GStreamer project.

In the Android.mk target shown above two modules are going to be compiled: “testlib” and “testapp”. Each module is declared with a “-:PROJECT” parameter. The rest of the lines belong to the current project (current module) until a new one is declared. I’ve left a separation between projects to illustrate this.

Next line should always be a “-:REL_TOP” and “-:ABS_TOP”, as recommended in README.txt. These two values are used to check all the paths in LDFLAGS, CFLAGS, etc. and substitute local paths with absolute paths. This is important, because normal makefiles usually assume that the rules are executed from the subdirectory the makefile is in, while what is considered as the working directory for Android.mk makefiles is always the directory from which ndk-build is invoked. As you can imagine, compiler directives such as “-I..” are going to have a very different meaning and cause the wrong behaviour when passed to the compiler by ndk-build.

After that, we indicate the type of target for the project (“-:STATIC” for libtestlib.a, “-:SHARED” for libtestlib.so, and “-:EXECUTABLE” for testapp). The extension of the library/app is automatically calculated.

The “-:SOURCES” parameter is used to indicate the files to be compiled. In this case we can use the list of files already calculated by the previous automake rules (see testlib.tgz and testapp.tgz examples) and stored in the “libtestlib_la_SOURCES” variable.

The “-:LDFLAGS” parameter is used to generate the needed library dependencies for the current module. In the case of testlib, those dependencies are already calculated by automake in the “libtestlib_la_LDFLAGS”. In the case of testapp, it depends on testlib, so we indicate it manually with “-ltestlib”. That will translate into this line in the generated Android.mk file:

  LOCAL_SHARED_LIBRARIES:=libtestlib

this will make the NDK build scripts to link against the proper “libtestlib.so” (if the lib was dynamic) or “libtestlib.a” (if it was static). The NDK already knows how testlib was built and how it has to be linked.

The final “> $@” line just means that make has to take the output of the androgenizer command and dump it to a file named just like the target it’s being build, that is, “Android.mk”.

After all the autoscan, configure.ac customization, autoheader, aclocal, libtoolize, automake and autoconf steps we can “./configure” and “make Android.mk” to generate the “Android.mk” file. Note that no full “make” is needed. Then we step back one directory (“cd ..” to the project root) and perform the actual build using NDK (which has to be downloaded from here, installed and be in our PATH; the version I used to write this post was NDK r8):

  ndk-build V=1

The V=1 option just prints out all the executed commands, which is very handy. The compilation result will be placed in the “libs” directory.

To understand the process a bit more, it’s a good idea to look into the generated Android.mk file:

  LOCAL_PATH:=$(call my-dir)
  include $(CLEAR_VARS)

  LOCAL_MODULE:=testlib
  LOCAL_SRC_FILES := 
          testlib.c
  LOCAL_LDFLAGS:=
          -avoid-version
  LOCAL_PRELINK_MODULE := false
  include $(BUILD_SHARED_LIBRARY)
  include $(CLEAR_VARS)

  LOCAL_MODULE:=testapp
  LOCAL_SRC_FILES := 
          testapp.c
  LOCAL_SHARED_LIBRARIES:=
          libtestlib
  LOCAL_PRELINK_MODULE := false
  include $(BUILD_EXECUTABLE)

Apart from some boilerplate (LOCAL_PATH, CLEAR_VARS, LOCAL_PRELINK_MODULE), we can distinguish the two modules (LOCAL_MODULE), its source files (LOCAL_SRC_FILES), inherited LDFLAGS (LOCAL_LDFLAGS), required libs (LOCAL_SHARED_LIBRARIES) and finally, the building instruction (BUILD_SHARED_LIBRARY, BUILD_EXECUTABLE).

Seeing the Application.mk and Android.mk supported directives are documented somewhere in your installation of the NDK. They support more options than the ones explained here. With this basic introductions, that documentation should be more understandable. Application.mk can be edited directly. Android.mk has to be customized using the “-:PASSTHROUGH” androgenizer parameters in Makefile.am. For example:

  -:PASSTHROUGH LOCAL_ARM_MODE:=arm

Unfortunately, real life autotools build scripts are a bit more complex than the examples shown here. Here are some debug tips that can be helpful:

• Use “ndk-build V=1” to get the exact command line used to compile each file. When something fails, execute that command line independently and examine all the flags passed to it trying to look for incorrect compiler flags or missing include paths. Try to figure out what the correct command line flags would be.
• Locate where the offending flags come from in the Android.mk. From there, locate them in the Androgenizer Makefile.am target. Usually some Makefile.am variable has the wrong values. Need to debug more on how Androgenizer is called? Go to the desired directory, delete Android.mk, make Android.mk and see the executed command line.
• Where the Makefile.am variables comes from? What Makefile.am variables should be used? Examine the generated Makefile to find the values of all possible Makefile.am variables. That’s the place to look to decide what to pass to Androgenizer and what not.