Index

Building a continuous-integration Android build server on FreeBSD: Part one: building APKs using Linux emulation

November 2016.

In this series of blog posts, we're going to create a Android build server for continuous integration on FreeBSD.

Part one (this post) will explain how to build Android APKs using Gradle on FreeBSD using the Linux emulation.
Part two will explain how to configure Gitlab-CI to be able to run builds automatically for each commit.
Part three will explain how to configure Jenkins to be able to run builds and email the APKs to people.

I'll be using a normal 10.2-RELEASE:


# uname -a
FreeBSD androidbuild 10.2-RELEASE-p11 FreeBSD 10.2-RELEASE-p11 #0: Wed Jan 27 15:56:01 CET 2016     root@androidbuild:/usr/obj/usr/src/sys/GENERIC  amd64

Parts of this posts are duplicate from my old post on how to build APKs on FreeBSD.

Step 1: installing the system dependencies

We're going to install many packages, including a lot of GNU-style one. So if you care about having your systems cleans, I suggest your do all this in a jail.

Install gradle:


# make -C /usr/ports/devel/gradle/ install clean

Go drink the longest coffee you can brew. That stuff installs a lot of dependencies.

Install bash and link it to /bin/bash:


# make -C /usr/ports/shells/bash install clean
# ln -s /usr/local/bin/bash /bin/bash

Install git:


# make -C /usr/ports/devel/git install clean

Install python:


# make -C /usr/ports/lang/python install clean

Create a nice user to execute the builds:


# pw useradd androidbuild
# mkdir /home/androidbuild
# chown androidbuild:androidbuild /home/androidbuild

Load FreeBSD's Linux emulation system and install a base CentOS 6 system:


# kldload linux
# cd /usr/ports/emulators/linux_base-c6 && make install distclean

The latest version of the build tools (24.0.0) use 64bit binaries and libraries. If you want to be able to build your apks with them, you'll also need the 64bit Linux emulation. In that case, you must be running a FreeBSD version >= 10.3. In that case, you should do instead:


# kldload linux
# kldload linux64
echo "OVERRIDE_LINUX_BASE_PORT=c6_64" >> /etc/make.conf
echo "OVERRIDE_LINUX_NONBASE_PORTS=c6_64" >> /etc/make.conf
# cd /usr/ports/emulators/linux_base-c6 && make install distclean

If you alread had the 32 bit CentOS base, uninstall the port, remove the /compat/linux files and reinstall the port again after setting the two entries in make.conf.

Step 2: setting-up the Android SDK

Download the SDK

Fetch and extract the latest version of the Linux SDK:


% fetch 'https://dl.google.com/android/android-sdk_r24.4.1-linux.tgz'
% tar xzf android-sdk_r24.4.1-linux.tgz
% setenv ANDROID_HOME /home/androidbuild/android-sdk-linux/

Patch the SDK

As it is now, the SDK will download build tools for Windows, since it obviously won't recognize our FreeBSD.
Since we're going to use the Linux emulation, we need to path the system so that it downloads Linux binaries:

Download the source of the SDK base:


% git clone https://android.googlesource.com/platform/tools/base

Apply the following patch:


diff -r -u a/common/src/main/java/com/android/SdkConstants.java b/common/src/main/java/com/android/SdkConstants.java
--- a/common/src/main/java/com/android/SdkConstants.java        2016-09-06 07:56:56.325948102 +0000
+++ b/common/src/main/java/com/android/SdkConstants.java        2016-09-06 07:58:10.721944140 +0000
@@ -635,6 +635,8 @@
             return PLATFORM_WINDOWS;
         } else if (os.startsWith("Linux")) {                //$NON-NLS-1$
             return PLATFORM_LINUX;
+        } else if (os.startsWith("FreeBSD")) {              //$NON-NLS-1$
+            return PLATFORM_LINUX;
         }

         return PLATFORM_UNKNOWN;
diff -r -u a/sdklib/src/main/java/com/android/sdklib/internal/repository/archives/ArchFilter.java b/sdklib/src/main/java/com/android/sdklib/internal/repository/archives/ArchFilter.java
--- a/sdklib/src/main/java/com/android/sdklib/internal/repository/archives/ArchFilter.java      2016-09-06 07:56:56.828948347 +0000
+++ b/sdklib/src/main/java/com/android/sdklib/internal/repository/archives/ArchFilter.java      2016-09-06 07:58:35.160941890 +0000
@@ -216,6 +216,8 @@
             hostOS = HostOs.WINDOWS;
         } else if (os.startsWith("Linux")) {                //$NON-NLS-1$
             hostOS = HostOs.LINUX;
+        } else if (os.startsWith("FreeBSD")) {                //$NON-NLS-1$
+            hostOS = HostOs.LINUX;
         }

         BitSize jvmBits;

Rebuild the patched files.


% javac common/src/main/java/com/android/SdkConstants.java
% javac sdklib/src/main/java/com/android/sdklib/internal/repository/archives/ArchFilter.java -cp "sdklib/src/main/java:common/src/main/java:annotations/src/main/java"

Replace the files inside the jar.


% cd sdklib/src/main/java/ && jar uf ${ANDROID_HOME}/tools/lib/sdklib.jar com/android/sdklib/internal/repository/archives/ArchFilter.class
% cd common/src/main/java && jar uf ${ANDROID_HOME}/tools/lib/common.jar com/android/SdkConstants.class

See this patch on the Android website: https://android-review.googlesource.com/#/c/100271/

Download the SDK packages and set-up the build tools

Go to the tool directory:


% cd ${ANDROID_HOME}/tools/

In the example, we're going to build an APK thats uses API version 23.
Let's download the right packages:


% ./android list sdk -u -a
[...]
7- Android SDK Build-tools, revision 23.0.3
31- SDK Platform Android 6.0, API 23, revision 3
119- Google APIs, Android API 23, revision 1
% ./android update sdk -u -a -t tools,platform-tools,7,31,119

Let's find the Linux binaries in the build tools, and brand them as such:


% find build-tools/23.0.3/ -maxdepth 1 -type f -print0 | xargs -0 -n 10 file | grep "ELF"
build-tools/23.0.3/mipsel-linux-android-ld:  ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux 2.6.8, stripped
build-tools/23.0.3/arm-linux-androideabi-ld: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux 2.6.8, stripped
build-tools/23.0.3/llvm-rs-cc:               ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux 2.6.24, BuildID[sha1]=8a4ffbc0e197147c4e968722e995605e1d06ea88, not stripped
build-tools/23.0.3/i686-linux-android-ld:    ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux 2.6.8, stripped
build-tools/23.0.3/bcc_compat:               ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux 2.6.24, BuildID[sha1]=d565d03b7bafd03d335bdd246832bb31c7cca527, not stripped
build-tools/23.0.3/aapt:                     ELF 32-bit LSB shared object, Intel 80386, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux 2.6.24, BuildID[sha1]=cfb63b4ad11d0c2d59f10329f0116706e99bf72e, not stripped
build-tools/23.0.3/aidl:                     ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux 2.6.24, BuildID[sha1]=3cbd3d742040d61877a3c6544778bf4701b2f62d, not stripped
build-tools/23.0.3/aarch64-linux-android-ld: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux 2.6.8, stripped
build-tools/23.0.3/split-select:             ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux 2.6.24, BuildID[sha1]=b3bfb153d0ffaef6ed84c316ff682381ba8d75b2, not stripped
build-tools/23.0.3/dexdump:                  ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux 2.6.24, BuildID[sha1]=a678a8163a2483107d285ffdc94fdde0d4fb2178, not stripped
build-tools/23.0.3/zipalign:                 ELF 32-bit LSB shared object, Intel 80386, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux 2.6.24, BuildID[sha1]=7d64216663df9fd3b4048952d095fbd07cb4284f, not stripped

% find build-tools/24.0.3/ -maxdepth 1 -type f -print0 | xargs -0 -n 10 file | grep "ELF" | awk 'BEGIN { FS = ":" } ; {print $1}' | xargs brandelf -t Linux
% find build-tools/24.0.3/ -maxdepth 1 -type f -print0 | xargs -0 -n 10 file | grep "ELF" | awk 'BEGIN { FS = ":" } ; {print $1}' | xargs chmod +x

The SDK is now configured and we should be able to build our apps there.

Step 3: building the APK

Fetch the sources of your app:


% git clone 'https://git.example.org/android-app'
% cd android-app

Be sure to have set environment variable ANDROID_HOME to the SDK location.
Start gradle and see if it complains.


% gradle tasks
------------------------------------------------------------
All tasks runnable from root project
------------------------------------------------------------

Android tasks
-------------
androidDependencies - Displays the Android dependencies of the project.
signingReport - Displays the signing info for each variant.
sourceSets - Prints out all the source sets defined in this project.
[...]
To see all tasks and more detail, run gradle tasks --all

To see more detail about a task, run gradle help --task <task>

BUILD SUCCESSFUL

Total time: 14.839 secs

Gradle is happy, let's try a build:


% gradle assembleRelease
[...]
BUILD SUCCESSFUL

Total time: 2 mins 24.784 secs

Let's look at the build output:


% ls Truc/build/outputs/apk/
truc-app-release-unaligned.apk   truc-app-release.apk

Success!