Building CyanogenMod 9 from source for Galaxy S

Intro

This tutorial is intended to sum up and list all steps required to prepare development environment for building Cyanogenmod 9 (CM) system from sources. Build target is Samsung Galaxy S (SGS), but tips I give are easily used with other devices.
This tutorial was created as an answer to the problem I faced when willing to contribute to CM - although the system is very popular and plenty of pages give flash/setup/build hints, none of those that I found was complete. However, being able to bypass all obstacles, I decided to help the others with the task.

I do not focus on explaining basic definitions and actions here, rather discussing more advanced problems that can be found in the process.

Update: Originally, the post was written for Samsung Galaxy S. After a few months, I faced the need to repeat the process for Samsung Galaxy S2 (SGS2). I revised the whole post, finding some bugs, adding clarifications and even simplifying the process. The biggest update is in the case of downloading customization files from a working device. With the help of pts.blog I fixed tunneling adb USB interface from Windows host to Ubuntu guest in VirtualBox (and maybe the difference between VirtualBox 4.1.12 and 4.1.22 is that the tunneling was fixed "in the meantime").

Dev environment:

1. Windows 7 Professional 64-bit as host
2. Ubuntu 10.04.04 64-bit as guest
3. VirtualBox 4.1.22
4. Asus UL80VT

Download CM 9 image

Latest version is available here: nightly builds.

CM 9 system image (downloaded zip package) contains files that are specific to a device (Galaxy S in this case) and are not a part of official/CM Android repositories. These are, for example, modem binaries, ril (radio interface layer) binaries, gps configuration files, memory allocation binaries etc. (in general - proprietary binaries).

CM system images come as zip archives. All files that are required for build customization are embedded into the archive and can be extracted manually. How to locate them will be described later on.

Update: In the other post I explain an easy way of flashing stock I9100 (Galaxy S2) with CM 9.1.

Create Ubuntu VM

The next step requires installing VirtualBox on Windows host. After installation, create new virtual machine that is 64-bit Ubuntu. Create new virtual disk for Ubuntu. I always use VDI format which is dynamically allocated, with maximum size over 100 GB (however, fixed-size virtual disk might be faster for building). That should be more than enough for downloading Android sources and performing system compilation. But beware that complete build uses around 32 GB of virtual disk space, so you do have to have at least such an amount of free space on the partition where you keep the vdi (virtual disk) file.

Very important thing about the virtual machine is to use bridged networking instead of NAT for the Internet access. This allows to resolve problems with repo synchronization. Having two processor cores in Asus laptop, I allow Ubuntu to use both of them, giving it also 2GB of RAM.

The next step should be to install VirtualBox Guest Additions in Ubuntu.

Configure Ubuntu

Having started and updated  the Ubuntu virtual machine, the next step is to equip it with all required development packages. Please follow official android page for that: source.android.com. Apply all sections for 64-bit version of Ubuntu.

What you will not be able to do in spite to that manual is installing Oracle's Java on Ubuntu. Please follow steps presented on askubuntu.com to accomplish this task.

There will be one additional package to install on Ubuntu specifically for CM 9 (sudo apt-get install schedtool), but you will be warned about that during repo initialization if not done earlier.

Android SDK (adb and other tools) is not required to have on Ubuntu for platform development purposes.

Configure adb USB tunneling from Windows host to Ubuntu: adb.

Right now Ubuntu should be ready for repo initialization and source download - but not the build process.

Download the source code

In order to download the code one could follow official Cyanogenmod wiki for building platform from source (wiki.cyanogenmod.com). However, it is not up-to-date. Below are extracted steps that are applicable right now to your Ubuntu installation:

1. mkdir -p ~/bin
2. mkdir -p ~/android/system
1. that one will contain complete CM 9 platform code (without kernel code)
3. curl https://dl-ssl.google.com/dl/googlesource/git-repo/repo > ~/bin/repo
1.  repo is Google's python tool for managing local Android source code repository. 
4. chmod a+x ~/bin/repo
5. reboot Ubuntu
6. cd ~/android/system/
7. repo init -u git://github.com/CyanogenMod/android.git -b ics
1. ics stands for Ice Cream Sandwich branch, i.e. Android 4.0.*
2. at this point, especially in corporate networks :), you may stumble upon following error:
1. Get git://github.com/CyanogenMod/android.git
2. fatal: unable to connect to  github.com:
3. github.com[0: 207.97.227.239]: errno=Connection timed out
3. simply change git:// to http:// protocol, as git is usually blocked in such networks
8. repo sync -j 16
1. this is when bridged networking for Ubuntu comes in handy, as NAT-ed network caused synchronization problems
2. -j param tells max number of simultaneous downloads

Download will take up to few hours depending on your Internet connection speed. Use the same command to repeatedly update your local repository with newer changes from the developers.

Retrieve customization files

Right now you have to start CM 9 build process that will fail soon. This is done in order to retrieve from CM 9 git servers SGS pre-configuration scripts.

petrelli@petrelli-laptop:~$ cd ~/android/system/
petrelli@petrelli-laptop:~/android/system$ . build/envsetup.sh && brunch

Brunch script performs build initialization. However, you have to define build target (SGS). From the menu that will be displayed, choose galaxysmtd or any other target that you want. Right now, scripts will notice there is no local configuration folder for your device and will try to download it from the network. Directory:

~/android/system/device/samsung/galaxysmtd/

should get created. Connect your device to the computer. Run extract-files.sh. You should get something like:

petrelli@petrelli-laptop:~/android/system/device/samsung/galaxys2$ ./extract-files.sh 
98 KB/s (9868 bytes in 0.097s)
263 KB/s (45356 bytes in 0.167s)
515 KB/s (632304 bytes in 1.198s)
140 KB/s (13768 bytes in 0.095s)
101 KB/s (9620 bytes in 0.092s)
97 KB/s (9980 bytes in 0.099s) [...]

The script is intended to download all proprietary files from SGS into source folders.

Download prebuilts

Download Prebuilts needed by the build: 

~/android/system/vendor/cm/get-prebuilts

Build

Go back to main repository folder and type:

. build/envsetup.sh

in order to refresh the list of buildable devices. The type

lunch

and choose galaxysmtd (or appropriate). And at the end:

make -jN

where N stands for number of processes in which to run the compiler. Having multiple-core CPUs can significantly speed up compilation process (but not linking). For example, having dual-core CPU you can easily choose N=3.

Building should proceed right now. For the first time, it will take up to few hours depending on your machine. Successive builds should be faster, as intermediate files are created in the build process.
Update: In the updated source tree there is an urge to switch to building kernel from source. I will investigate that later on.

Problems

While building CM 9.1 for SGS2 I encountered following problem:

make: *** No rule to make target `out/target/product/galaxys2/obj/lib/libUMP.so', needed by `out/target/product/galaxys2/obj/SHARED_LIBRARIES/gralloc.galaxys2_intermediates/LINKED/gralloc.galaxys2.so'.  Stop.

The general problem is that extract-files.sh script that extracts proprietary binaries from the device into build folders copies EGL section into vendor/samsung/galaxys2-common/proprietary/ instead of vendor/samsung/galaxys2/proprietary/ folder. Thus fixing the EGL section in the script worked for me.

Grande finale

Install Google apps

Clean CM 9 build lacks Google apps like Google Play or Google Maps. Without Google Play you are not able to download any other apps, so it is desired to have that installed. Download latest binary from CyanogenMod and install it according to the instruction.