Android System Development

Android System
Development

Embedded Linux
Experts

free electrons
Copyright 2004-2016, Free Electrons.
Creative Commons BY-SA 3.0 license.
Latest update: February 18, 2016.
Document updates and sources:
http://free- electrons.com/doc/training/android
Corrections, suggestions, contributions and translations are welcome!
Send them to feedback@free-electrons.com

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

1/443

Rights to copy
Copyright 2004-2016, Free Electrons
License: Creative Commons Attribution - Share Alike 3.0
http://creativecommons.org/licenses/by-sa/3.0/legalcode
You are free:
to copy, distribute, display, and perform the work
to make derivative works
to make commercial use of the work
Under the following conditions:
Attribution. You must give the original author credit.
Share Alike. If you alter, transform, or build upon this work, you may distribute
the resulting work only under a license identical to this one.
For any reuse or distribution, you must make clear to others the license terms of
this work.
Any of these conditions can be waived if you get permission from the copyright
holder.
Your fair use and other rights are in no way aected by the above.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

2/443

Hyperlinks in the document

There are many hyperlinks in the document

Regular hyperlinks:
http://kernel.org/

Kernel documentation links:

Documentation/kmemcheck.txt

Links to kernel source les and directories:

drivers/input
include/linux/fb.h

Links to the declarations, denitions and instances of kernel

symbols (functions, types, data, structures):
platform_get_irq()
GFP_KERNEL
struct file_operations

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

3/443

Free Electrons at a glance

Engineering company created in 2004

(not a training company!)

Locations: Orange, Toulouse, Lyon (France)

Serving customers all around the world

See http://free-electrons.com/company/customers/

Head count: 9
Only Free Software enthusiasts!

Focus: Embedded Linux, Linux kernel, Android Free Software

/ Open Source for embedded and real-time systems.

Activities: development, training, consulting, technical

support.

Added value: get the best of the user and development

community and the resources it oers.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

4/443

Free Electrons on-line resources

All our training materials:

http://free-electrons.com/docs/
Technical blog:
http://free-electrons.com/blog/
Quarterly newsletter:
http://lists.freeelectrons.com/mailman/listinfo/newsletter
News and discussions (Google +):
https://plus.google.com/+FreeElectronsDevelopers
News and discussions (LinkedIn):
http://linkedin.com/groups/Free-Electrons-4501089
Quick news (Twitter):
http://twitter.com/free_electrons
Linux Cross Reference - browse Linux kernel sources on-line:
http://lxr.free-electrons.com

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

5/443

Generic course information

Generic course
information

Embedded Linux
Experts

free electrons
Copyright 2004-2016, Free Electrons.
Creative Commons BY-SA 3.0 license.
Corrections, suggestions, contributions and translations are welcome!

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

6/443

Hardware used in this training session

BeagleBone Black, from CircuitCo
Texas Instruments AM335x (ARM Cortex-A8)
Powerful CPU, with 3D acceleration,

additional processors (PRUs) and lots of

peripherals.
512 MB of RAM
2 GB of on-board eMMC storage

(4 GB in Rev C)
USB host and USB device ports
microSD slot
HDMI port
2 x 46 pins headers, with access to many

expansion buses (I2C, SPI, UART and more)

A huge number of expansion boards, called

capes. See http://beagleboardtoys.com/.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

7/443

Do not damage your BeagleBone Black!

Do not remove power abruptly:

Boards components have been damaged by removing the

power or USB cable in an abrupt way, not leaving the PMIC
the time to switch o the components in a clean way. See
http://bit.ly/1FWHNZi
Reboot (reboot) or shutdown (halt) the board in software
when Linux is running.
You can also press the RESET button to reset and reboot.
When there is no software way, you can also switch o the
board by pressing the POWER button for 8 seconds.

Do not leave your board powered on a metallic surface (like a

laptop with a metal nish).

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

8/443

Course outline - Day 1

Building Android

Introduction to Android

Getting Android sources

Building and booting Android

Introduction to the Linux kernel

Compiling and booting the Linux kernel

Labs: download Android sources, compile them and boot them

with the Android emulator. Recompile the Linux kernel.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

9/443

Course outline - Day 2

Android kernel, boot and lesystem details

Android changes to the Linux kernel

Android bootloaders

Booting Android

Using ADB

Android lesystem

Labs: customize, compile and boot Android for the BeagleBone

Black board.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

10/443

Course outline - Day 3

Supporting a new product and customizing it

Android build system. Add a new module and product.

Android native layer - Bionic, Toolbox, init, various daemons,

Dalvik, hardware abstraction, JNI...

Labs: Use ADB, create a new product, customize the product for
the BeagleBone Black board.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

11/443

Course outline - Day 4

Android framework and applications

Android framework for applications

Introduction to application development

Android packages

Advise and resources

Labs: compile an external library and a native application to

control a USB missile launcher. Create a JNI library and develop
an Android application to control the device.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

12/443

Participate!
During the lectures...

Don't hesitate to ask questions. Other people in the audience

may have similar questions too.

This helps the trainer to detect any explanation that wasn't

clear or detailed enough.

Don't hesitate to share your experience, for example to

compare Linux / Android with other operating systems used
in your company.

Your point of view is most valuable, because it can be similar

to your colleagues' and dierent from the trainer's.

Your participation can make our session more interactive and

make the topics easier to learn.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

13/443

Practical lab guidelines

During practical labs...

We cannot support more than 8 workstations at once (each

with its board and equipment). Having more would make the
whole class progress slower, compromising the coverage of the
whole training agenda (exception for public sessions: up to 10
people).

So, if you are more than 8 participants, please form up to 8

working groups.

Open the electronic copy of your lecture materials, and use it

throughout the practical labs to nd the slides you need again.

Don't hesitate to copy and paste commands from the PDF

slides and labs.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

14/443

Advise: write down your commands!

During practical labs, write down all your commands in a text le.

You can save a lot of time re-using

commands in later labs.

This helps to replay your work if

you make signicant mistakes.

You build a reference to remember

commands in the long run.
That's particular useful to keep
kernel command line settings that
you used earlier.
Also useful to get help from the
instructor, showing the commands
that you run.

Lab commands
Cross-compiling kernel:
export ARCH=arm
export CROSS_COMPILE=arm-linuxmake sama5_defcong
Booting kernel through tftp:
setenv bootargs console=ttyS0 root=/dev/nfs
setenv bootcmd tftp 0x21000000 zImage; tftp
0x22000000 dtb; bootz 0x21000000 - 0x2200...
Making ubifs images:
mkfs.ubifs -d rootfs -o root.ubifs -e 124KiB
-m 2048 -c 1024
Encountered issues:
Restart NFS server after editing /etc/exports!

gedit ~/lab-history.txt
free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

15/443

Cooperate!

As in the Free Software and Open Source community, cooperation

during practical labs is valuable in this training session:

If you complete your labs before other people, don't hesitate

to help other people and investigate the issues they face. The
faster we progress as a group, the more time we have to
explore extra topics.

Explain what you understood to other participants when

needed. It also helps to consolidate your knowledge.

Don't hesitate to report potential bugs to your instructor.

Don't hesitate to look for solutions on the Internet as well.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

16/443

Command memento sheet

This memento sheet gives

command examples for the most
typical needs (looking for les,
extracting a tar archive...)

It saves us 1 day of UNIX / Linux

command line training.

Our best tip: in the command line

shell, always hit the Tab key to
complete command names and le
paths. This avoids 95% of typing
mistakes.

Get an electronic copy on

http://free-electrons.com/
doc/training/embeddedlinux/command_memento.pdf

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

17/443

vi basic commands

The vi editor is very useful to

make quick changes to les in an
embedded target.

Though not very user friendly at

rst, vi is very powerful and its
main 15 commands are easy to
learn and are sucient for 99% of
everyone's needs!

Get an electronic copy on

http://free-electrons.com/
doc/training/embeddedlinux/vi_memento.pdf

You can also take the quick tutorial

by running vimtutor. This is a
worthy investment!

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

18/443

Practical lab - Training Setup

Prepare your lab environment

Download the lab archive

Enforce correct permissions

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

19/443

Introduction to Android

Introduction to
Android

Embedded Linux
Experts

free electrons
Copyright 2004-2016, Free Electrons.
Creative Commons BY-SA 3.0 license.
Corrections, suggestions, contributions and translations are welcome!

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

20/443

Introduction to Android

Features

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

21/443

Features

All you can expect from a modern mobile OS:

Application ecosystem, allowing to easily add and remove

applications and publish new features across the entire system
Support for all the web technologies, with a browser built on
top of the well-established Blink rendering engine
Support for hardware accelerated graphics through OpenGL ES
Support for all the common wireless mechanisms: GSM,
CDMA, UMTS, LTE, Bluetooth, WiFi, NFC.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

22/443

Introduction to Android

History

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

23/443

Early Years

Began as a start-up in Palo Alto, CA, USA in 2003

Focused from the start on software for mobile devices
Very secretive at the time, even though founders achieved a
lot in the targeted area before founding it
Finally bought by Google in 2005
Andy Rubin, founder of Android, Inc was also CEO of Danger,
Inc, a company producing one of the early smartphones, the
Sidekick

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

24/443

Opening Up

Google announced the Open Handset Alliance in 2007, a

consortium of major actors in the mobile area built around
Android

Hardware vendors: Intel, Texas Instruments, Qualcomm,

Nvidia, etc.
Software companies: Google, eBay, etc.
Hardware manufacturers: Motorola, HTC, Sony Ericsson,
Samsung, etc.
Mobile operators: T-Mobile, Telefonica, Vodafone, etc.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

25/443

Android Open Source Project (AOSP)

At every new version, Google releases its source code through

this project so that community and vendors can work with it.

One major exception: Honeycomb has not been released

because Google stated that its source code was not clean
enough to release it.

One can fetch the source code and contribute to it, even
though the development process is very locked by Google

Only a few devices are supported through AOSP though, only

the two most recent Android development phones and tablets
(part of the Nexus brand) and the pandaboard

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

26/443

Android Releases

Each new version is given a dessert name

Released in alphabetical order

Latest releases:

Android
Android
Android
Android
Android

2.3 Gingerbread
3.X Honeycomb
4.0 Ice Cream Sandwich
4.1/4.2/4.3 Jelly Bean
4.4 KitKat

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

27/443

Android Versions

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

28/443

Introduction to Android

Architecture

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

29/443

Architecture

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

30/443

The Linux Kernel

Used as the foundation of the Android system

Numerous additions from the stock Linux, including new IPC

(Inter-Process Communication) mechanisms, alternative
power management mechanism, new drivers and various
additions across the kernel

These changes are beginning to go into the staging/ area of

the kernel, as of 3.3, after being a complete fork for a long
time

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

31/443

Android Libraries

Gather a lot of Android-specic libraries to interact at a

low-level with the system, but third-parties libraries as well

Bionic is the C library, SurfaceManager is used for drawing

surfaces on the screen, etc.

But also Blink, SQLite, OpenSSL coming from the free

software world

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

32/443

Android Runtime

Handles the execution of Android applications

Almost entirely written from scratch by Google

Contains Dalvik, the virtual machine that executes every

application that you run on Android, and the core library for
the Java runtime, coming from Apache Harmony project

Also contains system daemons, init executable, basic binaries,

etc.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

33/443

Android Framework

Provides an API for developers to create applications

Exposes all the needed subsystems by providing an abstraction

Allows to easily use databases, create services, expose data to

other applications, receive system events, etc.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

34/443

Android Applications

AOSP also comes with a set of applications such as the phone

application, a browser, a contact management application, an
email client, etc.

However, the Google apps and the Android Market app aren't
free software, so they are not available in AOSP. To obtain
them, you must contact Google and pass a compatibility test.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

35/443

Introduction to Android

Hardware Requirements for Android

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

36/443

Android Hardware Requirements

Google produces a document updated every new Android

version called the Compatibility Denition Document (CDD).

This document provides all the information you need on the

expectations Google have about what should be an Android
device

It details both the hardware and the global behaviour of the

system.

While nothing forces you to follow that document if you don't

care about the Google applications, it usually gives a good
idea of the current hardware requirements.

We'll be detailing the requirements for KitKat

http://source.android.com/compatibility/androidcdd.pdf

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

37/443

SoC requirements

Since Android in itself is quite huge, the hardware required is

quite powerful.
Unlike Linux, Android ocially supports only a few
architectures

ARM v7a (basically, all the SoCs based on the Cortex-A CPUs)
x86
MIPS

You also need to have a powerful enough GPU with OpenGL

ES support. Latest versions of Android require the 3D
hardware acceleration

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

38/443

Storage and RAM needed

The required RAM size is also quite huge, 340MB are required
for the kernel and user space memory

Required storage is quite huge as well. An image of the

system is around 200-300MB, and you must have 350MB of
data space for the user plus 1GB of shared storage for the
applications.

This is the minimum, and Google actually strongly suggest to

have at least 2GB dedicated to the applications in order to be
able to upgrade to a later version

Google recommends to use block devices for storage and not

ash devices.

The shared space has to be accessible from a host computer

by some way, like NFS, USB Mass Storage, MTP, etc.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

39/443

External Peripherals 1/2

No form of communication supported is mandatory, but you

need at least one form of data networking with a throughput
of at least 200 kbit per second.

You will also need obviously a rather large screen with a

pointer device, presumably a touchscreen.

Screens supported must have a screen size of at least 2.5

inches, with a minimal resolution of 426x320, with a ratio
between 4:3 and 16:9 and with a color depth of at least 16bits.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

40/443

External Peripherals 2/2

Sensors are not mandatory, but depending of the class of

sensors, they are:

Recommended

Optional

Accelerometer
Magnetometer
GPS
Gyroscope
Barometer
Photometer
Proximity Sensor

Optional but discouraged

Thermometer

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

41/443

Unusual Android Devices: Nook E-Book Reader

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

42/443

Unusual Android Devices: Portable Console

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

43/443

Unusual Android Devices: Microwave Oven

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

44/443

Unusual Android Devices: Treadmill

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

45/443

When to choose Android

All of the requirements listed above are only if you want to be

eligible to the Android Play Store

If you don't want to get the store, you can obviously ignore
these
However, Android really makes sense in a system that has at
least:

A large screen
A powerful SoC, with several CPUs, plenty of RAM and
storage space (around 2GB) and a decent GPU

This is not an advisable choice when you want to build a

headless system, or a cheap system with limited resources

In this case, a regular Linux system is denitely more

appropriate. It will save you engineering costs, reduce the
price of your hardware, and bring the same set of features you
could expect from a headless Android

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

46/443

Practical lab - Android Source Code

Install all the development

packages needed to fetch and
compile Android

Download the repo utility

Use repo to download the source

code for Android and for all its
components

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

47/443

Android Source Code and Compilation

Android Source
Code and
Compilation

Embedded Linux
Experts

free electrons
Copyright 2004-2016, Free Electrons.
Creative Commons BY-SA 3.0 license.
Corrections, suggestions, contributions and translations are welcome!

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

48/443

Android Source Code and Compilation

How to get the source code

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

49/443

Source Code Location

The AOSP project is available at

http://source.android.com

On this site, along with the code, you will nd some resources
such as technical details, how to setup a machine to build
Android, etc.

The source code is split into several Git repositories for

version control. But as there is a lot of source code, a single
Git repository would have been really slow

Google split the source code into a one Git repository per
component

You can easily browse these git repositories using

https://code.google.com/p/android-sourcebrowsing/source/browse/

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

50/443

Source code licenses

Mostly two kind of licenses:

GPL/LGPL Code: Linux

Apache/BSD: All the rest
In the external folder, it depends on the component

While you might expect Google's apps for Android, like the
Android Market (now called Google Play Store), to be in the
AOSP as well, these are actually proprietary and you need to
be approved by Google to get them.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

51/443

Repo

This makes hundreds of Git repositories

To avoid making it too painful, Google also created a tool:

repo

Repo aggregates these Git repositories into a single folder

from a manifest le describing how to nd these and how to
put them together

Also aggregates some common Git commands such as diff or

status that are run across all the Git repositories

You can also execute a shell command in each repository

managed by Repo using the repo forall command

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

52/443

Repo's manifest

repo relies on a git repository that will contain XML les

called manifests

These manifests gives the information about where to

download some source code and where to store it. It can also
provide some additional and optional information such as a
revision to use, an alternative server to download from, etc.

The main manifests are stored in this git repo, and are shared
between all the users, but you can add some local manifests.

repo will also use any XML le that is under

.repo/local_manifests

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

53/443

Manifests syntax

<?xml version="1.0" encoding="UTF-8"?>

<manifest>
<remote name="github"
fetch="https://github.com/" />
<default remote="github" />
<project name="foo/bar" path="device/foo/bar" revision="v14.42" />
<remove-project name="foo/bar" />
</manifest>

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

54/443

Android Source Code and Compilation

Source code organization

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

55/443

Source Code organization 1/3

Once the source code is downloaded, you will nd several

folders in it
bionic/ is where Android's standard C library is stored

bootable/ contains code samples regarding the boot of an

Android device. In this folder, you will nd the
protocol used by all Android bootloaders and a
recovery image
build/ holds the core components of the build system
cts/ The Compatibility Test Suite
dalvik/ contains the source code of the Dalvik virtual
machine

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

56/443

Source Code Organization 2/3

development/ holds the development tools, debug applications,

API samples, etc
device/ contains the device-specic components
docs/ contains HTML documentation hosted at
http://source.android.com
external/ is one of the largest folders in the source code, it
contains all the external projects used in the Android
code
frameworks/ holds the source code of the various parts of the
framework
hardware/ contains all the hardware abstraction layers

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

57/443

Source Code Organization 3/3

libcore/ is the Java core library
libnativehelper/ contains a few JNI helpers for the Android base
classes
ndk/ is the place where you will nd the Native
Development Kit, which allows to build native
applications for Android
packages/ contains the standard Android applications
prebuilt/ holds all the prebuilt binaries, most notably the
toolchains
sdk/ is where you will nd the Software Development Kit
system/ contains all the basic pieces of the Android system:
init, shell, the volume manager, etc.

You can get a more precise description at

http://elinux.org/Master-android

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

58/443

Android Source Code and Compilation

Compilation

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

59/443

Android Compilation Process

Android's build system relies on the well-tried GNU/Make

software

Android is using a ``product'' notion which corresponds to

the specications of a shipping product, i.e. crespo for the
Google Nexus S vs crespo4g for the Sprint's Nexus S with
LTE support

To start using the build system, you need to include the le

build/envsetup.sh that denes some useful macros for
Android development or sets the PATH variable to include the
Android-specic commands

source build/envsetup.sh

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

60/443

Prepare the process

Now, we can get a list of all the products available and select
them with the lunch command

lunch will also ask for a build variant, to choose between eng,
user and userdebug, which corresponds to which kind of
build we want, and which packages it will add

You can also select variants by passing directly the combo

product-variant as argument to lunch

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

61/443

Compilation

You can now start the compilation just by running make

This will run a full build for the currently selected product

There are many other build commands:

make <package> Builds only the package, instead of going
through the entire build
make clean Cleans all the les generated by previous
compilations
make clean-<package> Removes all the les generated by
the compilation of the given package
mm Builds all the modules in the current directory
mmm <directory> builds all the modules in the given
directory

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

62/443

Android Source Code and Compilation

Contribute

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

63/443

Gerrit

For the Android development process, Google also developed

a tool to manage projects and ease code reviews.

It once again uses Git to do so and Repo is also built around

it so that you can easily contribute to Android

To do so, start a new branch with repo start <branchname>

Do your usual commits with Git

When you are done, upload to Gerrit using repo upload

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

64/443

Practical lab - First Compilation

Congure which system to build

Android for

Compile your rst Android root

lesystem

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

65/443

Linux kernel introduction

Linux kernel
introduction

Embedded Linux
Experts

free electrons
Copyright 2004-2016, Free Electrons.
Creative Commons BY-SA 3.0 license.
Corrections, suggestions, contributions and translations are welcome!

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

66/443

Linux kernel introduction

Linux features

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

67/443

History

The Linux kernel is one component of a system, which also

requires libraries and applications to provide features to end
users.
The Linux kernel was created as a hobby in 1991 by a Finnish
student, Linus Torvalds.

Linux quickly started to be used as the kernel for free software

operating systems

Linus Torvalds has been able to create a large and dynamic

developer and user community around Linux.

Nowadays, more than one thousand people contribute to each

kernel release, individuals or companies big and small.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

68/443

Linux kernel key features

Portability and hardware

support. Runs on most
architectures.

Security. It can't hide its

aws. Its code is reviewed
by many experts.

Scalability. Can run on

super computers as well as
on tiny devices (4 MB of
RAM is enough).

Stability and reliability.

Modularity. Can include

only what a system needs
even at run time.

Compliance to standards
and interoperability.

Exhaustive networking
support.

Easy to program. You can

learn from existing code.
Many useful resources on
the net.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

69/443

Linux kernel in the system

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

70/443

Linux kernel main roles

Manage all the hardware resources: CPU, memory, I/O.

Provide a set of portable, architecture and hardware

independent APIs to allow user space applications and
libraries to use the hardware resources.
Handle concurrent accesses and usage of hardware
resources from dierent applications.

Example: a single network interface is used by multiple user

space applications through various network connections. The
kernel is responsible to ``multiplex'' the hardware resource.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

71/443

System calls

The main interface between the kernel and user space is the
set of system calls
About 300 system calls that provide the main kernel services

File and device operations, networking operations,

inter-process communication, process management, memory
mapping, timers, threads, synchronization primitives, etc.

This interface is stable over time: only new system calls can
be added by the kernel developers

This system call interface is wrapped by the C library, and

user space applications usually never make a system call
directly but rather use the corresponding C library function

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

72/443

Pseudo lesystems

Linux makes system and kernel information available in user

space through pseudo lesystems, sometimes also called
virtual lesystems

Pseudo lesystems allow applications to see directories and

les that do not exist on any real storage: they are created
and updated on the y by the kernel
The two most important pseudo lesystems are

proc, usually mounted on /proc:

Operating system related information (processes, memory
management parameters...)
sysfs, usually mounted on /sys:
Representation of the system as a set of devices and buses.
Information about these devices.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

73/443

Inside the Linux kernel

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

74/443

Linux license

The whole Linux sources are Free Software released under the
GNU General Public License version 2 (GPL v2).
For the Linux kernel, this basically implies that:

When you receive or buy a device with Linux on it, you should
receive the Linux sources, with the right to study, modify and
redistribute them.
When you produce Linux based devices, you must release the
sources to the recipient, with the same rights, with no
restriction.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

75/443

Supported hardware architectures

See the arch/ directory in the kernel sources

Minimum: 32 bit processors, with or without MMU, and gcc

support

32 bit architectures (arch/ subdirectories)

Examples: arm, avr32, blackfin, c6x, m68k, microblaze,
mips, score, sparc, um

64 bit architectures:
Examples: alpha, arm64, ia64, tile

32/64 bit architectures

Examples: powerpc, x86, sh, sparc

Find details in kernel sources: arch/<arch>/Kconfig,

arch/<arch>/README, or Documentation/<arch>/

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

76/443

Linux kernel introduction

Linux versioning scheme and

development process

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

77/443

Until 2.6 (1)

One stable major branch every 2 or 3 years

One development branch to integrate new functionalities and

major changes

Identied by an even middle number

Examples: 1.0.x, 2.0.x, 2.2.x, 2.4.x

Identied by an odd middle number

Examples: 2.1.x, 2.3.x, 2.5.x
After some time, a development version becomes the new base
version for the stable branch

Minor releases once in while: 2.2.23, 2.5.12, etc.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

78/443

Until 2.6 (2)

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

79/443

Changes since Linux 2.6

Since 2.6.0, kernel developers have been able to introduce

lots of new features one by one on a steady pace, without
having to make disruptive changes to existing subsystems.

Since then, there has been no need to create a new

development branch massively breaking compatibility with the
stable branch.

Thanks to this, more features are released to users at a

faster pace.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

80/443

Versions since 2.6.0

From 2003 to 2011, the ocial kernel versions were named

2.6.x.

Linux 3.0 was released in July 2011

Linux 4.0 was released in April 2015

This is only a change to the numbering scheme

Ocial kernel versions are now named x.y

(3.0, 3.1, 3.2, ..., 3.19, 4.0, 4.1, etc.)
Stabilized versions are named x.y.z (3.0.2, 4.2.7, etc.)
It eectively only removes a digit compared to the previous
numbering scheme

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

81/443

New development model

Using merge and bug xing windows

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

82/443

New development model - Details

After the release of a 4.x version (for example), a two-weeks

merge window opens, during which major additions are
merged.

The merge window is closed by the release of test version

4.(x+1)-rc1

The bug xing period opens, for 6 to 10 weeks.

At regular intervals during the bug xing period, 4.(x+1)-rcY

test versions are released.

When considered suciently stable, kernel 4.(x+1) is

released, and the process starts again.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

83/443

More stability for the kernel source tree

Issue: bug and security xes only released for

most recent stable kernel versions.

Some people need to have a recent kernel,

but with long term support for security
updates.

You could get long term support from a

commercial embedded Linux provider.

You could reuse sources for the kernel used in

Ubuntu Long Term Support releases (5 years
of free security updates).

The http://kernel.org front page shows

which versions will be supported for some
time (up to 2 or 3 years), and which ones
won't be supported any more ("EOL: End Of
Life")

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

84/443

What's new in each Linux release?

The ocial list of changes for each Linux release is just a

huge list of individual patches!
commit aa6e52a35d388e730f4df0ec2ec48294590cc459
Author: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Date: Wed Jul 13 11:29:17 2011 +0200
at91: at91-ohci: support overcurrent notification
Several USB power switches (AIC1526 or MIC2026) have a digital output
that is used to notify that an overcurrent situation is taking
place. This digital outputs are typically connected to GPIO inputs of
the processor and can be used to be notified of these overcurrent
situations.
Therefore, we add a new overcurrent_pin[] array in the at91_usbh_data
structure so that boards can tell the AT91 OHCI driver which pins are
used for the overcurrent notification, and an overcurrent_supported
boolean to tell the driver whether overcurrent is supported or not.
The code has been largely borrowed from ohci-da8xx.c and
ohci-s3c2410.c.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Nicolas Ferre <nicolas.ferre@atmel.com>

Very dicult to nd out the key changes and to get the global
picture out of individual changes.

Fortunately, there are some useful resources available

http://wiki.kernelnewbies.org/LinuxChanges (4.2 and

4.3 are missing)
http://lwn.net
http://linuxfr.org, for French readers

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

85/443

Linux kernel introduction

Kernel conguration

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

86/443

Kernel conguration and build system

The kernel conguration and build system is based on

multiple Makeles

One only interacts with the main Makefile, present at the

top directory of the kernel source tree
Interaction takes place

using the make tool, which parses the Makele

through various targets, dening which action should be done
(conguration, compilation, installation, etc.). Run make help
to see all available targets.

Example

cd linux-3.6.x/
make <target>

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

87/443

Kernel conguration (1)

The kernel contains thousands of device drivers, lesystem

drivers, network protocols and other congurable items

Thousands of options are available, that are used to

selectively compile parts of the kernel source code

The kernel conguration is the process of dening the set of

options with which you want your kernel to be compiled
The set of options depends

On your hardware (for device drivers, etc.)

On the capabilities you would like to give to your kernel
(network capabilities, lesystems, real-time, etc.)

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

88/443

Kernel conguration (2)

The conguration is stored in the .config le at the root of

kernel sources

As options have dependencies, typically never edited by hand,

but through graphical or text interfaces:

Simple text le, key=value style

make xconfig, make gconfig (graphical)

make menuconfig, make nconfig (text)
You can switch from one to another, they all load/save the
same .config le, and show the same set of options

To modify a kernel in a GNU/Linux distribution: the

conguration les are usually released in /boot/, together
with kernel images: /boot/config-3.2.0-31-generic

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

89/443

Kernel or module?

The kernel image is a single le, resulting from the linking

of all object les that correspond to features enabled in the
conguration

This is the le that gets loaded in memory by the bootloader

All included features are therefore available as soon as the
kernel starts, at a time where no lesystem exists

Some features (device drivers, lesystems, etc.) can however

be compiled as modules

These are plugins that can be loaded/unloaded dynamically to

add/remove features to the kernel
Each module is stored as a separate le in the lesystem,
and therefore access to a lesystem is mandatory to use
modules
This is not possible in the early boot procedure of the kernel,
because no lesystem is available

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

90/443

Kernel option types

There are dierent types of options

bool options, they are either

tristate options, they are either

true (to include the feature in the kernel) or

false (to exclude the feature from the kernel)
true (to include the feature in the kernel image) or
module (to include the feature as a kernel module) or
false (to exclude the feature)

int options, to specify integer values

hex options, to specify hexadecimal values
string options, to specify string values

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

91/443

Kernel option dependencies

There are dependencies between kernel options

For example, enabling a network driver requires the network

stack to be enabled
Two types of dependencies

depends on dependencies. In this case, option A that depends

on option B is not visible until option B is enabled
select dependencies. In this case, with option A depending
on option B, when option A is enabled, option B is
automatically enabled

make xconfig allows to see all options, even the ones that
cannot be selected because of missing dependencies. In this
case, they are displayed in gray.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

92/443

make xcong

make xconfig

The most common graphical interface to congure the kernel.

Make sure you read

help -> introduction: useful options!

File browser: easier to load conguration les

Search interface to look for parameters

Required Debian / Ubuntu packages: libqt4-dev g++

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

93/443

make xcong screenshot

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

94/443

make xcong search interface

Looks for a keyword in the parameter name. Allows to select or
unselect found parameters.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

95/443

Kernel conguration options

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

96/443

Corresponding .cong le excerpt

Options are grouped by sections and are prexed with CONFIG_.
#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=m
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
CONFIG_UDF_FS=y
CONFIG_UDF_NLS=y
#
# DOS/FAT/NT Filesystems
#
# CONFIG_MSDOS_FS is not set
# CONFIG_VFAT_FS is not set
CONFIG_NTFS_FS=m
# CONFIG_NTFS_DEBUG is not set
CONFIG_NTFS_RW=y
free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

97/443

make gcong

make gconfig

GTK based graphical

conguration interface.
Functionality similar to that
of make xconfig.

Just lacking a search

functionality.

Required Debian packages:

libglade2-dev

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

98/443

make menucong

make menuconfig

Useful when no graphics are

available. Pretty convenient
too!

Same interface found in

other tools: BusyBox,
Buildroot...

Required Debian packages:

libncurses-dev

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

99/443

make ncong

make nconfig

A newer, similar text

interface

More user friendly (for

example, easier to access
help information).

Required Debian packages:

libncurses-dev

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

100/443

make oldcong

make oldconfig

Needed very often!

Useful to upgrade a .config le from an earlier kernel release

Issues warnings for conguration parameters that no longer

exist in the new kernel.

Asks for values for new parameters (while xconfig and

menuconfig silently set default values for new parameters).

If you edit a .config le by hand, it's strongly recommended to

run make oldconfig afterwards!

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

101/443

Undoing conguration changes

A frequent problem:

After changing several kernel conguration settings, your

kernel no longer works.

If you don't remember all the changes you made, you can get
back to your previous conguration:
$ cp .config.old .config

All the conguration interfaces of the kernel (xconfig,

menuconfig, oldconfig...) keep this .config.old backup
copy.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

102/443

Conguration per architecture

The set of conguration options is architecture dependent

Some conguration options are very architecture-specic

Most of the conguration options (global kernel options,
network subsystem, lesystems, most of the device drivers) are
visible in all architectures.

By default, the kernel build system assumes that the kernel is

being built for the host architecture, i.e. native compilation

The architecture is not dened inside the conguration, but at

a higher level

We will see later how to override this behaviour, to allow the

conguration of kernels for a dierent architecture

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

103/443

Linux kernel introduction

Compiling and installing the kernel

for the host system

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

104/443

Kernel compilation

make

in the main kernel source directory

Remember to run multiple jobs in parallel if you have multiple
CPU cores. Example: make -j 4
No need to run as root!

Generates

vmlinux, the raw uncompressed kernel image, in the ELF

format, useful for debugging purposes, but cannot be booted
arch/<arch>/boot/*Image, the nal, usually compressed,
kernel image that can be booted

bzImage for x86, zImage for ARM, vmImage.gz for Blackn,

etc.

arch/<arch>/boot/dts/*.dtb, compiled Device Tree les (on

some architectures)
All kernel modules, spread over the kernel source tree, as .ko
les.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

105/443

Kernel installation

make install

Installs

Does the installation for the host system by default, so needs

to be run as root. Generally not used when compiling for an
embedded system, as it installs les on the development
workstation.
/boot/vmlinuz-<version>
Compressed kernel image. Same as the one in
arch/<arch>/boot
/boot/System.map-<version>
Stores kernel symbol addresses
/boot/config-<version>
Kernel conguration for this version

Typically re-runs the bootloader conguration utility to take

the new kernel into account.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

106/443

Module installation

make modules_install

Does the installation for the host system by default, so needs

to be run as root

Installs all modules in /lib/modules/<version>/

kernel/
Module .ko (Kernel Object) les, in the same directory
structure as in the sources.
modules.alias
Module aliases for module loading utilities. Example line:
alias sound-service-?-0 snd_mixer_oss
modules.dep, modules.dep.bin (binary hashed)
Module dependencies
modules.symbols, modules.symbols.bin (binary hashed)
Tells which module a given symbol belongs to.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

107/443

Kernel cleanup targets

Clean-up generated les (to force

re-compilation):
make clean

Remove all generated les. Needed when

switching from one architecture to another.
Caution: it also removes your .config le!
make mrproper

Also remove editor backup and patch reject les

(mainly to generate patches):
make distclean

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

108/443

Linux kernel introduction

Cross-compiling the kernel

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

109/443

Cross-compiling the kernel

When you compile a Linux kernel for another CPU architecture

Much faster than compiling natively, when the target system

is much slower than your GNU/Linux workstation.

Much easier as development tools for your GNU/Linux

workstation are much easier to nd.

To make the dierence with a native compiler, cross-compiler

executables are prexed by the name of the target system,
architecture and sometimes library. Examples:
mips-linux-gcc, the prex is mips-linuxarm-linux-gnueabi-gcc, the prex is arm-linux-gnueabi-

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

110/443

Specifying cross-compilation (1)

The CPU architecture and cross-compiler prex are dened through

the ARCH and CROSS_COMPILE variables in the toplevel Makele.

ARCH is the name of the architecture. It is dened by the

name of the subdirectory in arch/ in the kernel sources

Example: arm if you want to compile a kernel for the arm

architecture.

CROSS_COMPILE is the prex of the cross compilation tools

Example: arm-linux- if your compiler is arm-linux-gcc

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

111/443

Specifying cross-compilation (2)

Two solutions to dene ARCH and CROSS_COMPILE:

Pass ARCH and CROSS_COMPILE on the make command line:

make ARCH=arm CROSS_COMPILE=arm-linux- ...
Drawback: it is easy to forget to pass these variables when
you run any make command, causing your build and
conguration to be screwed up.

Dene ARCH and CROSS_COMPILE as environment variables:

export ARCH=arm
export CROSS_COMPILE=arm-linuxDrawback: it only works inside the current shell or terminal.
You could put these settings in a le that you source every
time you start working on the project. If you only work on a
single architecture with always the same toolchain, you could
even put these settings in your ~/.bashrc le to make them
permanent and visible from any terminal.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

112/443

Predened conguration les

Default conguration les available, per board or per-CPU

family

They are stored in arch/<arch>/configs/, and are just

minimal .config les
This is the most common way of conguring a kernel for
embedded platforms

Run make help to nd if one is available for your platform

To load a default conguration le, just run

make acme_defconfig

This will overwrite your existing .config le!

To create your own default conguration le

make savedefconfig, to create a minimal conguration le

mv defconfig arch/<arch>/configs/myown_defconfig

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

113/443

Conguring the kernel

After loading a default conguration le, you can adjust the

conguration to your needs with the normal xconfig,
gconfig or menuconfig interfaces
As the architecture is dierent from your host architecture

Some options will be dierent from the native conguration

(processor and architecture specic options, specic drivers,
etc.)
Many options will be identical (lesystems, network protocols,
architecture-independent drivers, etc.)

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

114/443

Device Tree

Many embedded architectures have a lot of non-discoverable

hardware.

Depending on the architecture, such hardware is either

described using C code directly within the kernel, or using a
special hardware description language in a Device Tree.

ARM, PowerPC, OpenRISC, ARC, Microblaze are examples of

architectures using the Device Tree.
A Device Tree Source, written by kernel developers, is
compiled into a binary Device Tree Blob, passed at boot time
to the kernel.

There is one dierent Device Tree for each board/platform

supported by the kernel, available in
arch/arm/boot/dts/<board>.dtb.

The bootloader must load both the kernel image and the
Device Tree Blob in memory before starting the kernel.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

115/443

Customize your board device tree!

Often needed for embedded board users:

To describe external devices attached

to non-discoverable busses (such as
I2C) and congure them.

To congure pin muxing: choosing

what SoC signals are made available
on the board external connectors.

To congure some system parameters:

ash partitions, kernel command line
(other ways exist)

Useful reference: Device Tree for

Dummies, Thomas Petazzoni (Apr.
2014): http://j.mp/1jQU6NR

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

116/443

Building and installing the kernel

Run make
Copy the nal kernel image to the target storage

make install is rarely used in embedded development, as the

kernel image is a single le, easy to handle

can be zImage, vmlinux, bzImage in arch/<arch>/boot

copying the Device Tree Blob might be necessary as well, they
are available in arch/<arch>/boot/dts

It is however possible to customize the make install behaviour

in arch/<arch>/boot/install.sh

make modules_install is used even in embedded

development, as it installs many modules and description les

make INSTALL_MOD_PATH=<dir>/ modules_install

The INSTALL_MOD_PATH variable is needed to install the
modules in the target root lesystem instead of your host root
lesystem.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

117/443

Booting with U-Boot

Recent versions of U-Boot can boot the zImage binary.

Older versions require a special kernel image format: uImage

uImage is generated from zImage using the mkimage tool. It is

done automatically by the kernel make uImage target.
On some ARM platforms, make uImage requires passing a
LOADADDR environment variable, which indicates at which
physical memory address the kernel will be executed.

In addition to the kernel image, U-Boot can also pass a

Device Tree Blob to the kernel.
The typical boot process is therefore:
1. Load zImage or uImage at address X in memory
2. Load <board>.dtb at address Y in memory
3. Start the kernel with bootz X - Y (zImage case), or
bootm X - Y (uImage case)
The - in the middle indicates no initramfs

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

118/443

Kernel command line

In addition to the compile time conguration, the kernel

behaviour can be adjusted with no recompilation using the
kernel command line
The kernel command line is a string that denes various
arguments to the kernel

It is very important for system conguration

root= for the root lesystem (covered later)
console= for the destination of kernel messages
Many more exist. The most important ones are documented in
Documentation/kernel-parameters.txt in kernel sources.

This kernel command line is either

Passed by the bootloader. In U-Boot, the contents of the

bootargs environment variable is automatically passed to the
kernel
Built into the kernel, using the CONFIG_CMDLINE option.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

119/443

Practical lab - Compile and Boot an Android Kernel

Extract the kernel patchset from

Android Kernel

Compile and boot a kernel for the

emulator

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

120/443

The Android Kernel

Changes
introduced in the
Android Kernel

Embedded Linux
Experts

free electrons
Copyright 2004-2016, Free Electrons.
Creative Commons BY-SA 3.0 license.
Corrections, suggestions, contributions and translations are welcome!

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

121/443

The Android Kernel

Wakelocks

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

122/443

Power management basics

Every CPU has a few states of power consumption, from

being almost completely o, to working at full capacity.

These dierent states are used by the Linux kernel to save

power when the system is run

For example, when the lid is closed on a laptop, it goes into

``suspend'', which is the most power conservative mode of a
device, where almost nothing but the RAM is kept awake

While this is a good strategy for a laptop, it is not necessarily

good for mobile devices

For example, you don't want your music to be turned o

when the screen is

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

123/443

Wakelocks

Android's answer to these power management constraints is

wakelocks

One of the most famous Android changes, because of the

ame wars it spawned

The main idea is instead of letting the user decide when the
devices need to go to sleep, the kernel is set to suspend as
soon and as often as possible.

In the same time, Android allows applications and kernel

drivers to voluntarily prevent the system from going to
suspend, keeping it awake (thus the name wakelock)
This implies to write the applications and drivers to use the
wakelock API.

Applications do so through the abstraction provided by the API

Drivers must do it themselves, which prevents to directly
submit them to the vanilla kernel

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

124/443

Wakelocks API

Kernel Space API

#include <linux/wakelock.h>
void wake_lock_init(struct wakelock *lock,
int type,
const char *name);
void wake_lock(struct wake_lock *lock);
void wake_unlock(struct wake_lock *lock);
void wake_lock_timeout(struct wake_lock *lock, long timeout);
void wake_lock_destroy(struct wake_lock *lock);

User-Space API
$ echo foobar > /sys/power/wake_lock
$ echo foobar > /sys/power/wake_unlock

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

125/443

The Android Kernel

Binder

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

126/443

Binder

RPC/IPC mechanism

Takes its roots from BeOS and the OpenBinder project, which
some of the current Android engineers worked on

Adds remote object invocation capabilities to the Linux Kernel

One of the very basic functionalities of Android. Without it,

Android cannot work.

Every call to the system servers go through Binder, just like

every communication between applications, and even
communication between the components of a single
application.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

127/443

Binder

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

128/443

The Android Kernel

klogger

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

129/443

Logging

Logs are very important to debug a system, either live or after

a fault occurred
In a regular Linux distribution, two components are involved
in the system's logging:

Linux' internal mechanism, accessible with the dmesg

command and holding the output of all the calls to printk()
from various parts of the kernel.
A syslog daemon, which handles the user space logs and
usually stores them in the /var/log directory

From Android developers' point of view, this approach has

two aws:

As the calls to syslog() go through as socket, they generate

expensive task switches
Every call writes to a le, which probably writes to a slow
storage device or to a storage device where writes are expensive

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

130/443

Logger

Android addresses these issues with logger, which is a kernel

driver, that uses 4 circular buers in the kernel memory area.

The buers are exposed in the /dev/log directory and you

can access them through the liblog library, which is in turn,
used by the Android system and applications to write to
logger, and by the logcat command to access them.

This allows to have an extensive level of logging across the

entire AOSP

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

131/443

The Android Kernel

Anonymous Shared Memory

(ashmem)

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

132/443

Shared memory mechanism in Linux

Shared memory is one of the standard IPC mechanisms

present in most OSes

Under Linux, they are usually provided by the POSIX SHM

mechanism, which is part of the System V IPCs

ndk/docs/system/libc/SYSV-IPC.html illustrates all the

love Android developers have for these

The bottom line is that they are awed by design in Linux,

and lead to code leaking resources, be it maliciously or not

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

133/443

Ashmem

Ashmem is the response to these aws

Notable dierences are:

Reference counting so that the kernel can reclaim resources

which are no longer in use
There is also a mechanism in place to allow the kernel to
shrink shared memory regions when the system is under
memory pressure.

The standard use of Ashmem in Android is that a process

opens a shared memory region and share the obtained le
descriptor through Binder.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

134/443

The Android Kernel

Alarm Timers

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

135/443

The alarm driver

Once again, the timer mechanisms available in Linux were not

sucient for the power management policy that Android was
trying to set up

High Resolution Timers can wake up a process, but don't re

when the system is suspended, while the Real Time Clock can
wake up the system if it is suspended, but cannot wake up a
particular process.

Developed the alarm timers on top of the Real Time Clock

and High Resolution Timers already available in the kernel

These timers will be red even if the system is suspended,

waking up the device to do so

Obviously, to let the application do its job, when the

application is woken up, a wakelock is grabbed

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

136/443

The Android Kernel

Low Memory Killer

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

137/443

Low Memory Killer

When the system goes out of memory, Linux throws the OOM
Killer to cleanup memory greedy processes
However, this behaviour is not predictable at all, and can kill
very important components of a phone (Telephony stack,
Graphic subsystem, etc) instead of low priority processes
(Angry Birds)
The main idea is to have another process killer, that kicks in
before the OOM Killer and takes into account the time since
the application was last used and the priority of the
component for the system
It uses various thresholds, so that it rst noties applications
so that they can save their state, then begins to kill
non-critical background processes, and then the foreground
applications
As it is run to free memory before the OOM Killer, the latter
will never be run, as the system will never run out of memory

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

138/443

The Android Kernel

The ION Memory Allocator

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

139/443

ION 1/2

ION was introduced with Ice Cream Sandwich (4.0) version of

Android

Its role is to allocate memory in the system, for most of the

possible cases, and to allow dierent devices to share buers,
without any copy, possibly from an user space application

It's for example useful if you want to retrieve an image from a

camera, and push it to the JPEG hardware encoder from an
user space application

The usual Linux memory allocators can only allocate a buer

that is up to 512 pages wide, with a page usually being 4kiB.

There was previously for Android (and Linux in general) some

vendor specic mechanism to allocate larger physically
contiguous memory areas (nvmap for nVidia, CMEM for TI, etc.)

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

140/443

ION 2/2

ION is here to unify the interface to allocate memory in the

system, no matter on which SoC you're running on.

It uses a system of heaps, with Linux publishing the heaps

available on a given system.

By default, you have three dierent heaps:

system Memory virtually contiguous memory, backed by
vmalloc
system contiguous Physically contiguous memory, backed by
kmalloc
carveout Large physically contiguous memory,
preallocated at boot

It also has a user space interface so that processes can

allocate memory to work on.

https://lwn.net/Articles/480055/

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

141/443

Comparison with mainline equivalents

ION has entered staging since 3.14. And:

The contiguous allocation of the buers is done through CMA

The buer sharing between devices is made through dma-buf
Its user space API also allows to allocate and share buers
from the user space, which was not possible otherwise.
This API is also used to set the allocation constraints devices
might have (for example, when one particular device can only
access a subset of the memory, or when it needs to setup an
IOMMU)

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

142/443

The Android Kernel

Network Security

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

143/443

Paranoid Network

In the standard Linux kernel, every application can open

sockets and communicate over the Network

However, Google was willing to apply a more strict policy with

regard to network access

Access to the network is a permission, with a per application

granularity

Filtered with the GID

You need it to access IP, Bluetooth, raw sockets or RFCOMM

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

144/443

The Android Kernel

Various Drivers and Fixes

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

145/443

Various additions

Android also has a lot of minor features added to the Linux

kernel:

RAM Console, a RAM-based console that survives a reboot to

hold kernel logs
pmem, a physically contiguous memory allocator, written
specically for the Qualcomm MSM SoCs. Obsolete Now.
ADB
YAFFS2
Timed GPIOs

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

146/443

The Android Kernel

Linux Mainline Patches Merge

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

147/443

History

The Android Kernel patches were kept for a long time out of
the ocial Linux release

They were rst integrated in 2.6.29, in

drivers/staging/android

They were then removed from the kernel 2.6.35, because

Google was unwilling to help the mainlining process

They were then added back in 3.3 (around 2 years later) and
are still there at the time

While Google did a great job at keeping most of their changes

as isolated from the core as possible, making this easy to
merge in the staging area, it wasn't true for the wakelocks,
due to their invasive nature.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

148/443

Wakelocks Support

The kernel developpers were not quite happy about the

in-kernel APIs used by the wakelocks

Due to the changes in every places of the kernel to state

wether or not we were allowed to suspend, it was not possible
to merge the changes as is: either you were getting all of it, or
none

Since version 3.5, two features were included in the kernel to

implement opportunistic suspend:
autosleep is a way to let the kernel trigger suspend or
hibernate whenever there are no active wakeup
sources.
wake locks are a way to create and manipulate wakeup
sources from user space. The interface is
compatible with the android one.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

149/443

Current State: Merged Patches

As of 3.10, the following patches/features are now found in

the mainline kernel:

Binder
Alarm Timers (under the name POSIX Alarm Timers
introduced in 2.6.38)
Ashmem
Klogger
Timed GPIOs
Low Memory Killer
RAM Console (superseded by pstore RAM backend introduced
in 3.5)

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

150/443

Current State: Missing Patches

As of 3.10, the following patches/features are missing from

the mainline kernel:

Paranoid Networking
ION Memory Allocator
USB Gadget
FIQ debugger
pmem (removed in 3.3)

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

151/443

Android Bootloaders

Android
Bootloaders

Embedded Linux
Experts

free electrons
Copyright 2004-2016, Free Electrons.
Creative Commons BY-SA 3.0 license.
Corrections, suggestions, contributions and translations are welcome!

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

152/443

Android Bootloaders

Boot Sequence

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

153/443

Bootloaders

The bootloader is a piece of code responsible for

Basic hardware initialization

Loading of an application binary, usually an operating system
kernel, from ash storage, from the network, or from another
type of non-volatile storage.
Possibly decompression of the application binary
Execution of the application

Besides these basic functions, most bootloaders provide a shell

with various commands implementing dierent operations.

Loading of data from storage or network, memory inspection,

hardware diagnostics and testing, etc.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

154/443

Bootloaders on x86 (1)

The x86 processors are typically bundled on a

board with a non-volatile memory containing a
program, the BIOS.
This program gets executed by the CPU after
reset, and is responsible for basic hardware
initialization and loading of a small piece of code
from non-volatile storage.

This piece of code is usually the rst 512 bytes

of a storage device

This piece of code is usually a 1st stage

bootloader, which will load the full bootloader
itself.

The bootloader can then oer all its features. It

typically understands lesystem formats so that
the kernel le can be loaded directly from a
normal lesystem.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

155/443

Bootloaders on x86 (2)

GRUB, Grand Unied Bootloader, the most powerful one.

http://www.gnu.org/software/grub/

Can read many lesystem formats to load the kernel image and
the conguration, provides a powerful shell with various
commands, can load kernel images over the network, etc.
See our dedicated presentation for details:
http://free-electrons.com/docs/grub/

Syslinux, for network and removable media booting (USB key,

CD-ROM)
http://www.kernel.org/pub/linux/utils/boot/syslinux/

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

156/443

Booting on embedded CPUs: case 1

When powered, the CPU starts executing code

at a xed address

There is no other booting mechanism provided

by the CPU

The hardware design must ensure that a NOR

ash chip is wired so that it is accessible at the
address at which the CPU starts executing
instructions

The rst stage bootloader must be programmed

at this address in the NOR

NOR is mandatory, because it allows random

access, which NAND doesn't allow

Not very common anymore (unpractical, and

requires NOR ash)

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

157/443

Booting on embedded CPUs: case 2

The CPU has an integrated boot code in ROM

This boot code is able to load a rst stage bootloader from a

storage device into an internal SRAM (DRAM not initialized
yet)

Storage device can typically be: MMC, NAND, SPI ash,

UART (transmitting data over the serial line), etc.

The rst stage bootloader is

BootROM on AT91 CPUs, ROM code on OMAP, etc.

Exact details are CPU-dependent

Limited in size due to hardware constraints (SRAM size)

Provided either by the CPU vendor or through community
projects

This rst stage bootloader must initialize DRAM and other

hardware devices and load a second stage bootloader into
RAM

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

158/443

Booting on ARM Atmel AT91

RomBoot: tries to nd a valid bootstrap image

from various storage sources, and load it into

SRAM (DRAM not initialized yet). Size limited
to 4 KB. No user interaction possible in standard
boot mode.
AT91Bootstrap: runs from SRAM. Initializes the

DRAM, the NAND or SPI controller, and loads

the secondary bootloader into RAM and starts it.
No user interaction possible.
U-Boot: runs from RAM. Initializes some other

hardware devices (network, USB, etc.). Loads the

kernel image from storage or network to RAM
and starts it. Shell with commands provided.
Linux Kernel: runs from RAM. Takes over the

system completely (bootloaders no longer exists).

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

159/443

Booting on ARM TI OMAP3

ROM Code: tries to nd a valid bootstrap image

from various storage sources, and load it into

SRAM or RAM (RAM can be initialized by ROM
code through a conguration header). Size
limited to <64 KB. No user interaction possible.
X-Loader or U-Boot: runs from SRAM.

Initializes the DRAM, the NAND or MMC

controller, and loads the secondary bootloader
into RAM and starts it. No user interaction
possible. File called MLO.
U-Boot: runs from RAM. Initializes some other

hardware devices (network, USB, etc.). Loads the

kernel image from storage or network to RAM
and starts it. Shell with commands provided. File
called u-boot.bin or u-boot.img.
Linux Kernel: runs from RAM. Takes over the

system completely (bootloaders no longer exists).

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

160/443

Booting on Marvell SoC

ROM Code: tries to nd a valid bootstrap image

from various storage sources, and load it into

RAM. The RAM conguration is described in a
CPU-specic header, prepended to the bootloader
image.
U-Boot: runs from RAM. Initializes some other

hardware devices (network, USB, etc.). Loads the

kernel image from storage or network to RAM
and starts it. Shell with commands provided. File
called u-boot.kwb.
Linux Kernel: runs from RAM. Takes over the

system completely (bootloaders no longer exists).

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

161/443

Generic bootloaders for embedded CPUs

We will focus on the generic part, the main bootloader,

oering the most important features.
There are several open-source generic bootloaders.
Here are the most popular ones:

U-Boot, the universal bootloader by Denx

The most used on ARM, also used on PPC, MIPS, x86, m68k,
NIOS, etc. The de-facto standard nowadays. We will study it
in detail.
http://www.denx.de/wiki/U-Boot
Barebox, a new architecture-neutral bootloader, written as a
successor of U-Boot. Better design, better code, active
development, but doesn't yet have as much hardware support
as U-Boot.
http://www.barebox.org

There are also a lot of other open-source or proprietary

bootloaders, often architecture-specic

RedBoot, Yaboot, PMON, etc.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

162/443

Android Bootloaders

Fastboot

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

163/443

Denition

Fastboot is a protocol to communicate with bootloaders over

USB

It is very simple to implement, making it easy to port on both

new devices and on host systems

Accessible with the fastboot command

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

164/443

The Fastboot protocol

It is very restricted, only 10 commands are dened in the

protocol specications

It is synchronous and driven by the host

Allows to:

Transmit data
Flash the various partitions of the device
Get variables from the bootloader
Control the boot sequence

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

165/443

Session example

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

166/443

Booting into Fastboot

On most devices, it's disabled by default (the bootloader

won't even implement it)
On devices that support it, such as Google Nexus', you have
several options:

Use a combination of keys at boot to start the bootloader

right away into its fastboot mode
Use the adb reboot bootloader command on your
workstation. The device will reboot in fastboot mode, awaiting
for inputs.

You can then interact with the device through the fastboot
command on your workstation

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

167/443

Major Fastboot Commands

You can get all the commands through fastboot -h

The most widely used commands are:

devices Lists the fastboot-capable devices
boot Downloads a kernel and boots on it
erase Erases a given ash partition name
ash Writes a given le to a given ash partition
getvar Retrieves a variable from the bootloader
continue Goes on with a regular boot

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

168/443

getvar Variables

Vendor-specic variables must also begin with a upper-case

letter. Variables beginning with a lower-case letter are
reserved for the Fastboot specications and their evolution.
version Version of the Fastboot protocol implemented
version-bootloader Version of the bootloader
version-baseband Version of the baseband rmware
product Name of the product
serialno Product serial number
secure Does the bootloader require signed images?

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

169/443

Android Build System: Basics

Android Build
System: Basics

Embedded Linux
Experts

free electrons
Copyright 2004-2016, Free Electrons.
Creative Commons BY-SA 3.0 license.
Corrections, suggestions, contributions and translations are welcome!

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

170/443

Android Build System: Basics

Basics

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

171/443

Build Systems

Build systems are designed to meet several goals:

Integrate all the software components, both third-party and

in-house into a working image
Be able to easily reproduce a given build

Usually, they build software using the existing building system

shipped with each component

Several solutions: Yocto, Buildroot, ptxdist.

Google came up with its own solution for Android, that never
relies on other build systems, except for GNU/Make

It allows to rely on very few tools, and to control every

software component in a consistent way.
But it also means that when you have to import a new
component, you have to rewrite the whole Makele to build it

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

172/443

First compilation

$ source build/envsetup.sh
$ lunch
You're building on Linux
Lunch menu... pick a combo:
1. generic-eng
2. simulator
3. full_passion-userdebug
4. full_crespo-userdebug
Which would you like? [generic-eng]
$ make
$ make showcommands

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

173/443

Android Build System: Basics

envsetup.sh

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

174/443

Purpose

Obviously modies the current environment, that's why we

have to source it

It adds many useful shell macros

These macros will serve several purposes:

Congure and set up the build system

Ease the navigation in the source code
Ease the development process

Some macros will modify the environment variables, to be

used by the build system later on

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

175/443

Environments variables exported 1/2

ANDROID_EABI_TOOLCHAIN

ANDROID_TOOLCHAIN

Equals to ANDROID_EABI_TOOLCHAIN

ANDROID_QTOOLS

Path to the Android prebuilt toolchain (.../prebuilt/linuxx86/toolchain/arm-eabi-4.4.3/bin)

Tracing tools for qemu (.../development/emulator/qtools).

This is weird however, since this path doesn't exist at all

ANDROID_BUILD_PATHS

Path containing all the folders containing tools for the build
(.../out/host/linux-x86/bin:$ANDROID_TOOLCHAIN:
$ANDROID_QTOOLS:$ANDROID_TOOLCHAIN:
$ANDROID_EABI_TOOLCHAIN)

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

176/443

Environments variables exported 2/2

JAVA_HOME

ANDROID_JAVA_TOOLCHAIN

Alias to ANDROID_JAVA_TOOLCHAIN

ANDROID_PRODUCT_OUT

Path to the Java toolchain ($JAVA_HOME/bin)

ANDROID_PRE_BUILD_PATHS

Path to the Java environment (/usr/lib/jvm/java-6-sun)

Path to where the generated les will be for this product

(.../out/target/product/<product_name>)

OUT

Alias to ANDROID_PRODUCT_OUT

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

177/443

Dened Commands 1/2

lunch Used to congure the build system

croot Changes the directory to go back to the root of the
Android source tree
cproj Changes the directory to go back to the root of the
current package
tapas Congure the build system to build a given
application
m Makes the whole build from any directory in the
source tree
mm Builds the modules dened in the current directory
mmm Builds the modules dened in the given directory

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

178/443

Dened Commands 2/2

cgrep Greps the given pattern on all the C/C++/header

les
jgrep Greps the given pattern on all the Java les
resgrep Greps the given pattern on all the resources les
mgrep Greps the given pattern on all the Makeles
sgrep Greps the given pattern on all Android source le
godir Go to the directory containing the given le
pid Use ADB to get the PID of the given process
gdbclient Use ADB to set up a remote debugging session
key_back Sends a input event corresponding to the Back key to
the device

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

179/443

Android Build System: Basics

Conguration of the Build System

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

180/443

Conguration

The Android build system is not much congurable compared

to other build systems, but it is possible to modify to some
extent

Among the several conguration options you have, you can

add extra ags for the C compiler, have a given package built
with debug options, specify the output directory, and rst of
all, choose what product you want to build.

This is done either through the lunch command or through a

buildspec.mk le placed at the top of the source directory

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

181/443

lunch

lunch is a shell function dened in build/envsetup.sh

It is the easiest way to congure a build. You can either

launch it without any argument and it will ask to choose
among a list of known ``combos'' or launch it with the
desired combos as argument.

It sets the environment variables needed for the build and

allows to start compiling at last

You can declare new combos through the add_lunch_combo

command

These combos are the aggregation of the product to build and

the variant to use (basically, which set of modules to install)

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

182/443

Variables Exported by Lunch

TARGET_PRODUCT

Which product to build. To build for the emulator, you will

have aosp_<arch>

TARGET_BUILD_VARIANT

Select which set of modules to build, among

user: Includes modules tagged user (Phone)

userdebug: Includes modules tagged user or debug (strace)
eng: Includes modules tagged user, debug or eng:
(e2fsprogs)

TARGET_BUILD_TYPE

Either release or debug. If debug is set, it will enable some

debug options across the whole system.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

183/443

buildspec.mk

While lunch is convenient to quickly switch from one

conguration to another. If you have only one product or you
want to do more ne-grained conguration, this is not really
convenient

The le buildspec.mk is here for that.

If you place it at the top of the sources, it will be used by the

build system to get its conguration instead of relying on the
environment variables

It oers more variables to modify, such as compiling a given

module with debugging symbols, additional C compiler ags,
change the output directory...

A sample is available in build/buildspec.mk.default, with

lots of comments on the various variables.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

184/443

Android Build System: Basics

Results

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

185/443

Output

All the output is generated in the out/ directory, outside of

the source code directory

This directory contains mostly two subdirectories: host/ and

target/

These directories contain all the objects les compiled during

the build process: .o les for C/C++ code, .jar les for
Java libraries, etc

It is an interesting feature, since it keeps all the generated

stu separate from the source code, and we can easily clean
without side eects

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

186/443

Images

It also generates the system images in the

out/target/product/<device_name>/ directory

These images are:

boot.img A basic Android image, containing only the
needed components to boot: a kernel image and
a minimal system
system.img The remaining parts of Android. Much bigger, it
contains most of the framework, applications
and daemons
userdata.img A partition that will hold the user generated
content. Mostly empty at compilation.
recovery.img A recovery image that allows to be able to
debug or restore the system when something
nasty happened.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

187/443

Android Boot Images

The boot images are actually an Android-specic format, that

holds most of what the bootloader expects

They contains useful information, like the kernel command

line, where to load the kernel, but also the image of the
kernel, and an optional initramfs image

A custom mkbootimg tool is used by Android to generate

these images at compilation time from the kernel and the
system it's generating

We can tweak the behaviour of that tool from the build

system conguration, that allows a great exibility

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

188/443

Android boot and recovery images

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

189/443

Boot sequence

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

190/443

Cleaning

Cleaning is almost as easy as rm -rf out/

make clean or make clobber deletes all generated les.

make installclean removes the installed les for the current

combo. It is useful when you work with several products to
avoid doing a full rebuild each time you change from one to
the other

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

191/443

Practical lab - Supporting a New Board

Boot Android on a real hardware

Troubleshoot simple problems on

Android

Generate a working build

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

192/443

Android Debug Bridge

Developing and
Debugging with
ADB

Embedded Linux
Experts

free electrons
Copyright 2004-2016, Free Electrons.
Creative Commons BY-SA 3.0 license.
Corrections, suggestions, contributions and translations are welcome!

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

193/443

Android Debug Bridge

Introduction

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

194/443

ADB

Usually on embedded devices, debugging is done either

through a serial port on the device or JTAG for low-level
debugging

This setup works well when developing a new product that

will have a static system. You develop and debug a system on
a product with serial and JTAG ports, and remove these ports
from the nal product.

For mobile devices, where you will have applications

developers that are not in-house, this is not enough.

To address that issue, Google developed ADB, that runs on

top of USB, so that another developer can still have
debugging and low-level interaction with a production device.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

195/443

Implementation

The code is split in 3 components:

ADBd can work either on top of TCP or USB.

ADBd, the part that runs on the device

ADB server, which is run on the host, acts as a proxy and
manages the connection to ADBd
ADB clients, which are also run on the host, and are what is
used to send commands to the device
For USB, Google has implemented a driver using the USB
gadget and the USB composite frameworks as it implements
either the ADB protocol and the USB Mass Storage
mechanism.
For TCP, ADBd just opens a socket

ADB can also be used as a transport layer between the

development platform and the device, disregarding whether it
uses USB or TCP as underneath layer

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

196/443

ADB Architecture

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

197/443

Android Debug Bridge

Use of ADB

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

198/443

ADB commands: Basics

start-server Starts the ADB server on the host

kill-server Kills the ADB server on the host
devices Lists accessible devices
connect Connects to a remote ADBd using TCP port 5555 by
default
disconnect Disconnects from a connected device
help Prints available commands with help information
version Prints the version number

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

199/443

ADB commands: Files and applications

push Copies a local le to the device
pull Copies a remote le from the device
sync There are three cases here:
If no argument is passed, copies the local
directories system and data if they dier from
/system and /data on the target.
If either system or data is passed, syncs this
directory with the associated partition on the
device
Else, syncs the given folder
install Installs the given Android package (apk) on the
device
uninstall Uninstalls the given package name from the device
free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

200/443

ADB commands: Debugging

logcat Prints the device logs. You can lter either on the
source of the logs or their on their priority level
shell Runs a remote shell with a command line interface.
If an argument is given, runs it as a command and
prints out the result
bugreport Gets all the relevant information to generate a bug
report from the device: logs, internal state of the
device, etc.
jdwp Lists the processes that support the JDWP protocol

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

201/443

ADB commands: Scripting 1/2

wait-for-device Blocks until the device gets connected to ADB.

You can also add additional commands to be run
when the device becomes available.
get-state Prints the current state of the device, offline,
bootloader or device
get-serialno Prints the serial number of the device
remount Remounts the /system partition on the device in
read/write mode

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

202/443

ADB commands: Scripting 2/2

reboot Reboots the device. bootloader and recovery

arguments are available to select the operation mode
you want to reboot to.
reboot-bootloader Reboots the device into the bootloader
root Restarts ADBd with root permissions on the device
Useful if the ro.secure property is set to 1 to
force ADB into user mode. But ro.debuggable
has to be set to 1 to allow to restart ADB as
root
usb Restarts ADBd listening on USB
tcpip Restarts ADBd listening on TCP on the given port

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

203/443

ADB commands: Easter eggs

lolcat Alias to adb logcat

hell Equivalent to adb shell, with a dierent color
scheme

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

204/443

Android Debug Bridge

Examples

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

205/443

ADB forward and gdb

adb forward tcp:5555 tcp:1234

See also gdbclient
free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

206/443

ADB forward and jdb

adb forward tcp:5555 jdwp:4242

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

207/443

Various commands

Wait for a device and install an application

Test an application by sending random user input

adb shell monkey -v -p com.free-electrons.foobar 500

Filter system logs

adb wait-for-device install foobar.apk

adb logcat ActivityManager:I FooBar:D *:S

You can also set the ANDROID_LOG_TAGS environment variable
on your workstation

Access other log buers

adb logcat -b radio

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

208/443

Practical lab - Use ADB

Debug your system and

applications

Get a shell on a device

Exchange les with a device

Install new applications

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

209/443

Android Filesystem

Android
Filesystem

Embedded Linux
Experts

free electrons
Copyright 2004-2016, Free Electrons.
Creative Commons BY-SA 3.0 license.
Corrections, suggestions, contributions and translations are welcome!

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

210/443

Android Filesystem

Principle and solutions

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

211/443

Filesystems

Filesystems are used to organize data in directories and les

on storage devices or on the network. The directories and les
are organized as a hierarchy

In Unix systems, applications and users see a single global

hierarchy of les and directories, which can be composed of
several lesystems.
Filesystems are mounted in a specic location in this
hierarchy of directories

When a lesystem is mounted in a directory (called mount

point), the contents of this directory reects the contents of
the storage device
When the lesystem is unmounted, the mount point is empty
again.

This allows applications to access les and directories easily,

regardless of their exact storage location

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

212/443

Filesystems (2)

Create a mount point, which is just a directory

$ mkdir /mnt/usbkey

It is empty
$ ls /mnt/usbkey
$

Mount a storage device in this mount point

$ mount -t vfat /dev/sda1 /mnt/usbkey
$

You can access the contents of the USB key

$ ls /mnt/usbkey
docs prog.c picture.png movie.avi
$

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

213/443

mount / umount

mount allows to mount lesystems

mount -t type device mountpoint

type is the type of lesystem
device is the storage device, or network location to mount
mountpoint is the directory where les of the storage device or
network location will be accessible
mount with no arguments shows the currently mounted
lesystems

umount allows to unmount lesystems

This is needed before rebooting, or before unplugging a USB

key, because the Linux kernel caches writes in memory to
increase performance. umount makes sure that these writes are
committed to the storage.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

214/443

Root lesystem

A particular lesystem is mounted at the root of the hierarchy,

identied by /

This lesystem is called the root lesystem

As mount and umount are programs, they are les inside a
lesystem.

They are not accessible before mounting at least one

lesystem.

As the root lesystem is the rst mounted lesystem, it

cannot be mounted with the normal mount command

It is mounted directly by the kernel, according to the root=

kernel option

When no root lesystem is available, the kernel panics

Please append a correct "root=" boot option
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown block(0,0)

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

215/443

Location of the root lesystem

It can be mounted from dierent locations

From the partition of a hard disk

From the partition of a USB key
From the partition of an SD card
From the partition of a NAND ash chip or similar type of
storage device
From the network, using the NFS protocol
From memory, using a pre-loaded lesystem (by the
bootloader)
etc.

It is up to the system designer to choose the conguration for

the system, and congure the kernel behaviour with root=

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

216/443

Mounting rootfs from storage devices

Partitions of a hard disk or USB key

Partitions of an SD card

root=/dev/sdXY, where X is a letter indicating the device, and

Y a number indicating the partition
/dev/sdb2 is the second partition of the second disk drive
(either USB key or ATA hard drive)
root=/dev/mmcblkXpY, where X is a number indicating the
device and Y a number indicating the partition
/dev/mmcblk0p2 is the second partition of the rst device

Partitions of ash storage

root=/dev/mtdblockX, where X is the partition number

/dev/mtdblock3 is the fourth partition of a NAND ash chip
(if only one NAND ash chip is present)

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

217/443

rootfs in memory: initramfs (1)

It is also possible to have the root lesystem integrated into

the kernel image
It is therefore loaded into memory together with the kernel
This mechanism is called initramfs

It integrates a compressed archive of the lesystem into the

kernel image
Variant: the compressed archive can also be loaded separately
by the bootloader.

It is useful for two cases

Fast booting of very small root lesystems. As the lesystem is

completely loaded at boot time, application startup is very fast.
As an intermediate step before switching to a real root
lesystem, located on devices for which drivers not part of the
kernel image are needed (storage drivers, lesystem drivers,
network drivers). This is always used on the kernel of
desktop/server distributions to keep the kernel image size
reasonable.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

218/443

rootfs in memory: initramfs (2)

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

219/443

rootfs in memory: initramfs (3)

The contents of an initramfs are dened at the kernel

conguration level, with the CONFIG_INITRAMFS_SOURCE
option

Can be the path to a directory containing the root lesystem

contents
Can be the path to a cpio archive
Can be a text le describing the contents of the initramfs
(see documentation for details)

The kernel build process will automatically take the contents

of the CONFIG_INITRAMFS_SOURCE option and integrate the
root lesystem into the kernel image

Details (in kernel sources):

Documentation/filesystems/ramfs-rootfs-initramfs.txt
Documentation/early-userspace/README

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

220/443

Android Filesystem

Contents

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

221/443

Filesystem organization on GNU/Linux

On most Linux based distributions, the lesystem layout is

dened by the Filesystem Hierarchy Standard

The FHS denes the main directories and their contents

/bin Essential command binaries
/boot Bootloader les, i.e. kernel images and related
stu
/etc Host-specic system-wide conguration les.

Android follows an orthogonal path, storing its les in folders

not present in the FHS, or following it when it uses a dened
folder

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

222/443

Filesystem organization on Android

Instead, the two main directories used by Android are

/system Immutable directory coming from the original
build. It contains native binaries and libraries,
framework jar les, conguration les, standard
apps, etc.
/data is where all the changing content of the system
are put: apps, data added by the user, data
generated by all the apps at runtime, etc.

These two directories are usually mounted on separate

partitions, from the root lesystem originating from a kernel
RAM disk.

Android also uses some usual suspects: /proc, /dev, /sys,

/etc, /sbin, /mnt where they serve the same function they
usually do

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

223/443

/system

./app All the pre-installed apps

./bin Binaries installed on the system (toolbox, vold,
surfaceinger)
./etc Conguration les
./fonts Fonts installed on the system
./framework Jar les for the framework
./lib Shared objects for the system libraries
./modules Kernel modules
./xbin External binaries

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

224/443

Other directories

Like we said earlier, Android most of the time either uses

directories not in the FHS, or directories with the exact same
purpose as in standard Linux distributions (/dev, /proc,
/sys), therefore avoiding collisions.

There are some collisions though, for /etc and /sbin, which
are hopefully trimmed down

This allows to have a full Linux distribution side by side with

Android with only minor tweaks

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

225/443

android_lesystem_cong.h

Located in system/core/include/private/
Contains the full lesystem setup, and is written as a C
header

UID/GID
Permissions for system directories
Permissions for system les

Processed at compilation time to enforce the permissions

throughout the lesystem

Useful in other parts of the framework as well, such as ADB

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

226/443

Android Filesystem

Device Files

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

227/443

Devices

One of the kernel important role is to allow applications to

access hardware devices
In the Linux kernel, most devices are presented to user space
applications through two dierent abstractions

Character device
Block device

Internally, the kernel identies each device by a triplet of

information

Type (character or block)

Major (typically the category of device)
Minor (typically the identier of the device)

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

228/443

Types of devices

Block devices

A device composed of xed-sized blocks, that can be read and

written to store data
Used for hard disks, USB keys, SD cards, etc.

Character devices

Originally, an innite stream of bytes, with no beginning, no

end, no size. The pure example: a serial port.
Used for serial ports, terminals, but also sound cards, video
acquisition devices, frame buers
Most of the devices that are not block devices are represented
as character devices by the Linux kernel

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

229/443

Android Filesystem

Minimal lesystem

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

230/443

Basic applications

In order to work, a Linux system needs at least a few applications

An init application, which is the rst user space application started
by the kernel after mounting the root lesystem
The kernel tries to run /sbin/init, /bin/init, /etc/init
and /bin/sh.
In the case of an initramfs, it will only look for /init. Another
path can be supplied by the rdinit kernel argument.
If none of them are found, the kernel panics and the boot
process is stopped.
The init application is responsible for starting all other user
space applications and services
A shell, to implement scripts, automate tasks, and allow a user to
interact with the system
Basic Unix applications, to copy les, move les, list les
(commands like mv, cp, mkdir, cat, etc.)
These basic components have to be integrated into the root
lesystem to make it usable

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

231/443

Overall booting process

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

232/443

Android Build System: Advanced

Android Build
System: Advanced

Embedded Linux
Experts

free electrons
Copyright 2004-2016, Free Electrons.
Creative Commons BY-SA 3.0 license.
Corrections, suggestions, contributions and translations are welcome!

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

233/443

Android Build System: Advanced

Add a New Module

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

234/443

Modules

Every component in Android is called a module

Modules are dened across the entire tree through the

Android.mk les

The build system abstracts many details to make the creation

of a module's Makele as trivial as possible

Of course, building a module that will be an Android

application and building a static library will not require the
same instructions, but these builds don't dier that much
either.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

235/443

Hello World

LOCAL_PATH := $(call my-dir)

include $(CLEAR_VARS)
LOCAL_SRC_FILES = hello_world.c
LOCAL_MODULE = HelloWorld
LOCAL_MODULE_TAGS = optional
include $(BUILD_EXECUTABLE)

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

236/443

Hello World

Every module variable is prexed by LOCAL_*

LOCAL_PATH tells the build system where the current module is

include $(CLEAR_VARS) cleans the previously declared

LOCAL_* variables. This way, we make sure we won't have
anything weird coming from other modules. The list of the
variables cleared is in build/core/clear_vars.mk

LOCAL_SRC_FILES contains a list of all source les to be

compiled

LOCAL_MODULE sets the module name

LOCAL_MODULE_TAGS denes the set of modules this module

should belong to

include $(BUILD_EXECUTABLE) tells the build system to

build this module as a binary

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

237/443

Tags

Tags are used to dene several sets of modules to be built

through the build variant selected by lunch
We have 3 build variants:

user

userdebug is user plus

Installs modules tagged with debug

ro.debuggable = 1
ADB is enabled by default

eng is userdebug, plus

Installs modules tagged with user

Installs non-packaged modules that have no tags specied
ro.secure = 1
ro.debuggable = 0
ADB is disabled by default

Installs modules tagged as eng and development

ro.secure = 0
ro.kernel.android.checkjni = 1

Finally, we have a fourth tag, optional, that will never be

directly integrated by a build variant, but deprecates user

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

238/443

Build Targets 1/3

BUILD_EXECUTABLE

BUILD_HOST_EXECUTABLE

Builds a binary to be run on bare metal

BUILD_JAVA_LIBRARY

Builds an ELF binary to be run on the host

BUILD_RAW_EXECUTABLE

Builds a normal ELF binary to be run on the target

Builds a Java library (.jar) to be used on the target

BUILD_STATIC_JAVA_LIBRARY

Builds a static Java library to be used on the target

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

239/443

Build Targets 2/3

BUILD_HOST_JAVA_LIBRARY

BUILD_SHARED_LIBRARY

Builds a shared library for the host

BUILD_HOST_STATIC_LIBRARY

Builds a static library for the target

BUILD_HOST_SHARED_LIBRARY

Builds a shared library for the target

BUILD_STATIC_LIBRARY

Builds a Java library to be used on the host

Builds a static library for the host

BUILD_RAW_STATIC_LIBRARY

Builds a static library to be used on bare metal

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

240/443

Build Targets 3/3

BUILD_PREBUILT

BUILD_HOST_PREBUILT

Used to install prebuilt les of multiple modules of known types

BUILD_PACKAGE

Used to install prebuilt les on the host

BUILD_MULTI_PREBUILT

Used to install prebuilt les on the target (conguration les,

kernel)

Builds a standard Android package (.apk)

BUILD_KEY_CHAR_MAP

Builds a device character map

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

241/443

Other useful variables

LOCAL_CFLAGS

LOCAL_SHARED_LIBRARIES

List of paths to extra headers used by this module

LOCAL_REQUIRED_MODULES

Equivalent to LOCAL_MODULE for Android packages

LOCAL_C_INCLUDES

List of shared libraries this module depends on at compilation

time

LOCAL_PACKAGE_NAME

Extra C compiler ags to use to build the module

Express that a given module depends on another at runtime,

and therefore should be included in the image as well

Many other similar options depending on what you want to do

You can get a complete list by reading
build/core/clear_vars.mk

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

242/443

Useful Make Macros

In the build/core/definitions.mk le, you will nd useful

macros to use in the Android.mk le, that mostly allows you
to:

Find les

Transform them

transform-c-to-o, ...

Copy them

and some utilities

all-makefiles-under, all-subdir-c-files, etc

copy-file-to-target, ...
my-dir, inherit-package, etc

All these macros should be called through Make's call

command, possibly with arguments

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

243/443

Prebuilt Package Example

LOCAL_PATH := $(call my-dir)

include $(CLEAR_VARS)
LOCAL_MODULE_TAGS := optional
LOCAL_MODULE := configuration_files.txt
LOCAL_MODULE_CLASS := ETC
LOCAL_MODULE_PATH := $(TARGET_OUT_ETC)
LOCAL_SRC_FILES := $(LOCAL_MODULE)
include $(BUILD_PREBUILT)

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

244/443

Making and cleaning a module (1/2)

To build a module from the top directory, just do

make ModuleName

The les generated will be put in

out/target/product/$TARGET_DEVICE/obj/<module_type>
/<module_name>_intermediates

However, building a simple module won't regenerate a new

image. This is just useful to make sure that the module
builds. You will have to do a full make to have an image that
contains your module

Actually, a full make will build your module at some point, but
you won't nd it in your generated image if it is tagged as
optional

If you want to enable it for all builds, add its name to the
PRODUCT_PACKAGES variables in the
build/target/product/core.mk le.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

245/443

Making and cleaning a module (2/2)

To clean a single module, do make clean-ModuleName

You can also get the list of the modules available in the build
system with the make modules target

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

246/443

Practical lab - Building a Library

Add an external library to the

Android build system

Compile it statically and

dynamically

Add a component to a build

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

247/443

Practical lab - Add a Native Application to the Build

Add an external binary to a system

Express dependencies on other

components of the build system

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

248/443

Android Build System: Advanced

Add a New Product

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

249/443

Dening new products

Devices are well supported by the Android build system. It

allows to build multiple devices with the same source tree, to
have a per-device conguration, etc.

All the product denitions should be put in

device/<company>/<device>

The entry point is the AndroidProducts.mk le, which should

dene the PRODUCT_MAKEFILES variable

This variable denes where the actual product denitions are

located.

It follows such an architecture because you can have several

products using the same device

If you want your product to appear in the lunch menu, you

need to create a vendorsetup.sh le in the device directory,
with the right calls to add_lunch_combo

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

250/443

Product, devices and boards

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

251/443

Minimal Product Declaration

$(call inherit-product, build/target/product/generic.mk)

PRODUCT_NAME := full_MyDevice
PRODUCT_DEVICE := MyDevice
PRODUCT_MODEL := Full flavor of My Brand New Device

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

252/443

Copy les to the target

$(call inherit-product, build/target/product/generic.mk)

PRODUCT_COPY_FILES += \
device/mybrand/mydevice/vold.fstab:system/etc/vold.fstab
PRODUCT_NAME := full_MyDevice
PRODUCT_DEVICE := MyDevice
PRODUCT_MODEL := Full flavor of My Brand New Device

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

253/443

Add a package to the build for this product

$(call inherit-product, build/target/product/generic.mk)

PRODUCT_PACKAGES += FooBar
PRODUCT_COPY_FILES += \
device/mybrand/mydevice/vold.fstab:system/etc/vold.fstab
PRODUCT_NAME := full_mydevice
PRODUCT_DEVICE := mydevice
PRODUCT_MODEL := Full flavor of My Brand New Device

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

254/443

Overlays

This is a mechanism used by products to override resources

already dened in the source tree, without modifying the
original code

This is used for example to change the wallpaper for one

particular device

Use the DEVICE_PACKAGE_OVERLAYS or

PRODUCT_PACKAGE_OVERLAYS variables that you set to a path
to a directory in your device folder

This directory should contain a structure similar to the source

tree one, with only the les that you want to override

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

255/443

Add a device overlay

$(call inherit-product, build/target/product/generic.mk)

PRODUCT_PACKAGES += FooBar
PRODUCT_COPY_FILES += \
device/mybrand/mydevice/vold.fstab:system/etc/vold.fstab
DEVICE_PACKAGE_OVERLAYS := device/mybrand/mydevice/overlay
PRODUCT_NAME := full_mydevice
PRODUCT_DEVICE := mydevice
PRODUCT_MODEL := Full flavor of My Brand New Device

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

256/443

Board Denition

You will also need a BoardConfig.mk le along with the

product denition

While the product denition was mostly about the build

system in itself, the board denition is more about the
hardware

However, this is poorly documented and sometimes

ambiguous so you will probably have to dig into the
build/core/Makefile at some point to see what a given
variable does

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

257/443

Minimal Board Denition

TARGET_NO_BOOTLOADER := true
TARGET_NO_KERNEL := true
TARGET_CPU_ABI := armeabi
HAVE_HTC_AUDIO_DRIVER := true
BOARD_USES_GENERIC_AUDIO := true
USE_CAMERA_STUB := true

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

258/443

Other Board Variables 1/2

TARGET_ARCH_VARIANT

TARGET_EXTRA_CFLAGS

Does the CPU have multiple cores?

TARGET_USERIMAGES_USE_EXT4

Extra C compiler ags to use during the whole build

TARGET_CPU_SMP

Variant of the selected architecture (for example

armv7-a-neon for most Cortex-A8 and A9 CPUs)

We want to use ext4 lesystems for our generated partitions

BOARD_SYSTEMIMAGE_PARTITION_SIZE

Size of the system partitions in bytes.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

259/443

Other Board Variables 2/2

BOARD_NAND_PAGE_SIZE

TARGET_NO_RECOVERY

For NAND ash, size of the pages as given by the datasheet

We don't want to build the recovery image

BOARD_KERNEL_CMDLINE

Boot arguments of the kernel

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

260/443

Kernel Integration into Android

Android is just a user space software stack, the build system

isn't designed to build the kernel

However, there is some facilities to integrate a precompiled

kernel into an Android image
To do so, you need to:

In BoardConfig.mk

Remove TARGET_NO_KERNEL if set

Set BOARD_KERNEL_BASE to the load address of your kernel

In your device Makele, have something like

ifeq ($(TARGET_PREBUILT_KERNEL),)
LOCAL_KERNEL := device/ti/panda/kernel
else
LOCAL_KERNEL := $(TARGET_PREBUILT_KERNEL)
endif
PRODUCT_COPY_FILES := \
$(LOCAL_KERNEL):kernel

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

261/443

Practical lab - System Customization

Use the product conguration

system

Change the default wallpaper

Add extra properties to the system

Use the product overlays

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

262/443

Android Native Layer

Android Native
Layer

Embedded Linux
Experts

free electrons
Copyright 2004-2016, Free Electrons.
Creative Commons BY-SA 3.0 license.
Corrections, suggestions, contributions and translations are welcome!

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

263/443

Android Native Layer

Denition and Components

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

264/443

Denition (1)

The usual development tools available on a GNU/Linux

workstation is a native toolchain

This toolchain runs on your workstation and generates code

for your workstation, usually x86
For embedded system development, it is usually impossible or
not interesting to use a native toolchain

The target is too restricted in terms of storage and/or memory

The target is very slow compared to your workstation
You may not want to install all development tools on your
target.

Therefore, cross-compiling toolchains are generally used.

They run on your workstation but generate code for your
target.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

265/443

Denition (2)

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

266/443

Machines in build procedures

Three machines must be distinguished when discussing

toolchain creation

The build machine, where the toolchain is built.

The host machine, where the toolchain will be executed.
The target machine, where the binaries created by the
toolchain are executed.

Four common build types are possible for toolchains

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

267/443

Dierent toolchain build procedures

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

268/443

Components

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

269/443

Binutils

Binutils is a set of tools to generate and manipulate binaries

for a given CPU architecture

as, the assembler, that generates binary code from assembler

source code
ld, the linker
ar, ranlib, to generate .a archives, used for libraries
objdump, readelf, size, nm, strings, to inspect binaries.
Very useful analysis tools!
strip, to strip useless parts of binaries in order to reduce their
size

http://www.gnu.org/software/binutils/

GPL license

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

270/443

Kernel headers (1)

The C library and compiled

programs needs to interact with
the kernel

Available system calls and their

numbers
Constant denitions
Data structures, etc.

Therefore, compiling the C library

requires kernel headers, and many
applications also require them.

Available in <linux/...> and

<asm/...> and a few other
directories corresponding to the
ones visible in include/ in the
kernel sources

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

271/443

Kernel headers (2)

System call numbers, in <asm/unistd.h>

#define __NR_exit
#define __NR_fork
#define __NR_read

1
2
3

Constant denitions, here in <asm-generic/fcntl.h>,

included from <asm/fcntl.h>, included from
<linux/fcntl.h>
#define O_RDWR 00000002

Data structures, here in <asm/stat.h>

struct stat {
unsigned long st_dev;
unsigned long st_ino;
[...]
};

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

272/443

Kernel headers (3)

The kernel to user space ABI is backward compatible

Binaries generated with a toolchain using kernel headers older

than the running kernel will work without problem, but won't
be able to use the new system calls, data structures, etc.
Binaries generated with a toolchain using kernel headers newer
than the running kernel might work on if they don't use the
recent features, otherwise they will break
Using the latest kernel headers is not necessary, unless access
to the new kernel features is needed

The kernel headers are extracted from the kernel sources using
the headers_install kernel Makele target.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

273/443

GCC

GNU Compiler Collection, the famous free

software compiler

Can compile C, C++, Ada, Fortran, Java,

Objective-C, Objective-C++, and generate code
for a large number of CPU architectures,
including ARM, AVR, Blackn, CRIS, FRV,
M32, MIPS, MN10300, PowerPC, SH, v850,
i386, x86_64, IA64, Xtensa, etc.

http://gcc.gnu.org/

Available under the GPL license, libraries under

the LGPL.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

274/443

C library

The C library is an essential component of

a Linux system

Interface between the applications and

the kernel
Provides the well-known standard C API
to ease application development

Several C libraries are available:

glibc, uClibc, musl, dietlibc, newlib, etc.

The choice of the C library must be made

at the time of the cross-compiling
toolchain generation, as the GCC compiler
is compiled against a specic C library.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

275/443

Android Native Layer

Bionic

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

276/443

Whole Android Stack

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

277/443

Bionic 1/2

Google developed another C library for Android: Bionic.

They didn't start from scratch however, they based their work
on the BSD standard C library.

The most remarkable thing about Bionic is that it doesn't

have full support for the POSIX API, so it might be a hurdle
when porting an already developed program to Android.
Among other things, are lacking:

Full pthreads API

No locales and wide chars support
No openpty(), syslog(), crypt(), functions
Removed dependency on the /etc/resolv.conf and
/etc/passwd les and using Android's own mechanisms
instead
Some functions are still unimplemented (see
getprotobyname()

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

278/443

Bionic 2/2

However, Bionic has been created this way for a number of

reasons

Keep the libc implementation as simple as possible, so that it

can be fast and lightweight (Bionic is a bit smaller than uClibc)
Keep the (L)GPL code out of the user space. Bionic is under
the BSD license

And it implements some Android-specics functions as well:

Access to system properties

Logging events in the system logs

In the prebuilt/ directory, Google provides a prebuilt

toolchain that uses Bionic

See http://androidxref.com/4.0.4/xref/ndk/docs/
system/libc/OVERVIEW.html for details about Bionic.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

279/443

Android Native Layer

Toolbox

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

280/443

Whole Android Stack

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

281/443

Why Toolbox?

A Linux system needs a basic set of programs to work

In normal Linux systems, these programs are provided by

dierent projects

An init program
A shell
Various basic utilities for le manipulation and system
conguration

coreutils, bash, grep, sed, tar, wget, modutils, etc. are all
dierent projects
Many dierent components to integrate
Components not designed with embedded systems constraints
in mind: they are not very congurable and have a wide range
of features

Busybox is an alternative solution, extremely common on

embedded systems

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

282/443

General purpose toolbox: BusyBox

Rewrite of many useful Unix command line utilities

Integrated into a single project, which makes it easy to work

with
Designed with embedded systems in mind: highly congurable,
no unnecessary features

All the utilities are compiled into a single executable,

/bin/busybox

Symbolic links to /bin/busybox are created for each

application integrated into Busybox

For a fairly featureful conguration, less than 500 KB

(statically compiled with uClibc) or less than 1 MB (statically
compiled with glibc).

http://www.busybox.net/

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

283/443

BusyBox commands!

Commands available in BusyBox 1.13

[, [[, addgroup, adduser, adjtimex, ar, arp, arping, ash, awk, basename, bbconfig, bbsh, brctl,
bunzip2, busybox, bzcat, bzip2, cal, cat, catv, chat, chattr, chcon, chgrp, chmod, chown, chpasswd,
chpst, chroot, chrt, chvt, cksum, clear, cmp, comm, cp, cpio, crond, crontab, cryptpw, cttyhack, cut,
date, dc, dd, deallocvt, delgroup, deluser, depmod, devfsd, df, dhcprelay, diff, dirname, dmesg,
dnsd, dos2unix, dpkg, dpkg_deb, du, dumpkmap, dumpleases, e2fsck, echo, ed, egrep, eject, env,
envdir, envuidgid, ether_wake, expand, expr, fakeidentd, false, fbset, fbsplash, fdflush, fdformat,
fdisk, fetchmail, fgrep, find, findfs, fold, free, freeramdisk, fsck, fsck_minix, ftpget, ftpput,
fuser, getenforce, getopt, getsebool, getty, grep, gunzip, gzip, halt, hd, hdparm, head, hexdump,
hostid, hostname, httpd, hush, hwclock, id, ifconfig, ifdown, ifenslave, ifup, inetd, init, inotifyd,
insmod, install, ip, ipaddr, ipcalc, ipcrm, ipcs, iplink, iproute, iprule, iptunnel, kbd_mode, kill,
killall, killall5, klogd, lash, last, length, less, linux32, linux64, linuxrc, ln, load_policy,
loadfont, loadkmap, logger, login, logname, logread, losetup, lpd, lpq, lpr, ls, lsattr, lsmod,
lzmacat, makedevs, man, matchpathcon, md5sum, mdev, mesg, microcom, mkdir, mke2fs, mkfifo, mkfs_
minix, mknod, mkswap, mktemp, modprobe, more, mount, mountpoint, msh, mt, mv, nameif, nc, netstat,
nice, nmeter, nohup, nslookup, od, openvt, parse, passwd, patch, pgrep, pidof, ping, ping6, pipe_
progress, pivot_root, pkill, poweroff, printenv, printf, ps, pscan, pwd, raidautorun, rdate, rdev,
readahead, readlink, readprofile, realpath, reboot, renice, reset, resize, restorecon, rm, rmdir,
rmmod, route, rpm, rpm2cpio, rtcwake, run_parts, runcon, runlevel, runsv, runsvdir, rx, script, sed,
selinuxenabled, sendmail, seq, sestatus, setarch, setconsole, setenforce, setfiles, setfont,
setkeycodes, setlogcons, setsebool, setsid, setuidgid, sh, sha1sum, showkey, slattach, sleep,
softlimit, sort, split, start_stop_daemon, stat, strings, stty, su, sulogin, sum, sv, svlogd, swapoff,
swapon, switch_root, sync, sysctl, syslogd, tac, tail, tar, taskset, tcpsvd, tee, telnet, telnetd,
test, tftp, tftpd, time, top, touch, tr, traceroute, true, tty, ttysize, tune2fs, udhcpc, udhcpd,
udpsvd, umount, uname, uncompress, unexpand, uniq, unix2dos, unlzma, unzip, uptime, usleep, uudecode,
uuencode, vconfig, vi, vlock, watch, watchdog, wc, wget, which, who, whoami, xargs, yes, zcat, zcip

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

284/443

Toolbox

As Busybox is under the GPL, Google developed an equivalent

tool, under the BSD license

Much fewer UNIX commands implemented than Busybox, but

other commands to use the Android-specics mechanism,
such as alarm, getprop or a modied log

Commands available in Toolbox in Jelly Bean

alarm, cat, chcon, chmod, chown, cmp, cp, date, dd, df, dmesg, du, dynarray, exists, getenforce,
getevent, getprop, getsebool, grep, hd, id, ifconfig, iftop, insmod, ioctl, ionice, kill, ln, load_
policy, log, ls, lsmod, lsof, lsusb, md5, mkdir, mount, mv, nandread, netstat, newfs_msdos, notify,
printenv, ps, r, readtty, reboot, renice, restorecon, rm, rmdir, rmmod, rotatefb, route, runcon,
schedtop, sendevent, setconsole, setenforce, setkey, setprop, setsebool, sleep, smd, start, stop,
sync, syren, top, touch, umount, uptime, vmstat, watchprops, wipe

The shell is provided by an external project, mksh, which is a

BSD-licenced implementation of ksh

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

285/443

Android Native Layer

Init

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

286/443

Whole Android Stack

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

287/443

Init

init is the name of the rst user space program

It is up to the kernel to start it, with PID 1, and the program

should never exit during system life

The kernel will look for init at /sbin/init, /bin/init,

/etc/init and /bin/sh. You can tweak that with the init=
kernel parameter

The role of init is usually to start other applications at boot

time, a shell, mount the various lesystems, etc.

Init also manages the shutdown of the system by undoing all

it has done at boot time

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

288/443

Android's init

Once again, Google has developed his own instead of relying

on an existing one.
However, it has some interesting features, as it can also be
seen as a daemon on the system

it manages device hotplugging, with basic permissions rules for

device les, and actions at device plugging and unplugging
it monitors the services it started, so that if they crash, it can
restart them
it monitors system properties so that you can take actions
when a particular one is modied

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

289/443

Init part

For the initialization part, init mounts the various lesystems

(/proc, /sys, data, etc.)

This allows to have an already setup environment before

taking further actions

Once this is done, it reads the init.rc le and executes it

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

290/443

init.rc le interpretation

Uses a unique syntax, based on events

There usually are several init conguration les, the main

init.rc le itself, plus the extra le included from it

By default, these included les hold either subsystem-specic

initialisation (USB, Kernel Tracing), or hardware-specic
instructions

It relies on system properties, evaluated at runtime, that

allows to have on the same system, conguration for several
dierent platforms, that will be used only when they are
relevant.

Most of the customizations should therefore go to the

platform-specic conguration le rather than to the generic
one

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

291/443

Syntax

Unlike most init script systems, the conguration relies on

system event and system property changes, allowed by the
daemon part of it

This way, you can trigger actions not only at startup or at

run-level changes like with traditional init systems, but also at
a given time during system life

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

292/443

Actions

on <trigger>
command
command

Here are a few trigger types:

boot

<property>=<value>

Triggered when init is loaded

Triggered when the given property is set to the given value

device-added-<path>

service-exited-<name>

Triggered when the given device node is added or removed

Triggered when the given service exits

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

293/443

Init triggers

Commands are also specic to Android, with sometimes a

syntax very close to the shell one (just minor dierences):
The complete list of triggers, by execution order is:

early-init
init
early-fs
fs
post-fs
early-boot
boot

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

294/443

Example
import /init.${ro.hardware}.rc
on boot
export PATH /sbin:/system/sbin:/system/bin
export LD_LIBRARY_PATH /system/lib
mkdir /dev
mkdir /proc
mkdir /sys
mount
mkdir
mkdir
mount
mount
mount

tmpfs tmpfs /dev

/dev/pts
/dev/socket
devpts devpts /dev/pts
proc proc /proc
sysfs sysfs /sys

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

295/443

Services

service <name> <pathname> [ <argument> ]*

<option>
<option>

Services are like daemons

They are started by init, managed by it, and can be restarted

when they exit

Many options, ranging from which user to run the service as,
rebooting in recovery when the service crashes too frequently,
to launching a command at service reboot.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

296/443

Example

on device-added-/dev/compass
start akmd
on device-removed-/dev/compass
stop akmd
service akmd /sbin/akmd
disabled
user akmd
group akmd

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

297/443

Uevent

Init also manages the runtime events generated by the kernel

when hardware is plugged in or removed, like udev does on a
standard Linux distribution

This way, it dynamically creates the devices nodes under /dev

You can also tweak its behavior to add specic permissions to

the les associated to a new event.

The associated conguration les are /ueventd.rc and

/ueventd.<platform>.rc

While ueventd.rc is always taken into account,

ueventd.<platform>.rc is only interpreted if the platform
currently running the system reports the same name

This name is either obtained by reading the le

/proc/cpuinfo or from the androidboot.hardware kernel
parameter

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

298/443

ueventd.rc syntax

<path>

<permission>

<user>

<group>

Example

/dev/bus/usb/*

0660

root

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

usb

299/443

Properties

Init also manages the system properties

Properties are a way used by Android to share values across

the system that are not changing quite often

Quite similar to the Windows Registry

These properties are splitted into several les:

/system/build.prop which contains the properties generated

by the build system, such as the date of compilation
/default.prop which contains the default values for certain
key properties, mostly related to the security and permissions
for ADB.
/data/local.prop which contains various properties specic
to the device
/data/property is a folder which purpose is to be able to edit
properties at run-time and still have them at the next reboot.
This folder is storing every properties prexed by persist. in
separate les containing the values.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

300/443

Modifying Properties

You can add or modify properties in the build system by using

either the PRODUCT_PROPERTY_OVERRIDES makele variable, or
by dening your own system.prop le in the device directory.
Their content will be appended to /system/build.prop at
compilation time

Modify the init.rc le so that at boot time it exports these

properties using the setprop command

Using the API functions such as the Java function

SystemProperties.set

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

301/443

Permissions on the Properties

Android, by default, only allows any given process to read the

properties.
You can set write permissions on a particular property or a group of
them using the le system/core/init/property_service.c

/* White list of permissions for setting property services. */

struct {
const char *prefix;
unsigned int uid;
unsigned int gid;
} property_perms[] = {
AID_RADIO,
0 },
{ "net.rmnet0.",
{ "net.dns",
AID_RADIO,
0 },
{ "net.",
AID_SYSTEM, 0 },
{ "dhcp.",
AID_SYSTEM, 0 },
AID_SHELL,
0 },
{ "log.",
{ "service.adb.root", AID_SHELL,
0 },
{ "persist.security.", AID_SYSTEM,
0 },
{ NULL, 0, 0 }
};
free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

302/443

Special Properties

ro.* properties are read-only. They can be set only once in

the system life-time. You can only change their value by
modifying the property les and reboot.

persist.* properties are stored on persistent storage each

time they are set.

ctl.start and ctl.stop properties used instead of storing

properties to start or stop the service name passed as the new
value

net.change property holds the name of the last net.*

property changed.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

303/443

Android Native Layer

Various daemons

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

304/443

Whole Android Stack

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

305/443

Vold

The VOLume Daemon

Just like init does, monitors new device events

While init was only creating device les and taking some
congured options, vold actually only cares about storage
devices
Its roles are to:

Auto-mount the volumes

Format the partitions on the device

There is no /etc/fstab in Android, but

/system/etc/vold.fstab has a somewhat similar role

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

306/443

rild

rild is the Radio Interface Layer Daemon

This daemon drives the telephony stack, both voice and data
communication

When using the voice mode, talks directly to the baseband,

but when issuing data transfers, relies on the kernel network
stack
It can handle two types of commands:

Solicited commands: commands that originate from the user:

dial a number, send an SMS, etc.
Unsolicited commands: commands that come from the
baseband: receiving an SMS, a call, signal strength changed,
etc.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

307/443

Others

netd

netd manages the various network connections: Bluetooth,

Wi, USB
Also takes any associated actions: detect new connections, set
up the tethering, etc.
It really is an equivalent to NetworkManager
On a security perspective, it also allows to isolate
network-related privileges in a single process

installd

Handles package installation and removal

Also checks package integrity, installs the native libraries on
the system, etc.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

308/443

Android Native Layer

SurfaceFlinger

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

309/443

Introduction to graphical stacks

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

310/443

Compositing window managers

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

311/443

SurfaceFlinger

This dierence in design adds some interesting features:

Eects are easy to implement, as it's up to the window

manager to mangle the various surfaces at will to display them
on the screen. Thus, you can add transparency, 3d eects, etc.
Improved stability. With a regular window manager, a message
is sent to every window to redraw its part of the screen, for
example when a window has been moved. But if an application
fails to redraw, the windows will become glitchy. This will not
happen with a compositing WM, as it will still display the
untouched surface.

SurfaceFlinger is the compositing window manager in

Android, providing surfaces to applications and rendering all
of them with hardware acceleration.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

312/443

SurfaceFlinger

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

313/443

Android Native Layer

Stagefright

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

314/443

Stagefright

StageFright is the multimedia playback engine in Android

since Eclair

In its goals, it is quite similar to GStreamer: Provide an

abstraction on top of codecs and libraries to easily play
multimedia les

It uses a plugin system, to easily extend the number of

formats supported, either software or hardware decoded

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

315/443

StageFright Architecture

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

316/443

StageFright plugins

To add support for a new format, you need to:

Develop a new Extractor class, if the container is not

supported yet.
Develop a new Decoder class, that implements the interface
needed by the StageFright core to read the data.
Associate the mime-type of the les to read to your new
Decoder in the /etc/media_codecs.xml le provided by your
device, either in the Decoders list.

No runtime extension of the decoders, this is done at

compilation time.

<Decoders>
<MediaCodec name="OMX.google.vorbis.decoder" type="audio/vorbis" />
<MediaCodec name="OMX.qcom.video.decoder.avc" type="video/avc" />
</Decoders>

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

317/443

Android Native Layer

Dalvik and Zygote

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

318/443

Whole Android Stack

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

319/443

Dalvik

Dalvik is the virtual machine, executing Android applications

It is an interpreter written in C/C++, and is designed to be

portable, lightweight and run well on mobile devices

It is also designed to allow several instances of it to be run at

the same time while consuming as little memory as possible
Two execution modes

portable: the interpreter is written in C, quite slow, but

should work on all platforms
fast: Uses the mterp mechanism, to dene routines either in
assembly or in C optimized for a specic platform. Instruction
dispatching is also done by computing the handler address
from the opcode number

It uses the Apache Harmony Java framework for its core

libraries

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

320/443

Zygote

Dalvik is started by Zygote

frameworks/base/cmds/app_process
At boot, Zygote is started by init, it then

Initializes a virtual machine in its address space

Loads all the basic Java classes in memory
Starts the system server
Waits for connections on a UNIX socket

When a new application should be started:

Android connects to Zygote through the socket to request the

start of a new application
Zygote forks
The child process loads the new application and start
executing it

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

321/443

Android Native Layer

Hardware Abstraction Layer

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

322/443

Whole Android Stack

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

323/443

Hardware Abstraction Layers

Usually, the kernel already provides a HAL for user space

However, from Google's point of view, this HAL is not
sucient and suers some restrictions, mostly:

Depending on the subsystem used in the kernel, the user space

interface diers
All the code in the kernel must be GPL-licensed

Google implemented its HAL with dynamically loaded user

space libraries

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

324/443

Library naming

It follows the same naming scheme as for init: the generic

implementation is called libfoo.so and the hardware-specic
one libfoo.hardware.so
The name of the hardware is looked up with the following
properties:

ro.hardware
ro.product.board
ro.board.platform
ro.arch

The libraries are then searched for in the directories:

/vendor/lib/hw
/system/lib/hw

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

325/443

Various layers

Audio (libaudio.so) conguration, mixing, noise

cancellation, etc.

Graphics (gralloc.so, hwcomposer.so, libhgl.so) handles

graphic memory buer allocations, OpenGL implementation,
etc.

libhgl.so should be provided by your vendor

hardware/libhardware/include/gralloc.h
hardware/libhardware/include/hwcomposer.h

Camera (libcamera.so) handles the camera functions:

autofocus, take a picture, etc.

hardware/libhardware/include/audio.h

hardware/libhardware/include/camera{2,3}.h

GPS (libgps.so) conguration, data acquisition

hardware/libhardware/include/hardware/gps.h

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

326/443

Various layers

Lights (liblights.so) Backlight and LEDs management

Sensors (libsensors.so) handles the various sensors on the

device: Accelerometer, Proximity Sensor, etc.

You can set the name of the library with the rild.lib and
rild.libargs properties to nd the library
hardware/ril/include/telephony/ril.h

Bluetooth (libbluetooth.so) Discovery and communication

with Bluetooth devices

hardware/libhardware/include/sensors.h

Radio Interface (libril-vendor-version.so) manages all

communication between the baseband and rild

hardware/libhardware/include/lights.h

hardware/libhardware/include/bluetooth.h

NFC (libnfc.so) Discover NFC devices, communicate with

it, etc.

hardware/libhardware/include/nfc.h

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

327/443

Example: rild

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

328/443

Android Native Layer

JNI

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

329/443

Whole Android Stack

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

330/443

What is JNI?

A Java framework to call and be called by native applications

written in other languages
Mostly used for:

Writing Java bindings to C/C++ libraries

Accessing platform-specic features
Writing high-performance sections

It is used extensively across the Android user space to interface

between the Java Framework and the native daemons

Since Gingerbread, you can develop apps in a purely native

way, possibly calling Java methods through JNI

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

331/443

C Code

#include "jni.h"
JNIEXPORT void JNICALL Java_com_example_Print_print(JNIEnv *env,
jobject obj,
jstring javaString)
{
const char *nativeString = (*env)->GetStringUTFChars(env,
javaString,
0);
printf("%s", nativeString);
(*env)->ReleaseStringUTFChars(env, javaString, nativeString);
}

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

332/443

JNI arguments

Function prototypes are following the template:

JNIEXPORT jstring JNICALL Java_ClassName_MethodName
(JNIEnv*, jobject)

JNIEnv is a pointer to the JNI Environment that we will use

to interact with the virtual machine and manipulate Java
objects within the native methods

jobject contains a pointer to the calling object. It is very

similar to this in C++

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

333/443

Types

There is no direct mapping between C Types and JNI types

You must use the JNI primitives to convert one to his
equivalent
However, there are a few types that are directly mapped, and
thus can be used directly without typecasting:
Native Type
unsigned char
signed char
unsigned short
short
long
long long
oat
double

JNI Type
jboolean
jbyte
jchar
jshort
jint
jlong
joat
jdouble

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

334/443

Java Code
package com.example;
class Print
{
private static native void print(String str);
public static void main(String[] args)
{
Print.print("HelloWorld!");
}
static
{
System.loadLibrary("print");
}
}

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

335/443

Calling a method of a Java object from C

JNIEXPORT void JNICALL Java_ClassName_Method(JNIEnv *env,

jobject obj)
{
jclass cls = (*env)->GetObjectClass(env, obj);
jmethodID hello = (*env)->GetMethodID(env,
cls,
"hello",
"(V)V");
if (!hello)
return;
(*env)->CallVoidMethod(env, obj, hello);
}

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

336/443

Instantiating a Java object from C

JNIEXPORT jobject JNICALL Java_ClassName_Method(JNIEnv *env,

jobject obj)
{
jclass cls = env->FindClass("java/util/ArrayList");
jmethodID init = env->GetMethodID(cls,
"<init>",
"()V");
jobject array = env->NewObject(cls, init);
return array;
}

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

337/443

Practical lab - Develop a JNI library

Develop bindings from Java to C

Integrate these bindings into the

build system

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

338/443

Android Framework and Applications

Android
Framework and
Applications

Embedded Linux
Experts

free electrons
Copyright 2004-2016, Free Electrons.
Creative Commons BY-SA 3.0 license.
Corrections, suggestions, contributions and translations are welcome!

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

339/443

Android Framework and Applications

Service Manager and Various Services

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

340/443

Whole Android Stack

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

341/443

System Server boot

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

342/443

The rst step: system_server.c

Located in frameworks/base/cmds/system_server

Started by Zygote through the SystemServer

Starts all the various native services:

SurfaceFlinger
SensorService

It then calls back the SystemServer object's init2 function to

go on with the initialization

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

343/443

Java Services Initialization

Located in frameworks/base/services/java/com/android/
server/SystemServer.java

Starts all the dierent Java services in a dierent thread by

registering them into the Service Manager

PowerManager, ActivityManager (also handles the

ContentProviders), PackageManager, BatteryService,
LightsService, VibratorService, AlarmManager,
WindowManager, BluetoothService, DevicePolicyManager,
StatusBarManager, InputMethodManager,
ConnectivityService, MountService,
NotificationManager, LocationManager, AudioService,
...

If you wish to add a new system service, you will need to add
it to one of these two parts to register it at boot time

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

344/443

Android Framework and Applications

Inter-Process Communication, Binder

and AIDLs

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

345/443

Whole Android Stack

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

346/443

IPCs

On modern systems, each process has its own address space,

allowing to isolate data
This allows for better stability and security: only a given process can
access its address space. If another process tries to access it, the
kernel will detect it and kill this process.
However, interactions between processes are sometimes needed,
that's what IPCs are for.
On classic Linux systems, several IPC mechanisms are used:
Signals
Semaphores
Sockets
Message queues
Pipes
Shared memory
Android, however, uses mostly:
Binder
Ashmem and Sockets

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

347/443

Binder 1/2

Uses shared memory for high performance

Uses reference counting to garbage collect objects no longer in

use

Data are sent through parcels, which is some kind of

serialization

Used across the whole system, e.g., clients connect to the

window manager through Binder, which in turn connects to
SurfaceFlinger using Binder

Each object has an identity, which does not change, even if

you pass it to other processes.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

348/443

Binder 2/2

This is useful if you want to separate components in distinct

processes, or to manage several components of a single
process (i.e. Activity's Windows).

Object identity is also used for security. Some token passed

correspond to specic permissions. Another security model to
enforce permissions is for every transaction to check on the
calling UID.

Binder also supports one-way and two-way messages

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

349/443

Binder Mechanism

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

350/443

Binder Implementation 1/2

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

351/443

Binder Implementation 2/2

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

352/443

Android Interface Denition Language (AIDL)

Very similar to any other Interface Denition Language you

might have encountered

Describes a programming interface for the client and the

server to communicate using IPCs
Looks a lot like Java interfaces. Several types are already
dened, however, and you can't extend this like what you can
do in Java:

All Java primitive types (int, long, boolean, etc.)

String
CharSequence
Parcelable
List of one of the previous types
Map

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

353/443

AIDLs HelloWorld

package com.example.android;
interface IRemoteService {
void HelloPrint(String aString);
}

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

354/443

Parcelable Objects

If you want to add extra objects to the AIDLs, you need to

make them implement the Parcelable interface

Most of the relevant Android objects already implement this

interface.

This is required to let Binder know how to serialize and

deserialize these objects

However, this is not a general purpose serialization

mechanism. Underlying data structures may evolve, so you
should not store parcelled objects to persistent storage

Has primitives to store basic types, arrays, etc.

You can even serialize le descriptors!

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

355/443

Implement Parcelable Classes

To make an object parcelable, you need to:

Make the object implement the Parcelable interface

Implement the writeToParcel function, which stores the
current state of the object to a Parcel object
Add a static eld called CREATOR, which implements the
Parcelable.Creator interface, and takes a Parcel,
deserializes the values and returns the object
Create an AIDL le that declares your new parcelable class

You should also consider Bundles, that are type-safe key-value

containers, and are optimized for reading and writing values

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

356/443

Intents

Intents are a high-level use of Binder

They describe the intention to do something

They are used extensively across Android

Activities, Services and BroadcastReceivers are started using

intents

Two types of intents:

explicit The developer designates the target by its name
implicit There is no explicit target for the Intent. The
system will nd the best target for the Intent by
itself, possibly asking the user what to do if
there are several matches

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

357/443

Android Framework and Applications

Various Java Services

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

358/443

Whole Android Stack

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

359/443

Android Java Services

There are lots of services implemented in Java in Android

They abstract most of the native features to make them

available in a consistent way

You get access to the system services using the

Context.getSystemService() call

You can nd all the accessible services in the documentation

for this function

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

360/443

ActivityManager

Manages everything related to Android applications

Starts Activities and Services through Zygote

Manages their lifecycle
Fetches content exposed through content providers
Dispatches the implicit intents
Adjusts the Low Memory Killer priorities
Handles non responding applications

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

361/443

PackageManager

Exposes methods to query and manipulate already installed

packages, so you can:

Get the list of packages

Get/Set permissions for a given package
Get various details about a given application (name, uids, etc)
Get various resources from the package

You can even install/uninstall an apk

installPackage/uninstallPackage functions are hidden in

the source code, yet public.
You can't compile code that is calling directly these functions
and they are not documented anywhere except in the code
But you can call them through the Java Reflection API, if
you have the proper permissions of course

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

362/443

PowerManager

Abstracts the Wakelocks functionality

Denes several states, but when a wakelock is grabbed, the
CPU will always be on

PARTIAL_WAKE_LOCK

Only the CPU is on, screen and keyboard backlight are o

SCREEN_DIM_WAKE_LOCK

SCREEN_BRIGHT_WAKE_LOCK

Screen backlight is partly on, keyboard backlight is o

Screen backlight is on, keyboard backlight is o

FULL_WAKE_LOCK

Screen and keyboard backlights are on

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

363/443

AlarmManager

Abstracts the Android timers

Allows to set a one time timer or a repetitive one

When a timer expires, the AlarmManager grabs a wakelock,

sends an Intent to the corresponding application and releases
the wakelock once the Intent has been handled

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

364/443

ConnectivityManager and WiManager

ConnectivityManager

Manages the various network connections

Falls back to other connections when one fails

Noties the system when one becomes available/unavailable
Allows the applications to retrieve various information about
connectivity

WiManager

Provides an API to manage all aspects of WiFi networks

List, modify or delete already congured networks

Get information about the current WiFi network if any
List currently available WiFi networks
Sends Intents for every change in WiFi state

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

365/443

Example: Vibrator Service

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

366/443

Android Framework and Applications

Extend the framework

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

367/443

Why extend it?

You might want to extend the existing Android framework to

add new features or allow other applications to use specic
devices available on your hardware

As you have the code, you could just hack the source to make
the framework suit your needs
This is quite problematic however:

You might break the API, introduce bugs, etc

Google requires you not to modify the Android public API
It is painful to track changes across the tree, to port the
changes to new versions
You don't always want to have such extensions for all your
products

As usual with Android, there's a device-specic way of

extending the framework: PlatformLibraries

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

368/443

PlatformLibraries

The modications are just plain Java libraries

You can declare any namespace you want, do whatever code
you want.
However, they are bundled as raw Java archives, so you
cannot embed resources in the modications
If you would still do this, you can add them to
frameworks/base/res, but you have to hide them
When using the Google Play Store, all the libraries including
these ones are submitted to Google, so that it can lter out
apps relying on libraries not available on your system
To avoid any application to link to any jar le, you have to
declare both in your application and in your library that you
will use and add a custom library
The library's xml permission le should go into the
/system/etc/permissions folder

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

369/443

PlatformLibrary Makele

LOCAL_PATH := $(call my-dir)

include $(CLEAR_VARS)
LOCAL_SRC_FILES := \
$(call all-subdir-java-files)
LOCAL_MODULE_TAGS := optional
LOCAL_MODULE:= com.example.android.pl
include $(BUILD_JAVA_LIBRARY)

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

370/443

PlatformLibrary permissions le

<?xml version="1.0" encoding="utf-8"?>

<permissions>
<library name="com.example.android.pl"
file="/system/framework/com.example.android.pl.jar"/>
</permissions>

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

371/443

PlatformLibrary Client Makele

LOCAL_PATH:= $(call my-dir)

include $(CLEAR_VARS)
LOCAL_MODULE_TAGS := optional
LOCAL_PACKAGE_NAME := PlatformLibraryClient
LOCAL_SRC_FILES := $(call all-java-files-under, src)
LOCAL_JAVA_LIBRARIES := com.example.android.pl
include $(BUILD_PACKAGE)

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

372/443

Practical lab - Develop a Framework Component

Modify the Android framework

Use JNI bindings

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

373/443

Android Application Development

Android
Application
Development

Embedded Linux
Experts

free electrons
Copyright 2004-2016, Free Electrons.
Creative Commons BY-SA 3.0 license.
Corrections, suggestions, contributions and translations are welcome!

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

374/443

Android Application Development

Basics

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

375/443

Whole Android Stack

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

376/443

Android applications

Android applications are written mostly in Java using Google's

SDK

Applications are bundled into an Android PacKage (.apk

les) which are archives containing the compiled code, data
and resources for the application, so applications are
completely self-contained

You can install applications either through a market (Google

Play Store, Amazon Appstore, F-Droid, etc) or manually
(through ADB or a le manager)

Of course, everything we have seen so far is mostly here to

provide a nice and unied environment to application
developers

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

377/443

Applications Security

Once installed, applications live in their own sandbox, isolated

from the rest of the system
The system assigns a Linux user to every application, so that
every application has its own user/group
It uses this UID and les permissions to allow the application
to access only its own les
Each process has its own instance of Dalvik, so code is
running isolated from other applications
By default, each application runs in its own process, which
will be started/killed during system life
Android uses the principle of least privilege. Each application
by default has only access to what it requires to work.
However, you can request extra permissions, make several
applications run in the same process, or with the same UID,
etc.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

378/443

Applications Components

Components are the basic blocks of each application

You can see them as entry points for the system in the
application
There is four types of components:

Activities
Broadcast Receivers
Content Providers
Services

Every application can start any component, even located in

other applications. This allows to share components easily,
and have very little duplication. However, for security reasons,
you start it through an Intent and not directly
When an application requests a component, the system starts
the process for this application, instantiates the needed class
and runs that component. We can see that there is no single
point of entry in an application like main()

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

379/443

Application Manifest

To declare the components present in your application, you

have to write a XML le, AndroidManifest.xml
This le is used to:

Declare available components

Declare which permissions these components need
Revision of the API needed
Declare hardware features needed
Libraries required by the components

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

380/443

Manifest HelloWorld

<?xml version="1.0" encoding="utf-8"?>

<manifest package="com.example.android">
<application>
<activity android:name=".ExampleActivity"
android:label="@string/example_label">
<intent-filter>
<action android:name="android.intent.action.MAIN"/>
<category android:name="android.intent.category.LAUNCHER"/>
</intent-filter>
</activity>
<uses-library android:name="com.example.android.pl" />
</application>
</manifest>

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

381/443

NDK

Google also provides a NDK to allow developers to write

native code

While the code is not run by Dalvik, the security guarantees

are still there

Allows to write faster code or to port existing C code to

Android more easily

Since Gingerbread, you can even code a whole application

without writing a single line of Java

It is still packaged in an apk, with a manifest, etc.

However, there are some drawbacks, the main one being that
you can't access the resources mechanism available from Java

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

382/443

Android Application Development

Activities

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

383/443

Activities

Activities are a single screen of the user interface of an

application
They are assembled to provide a consistent interface. If we
take the example of an email application, we will have:

An activity listing the received mails

An activity to compose a new mail
An activity to read a mail

Other applications might need one of these activities. To

continue with this example, the Camera application might
want to start the composing activity to share the just-shot
picture
It is up to the application developer to advertise available
activities to the system
When an activity starts a new activity, the latter replaces the
former on the screen and is pushed on the back stack which
holds the last used activities, so when the user is done with
the newer activity, it can easily go back to the previous one

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

384/443

Back Stack

Credits: http://developer.android.com

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

385/443

Back Stack

Credits: http://developer.android.com

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

386/443

Activity Lifecycle 1/3

As there is no single entry point and as the system manages

the activities, activities have to dene callbacks that the
system can call at some point in time

Activities can be in one of the three states on Android

Running The activity is on the foreground and has focus
Paused The activity is still visible on the screen but no
longer has focus. It can be destroyed by the
system under very heavy memory pressure
Stopped The activity is no longer visible on the screen. It
can be killed at any time by the system

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

387/443

Activity Lifecycle 2/3

There are callbacks for every change from one of these states
to another

The most important ones are onCreate and onPause

All components of an application run in the same thread. If

you do long operations in the callbacks, you will block the
entire application (UI included). You should always use
threads for every long-running task.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

388/443

Activity Lifecycle 3/3

Credits: http://developer.android.com
free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

389/443

Saving Activity State 1/2

As applications tend to be killed and restarted quite often, we

need a way to store our internal state when killed and reload
it when restarted

Once again, this is done through callbacks

Before killing the application, the system calls the

onSaveInstanceState callback and when restarting it, it calls
onRestoreInstanceState

In both cases, it provides a Bundle as argument to allow the

activity to store what's needed and reload it later, with little
overhead

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

390/443

Saving Activity State 2/2

This make the creation/suppression of activities awless for

the user, while allowing to save as much memory as we need

These callbacks are not always called though. If the activity is

killed because the user left it in a permanent way (through the
back button), it won't be called

By default, these activities are also called when rotating the

device, because the activity will be killed and restarted by the
system to load new resources

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

391/443

Activity Lifecycle

Credits: http://developer.android.com

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

392/443

Activity Callbacks

Credits: http://developer.android.com

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

393/443

Activity HelloWorld
public class ExampleActivity extends Activity {
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.example);
Log.i("ExampleActivity", "Activity created!");
}
protected void onStart() {
super.onStart();
}
protected void onResume() {
super.onResume();
}
protected void onPause() {
super.onPause();
}
protected void onStop() {
super.onStop();
}
protected void onDestroy() {
super.onDestroy();
}
}
free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

394/443

Android Application Development

Services

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

395/443

Services

Services are components running in the background

They are used either to perform long running operations or to

work for remote processes

A service has no user interface, as it is supposed to run when

the user does something else

From another component, you can either work with a service

in a synchronous way, by binding to it, or asynchronous, by
starting it

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

396/443

Service Manifest

<?xml version="1.0" encoding="utf-8"?>

<manifest package="com.example.android">
<application>
<service android:name=".ExampleService"/>
</application>
</manifest>

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

397/443

Services Types

We can see services as a set including:

Started Services, that are created when other components call

startService. Such a service runs as long as needed, whether
the calling component is still alive or not, and can stop itself or
be stopped. When the service is stopped, it is destroyed by the
system

You can also subclass IntentService to have a started

service. However, while much easier to implement, this service
will not handle multiple requests simultaneously.

Bound Services, that are bound to by other components by

calling bindService. They oer a client/server like interface,
interacting with each other. Multiple components can bind to
it, and a service is destroyed only when no more components
are bound to it

Services can be of both types, given that callbacks for these

two do not overlap completely
Services are started by passing Intents either to the
startService or bindService commands

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

398/443

Services Lifecycle

Credits: http://developer.android.com
free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

399/443

Bound Services

There are three possible ways to implement a bound service:

By extending the Binder class. It works only when the clients

are local and run in the same process though.
By using a Messenger, that will provide the interface for your
service to remote processes. However, it does not perform
multi-threading, all requests are queued up.
By writing your own AIDL le. You will then be able to
implement your own interface and write thread-safe code, as
you are very likely to receive multiple requests at once

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

400/443

Bound Services and Started Lifecycle

Credits: http://developer.android.com
free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

401/443

Android Application Development

Content Providers

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

402/443

Content Providers

They provide access to organized data in a manner quite

similar to relational databases

They allow to share data with both internal and external

components and centralize them

Security is also enforced by permissions like usual, but they

also do not allow remote components to issue arbitrary
requests like what we can do with relational databases

Instead, Content Providers rely on URIs to allow for a

restricted set of requests with optional parameters, only
permitting the user to lter by values and by columns

You can use any storage back-end you want, while exposing a
quite neutral and consistent interface to other applications

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

403/443

Content URIs

URIs are often built with the following pattern:

content://<package>.provider/<path> to access particular

tables
content://<package>.provider/<path>/<id> to access
single rows inside the given table

Facilities are provided to deal with these

On the application side:

ContentUri to append and manage numerical IDs in URIs

Uri.Builder and Uri classes to deal with URIs and strings

On the provider side:

UriMatcher associates a pattern to an ID, so that you can

easily match incoming URIs, and use switch over them.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

404/443

Implementing a Content Provider

public class ExampleProvider extends ContentProvider {

private static final UriMatcher sUriMatcher;
static {
sUriMatcher.addURI("com.example.android.provider", "table1", 1);
sUriMatcher.addURI("com.example.android.provider", "table1/#", 2);
}
public Cursor query(Uri uri, String[] projection, String selection,
String[] selectionArgs, String sortOrder) {
switch (sUriMatcher.match(uri)) {
default:
System.out.println("Hello World!");
break;
}
}

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

405/443

Implementing a Content Provider

public Uri insert(Uri uri, ContentValues values) {

return null;
}
public int update(Uri uri, ContentValues values, String selection,
String[] selectionArgs) {
return 0;
}
public int delete(Uri uri, String selection, String[] selectionArgs) {
return 0;
}
public boolean onCreate() {
return true;
}
}

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

406/443

Android Application Development

Managing the Intents

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

407/443

Intents

Intents are basically a bundle of several pieces of information,

mostly

Component Name

Action

Data

The action to perform or that has been performed

The data to act upon, written as a URI, like
tel://0123456789

Category

Contains both the full class name of the target component

plus the package name dened in the Manifest

Contains additional information about the nature of the

component that will handle the intent, for example the
launcher or a preference panel

The component name is optional. If it is set, the intent will

be explicit. Otherwise, the intent will be implicit

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

408/443

Intent Resolution

When using explicit intents, dispatching is quite easy, as the

target component is explicitly named. However, it is quite rare
that a developer knows the component name of external
applications, so it is mostly used for internal communication.

Implicit intents are a bit more tricky to dispatch. The system

must nd the best candidate for a given intent.

To do so, components that want to receive intents have to

declare them in their manifests Intent lters, so that the
system knows what components it can respond to.

Components without intent lters will never receive implicit

intents, only explicit ones

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

409/443

Intent Filters 1/2

They are only about notifying the system about handled

implicit intents
Filters are based on matching by category, action and data.
Filtering by only one of these three (by category for example)
is ne.

A lter can list several actions. If an intent action eld

corresponds to one of the actions listed here, the intent will
match
It can also list several categories. However, if none of the
categories of an incoming intent are listed in the lter, then
intent won't match.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

410/443

Intent Filters 2/2

You can also use intent matching from your application by

using the query* methods from the PackageManager to get a
matching component from an Intent.

For example, the launcher application does that to display

only activities with lters that specify the category
android.intent.category.LAUNCHER and the action
android.intent.action.MAIN

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

411/443

Real Life Manifest Example: Notepad

<manifest package="com.example.android.notepad">
<application android:icon="@drawable/app_notes"
android:label="@string/app_name" >
<activity android:name="NotesList"
android:label="@string/title_notes_list">
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
<intent-filter>
<action android:name="android.intent.action.VIEW" />
<action android:name="android.intent.action.EDIT" />
<action android:name="android.intent.action.PICK" />
<category android:name="android.intent.category.DEFAULT" />
<data android:mimeType="vnd.android.cursor.dir/vnd.google.note" />
</intent-filter>
</activity>
</application>
</manifest>

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

412/443

Broadcasted intents

Intents can also be broadcast thanks to two functions:

sendBroadcast that broadcasts an intent that will be handled

by all its handlers at the same time, in an undened order
sendOrderedBroadcast broadcasts an intent that will be
handled by one handler at a time, possibly with propagation of
the result to the next handler, or the possibility for a handler
to cancel the broadcast

Broadcasts are used for system wide notication of important

events: booting has completed, a package has been removed,
etc.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

413/443

Broadcast Receivers

Broadcast receivers are the fourth type of components that

can be integrated into an application. They are specically
designed to deal with broadcast intents.

Their overall design is quite easy to understand: there is only

one callback to implement: onReceive

The lifecycle is quite simple too: once the onReceive callback

has returned, the receiver is considered no longer active and
can be destroyed at any moment

Thus you must not use asynchronous calls (Bind to a service

for example) from the onReceive callback, as there is no way
to be sure that the object calling the callback will still be alive
in the future.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

414/443

Android Application Development

Processes and Threads

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

415/443

Process Management in Android

By default in Android, every component of a single

application runs in the same process.
When the system wants to run a new component:

If the application has no running component yet, the system

will start a new process with a single thread of execution in it
Otherwise, the component is started within that process

If you happen to want a component of your application to run

in its own process, you can still do it through the
android:process XML attribute in the manifest.

When the memory constraints are high, the system might

decide to kill a process to get some memory back. This is done
based on the importance of the process to the user. When a
process is killed, all the components running inside are killed.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

416/443

Processes priority

Foreground processes have the topmost priority. They host

either

Visible processes host

An activity the user is interacting with

A service bound to such an activity
A service running in the foreground (started with
startForeground)
A service running one of its lifecycle callbacks
A broadcast receiver running its onReceive method
An activity that is no longer in the foreground but still is
visible on the screen
A service that is bound to a visible activity

Service Processes host a service that has been started by

startService
Background Processes host activities that are no longer visible
to the user
Empty Processes

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

417/443

Threads

As there is only one thread of execution, both the application

components and UI interactions are done in sequential order

So a long computation, I/O, background tasks cannot be run

directly into the main thread without blocking the UI

If your application is blocked for more than 5 seconds, the

system will display an ``Application Not Responding'' dialog,
which leads to poor user experience

Moreover, UI functions are not thread-safe in Android, so you

can only manipulate the UI from the main thread.
So, you should:

Dispatch every long operation either to a service or a worker

thread
Use messages between the main thread and the worker threads
to interact with the UI.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

418/443

Threads in Android

There are two ways of implementing worker threads in

Android:

Use the standard Java threads, with a class extending

Runnable

This works, of course, but you will need to do messaging

between your worker thread and the main thread, either
through handlers or through the View.post function

Use Android's AsyncTask

A class that has four callbacks: doInBackground,

onPostExecute, onPreExecute, onProgressUpdate
Useful, because only doInBackground is called from a worker
thread, others are called by the UI thread

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

419/443

Android Application Development

Resources

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

420/443

Applications Resources

Applications contain more than just compiled source code:

images, videos, sound, etc.

In Android, anything related to the visual appearance of the

application is kept separate from the source code: activities
layout, animations, menus, strings, etc.

Resources should be kept in the res/ directory of your

application.

At compilation, the build tool will create a class R, containing

references to all the available resources, and associating an ID
to it

This mechanism allows you to provide several alternatives to

resources, depending on locales, screen size, pixel density, etc.
in the same application, resolved at runtime.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

421/443

Resources Directory

All resources are located in the res/ subdirectory

anim/ contains animation denitions

color/ contains the color denitions
drawable/ contains images, "9-patch" graphics, or XML-les
dening drawables (shapes, widgets, relying on a image le)
layout/ contains XML dening applications layout
menu/ contains XML les for the menu layouts
raw/ contains les that are left untouched
values/ contains strings, integers, arrays, dimensions, etc
xml/ contains arbitrary XML les

All these les are accessed by applications through their IDs.

If you still want to use a le path, you need to use the
assets/ folders

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

422/443

Resources

Credits: http://developer.android.com
free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

423/443

Alternative Resources

Alternative resources are provided using extended sub-folder

names, that should be named using the pattern
<folder_name>-<qualifier>
There is a number of qualiers, depending on which case you
want to provide an alternative for. The most used ones are
probably:

locales (en, fr, fr-rCA, ...)

screen orientation (land, port)
screen size (small, large,...)
screen density (mdpi, ldpi, ...)
and much others

You can specify multiple qualiers by chaining them,

separated by dashes. If you want layouts to be applied only
when on landscape on high density screens, you will save them
into the directory layout-land-hdpi

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

424/443

Resources Selection

Credits: http://developer.android.com
free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

425/443

Android Application Development

Data Storage

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

426/443

Data Storage on Android

An application might need to write to arbitrary les and read

from them, for caching purposes, to make settings persistent,
etc.

But the system can't just let you read and write to any
random le on the system, this would be a major security aw

Android provides some mechanisms to address the two

following concerns: allow an application to write to les, while
integrating it into the Android security model
There are four major mechanisms:

Preferences
Internal data
External data
Databases

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

427/443

Shared Preferences

Shared Preferences allows to store and retrieve data in a

persistent way

They are stored using key-value pairs, but can only store basic
types: int, oat, string, boolean

They are persistent, so you don't have to worry about them

disappearing when the activity is killed

You can get an instance of the class managing the preferences

through the function getPreferences

You may also want several set of preferences for your

application and the function getSharedPreferences for that

You can edit them by calling the method edit on this

instance. Don't forget to call commit when you're done!

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

428/443

Internal Storage

You can also save les directly to the internal storage device

These les are not accessible by default by other applications

Such les are deleted when the user removes the application

You can request a FileOutputStream class to such a new le

by calling the method openFileOutput

You can pass extra ags to this method to either change the
way the le is opened or its permissions

These les will be created at runtime. If you want to have

les at compile time, use resources instead

You can also use internal storage for caching purposes. To do

so, call getCacheDir that will return a File object allowing
you to manage the cache folder the way you want to. Cache
les may be deleted by Android when the system is low on
internal storage.

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

429/443

External Storage

External storage is either the SD card or an internal storage

device
Each le stored on it is world-readable, and the user has direct
access to it, since that is the device exported when USB mass
storage is used.
Since this storage may be removable, your application should
check for its presence, and that it behaves correctly
You can either request a sub-folder created only for your
application using the getExternalFilesDir method, with a
tag giving which type of les you want to store in this
directory. This folder will be removed at un-installation.
Or you can request a public storage space, shared by all
applications, and never removed by the system, using
getExternalStoragePublicDirectory
You can also use it for caching, with getExternalCacheDir

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

430/443

SQLite Databases

Databases are often abstracted by Content Providers, that will

abstract requests, but Android adds another layer of
abstraction
Databases are managed through subclasses of
SQLiteOpenHelper that will abstract the structure of the
database
It will hold the requests needed to build the tables, views,
triggers, etc. from scratch, as well as requests to migrate to a
newer version of the same database if its structure has to
evolve.
You can then get an instance of SQLiteDatabase that allows
to query the database
Databases created that way will be only readable from your
application, and will never be automatically removed by the
system
You can also manipulate the database using the sqlite3
command in the shell

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

431/443

Android Application Development

Android Packages (apk)

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

432/443

Content of an APK

META-INF a directory containing all the Java metadata

MANIFEST.MF the Java Manifest le, containing various

metadata about the classes present in the archive
CERT.RSA Certicate of the application
CERT.SF List of resources present in the package and
associated SHA-1 hash

AndroidManifest.xml

res contains all the resources, compiled to binary xml for the
relevant resources

classes.dex contains the compiled Java classes, to the

Dalvik EXecutable format, which is a uncompressed format,
containing Dalvik instructions

resources.arsc is the resources table. It keeps track of the

package resources, associated IDs and packages

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

433/443

APK Building

Credits: http://developer.android.com
free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

434/443

APK Building

Credits: http://developer.android.com
free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

435/443

Practical lab - Write an Application with the SDK

Write an Android application

Integrate an application in the

Android build system

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

436/443

Advices and Resources

Advices and
Resources

Embedded Linux
Experts

free electrons
Copyright 2004-2016, Free Electrons.
Creative Commons BY-SA 3.0 license.
Corrections, suggestions, contributions and translations are welcome!

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

437/443

Android Internals

Embedded Android: Porting, Extending, and

Customizing, April 2013

By Karim Yaghmour, O'Reilly

From what we know from the preview

version, good reference book and guide
on all hidden and undocumented Android
internals

Our rating: 3 stars

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

438/443

Android Development

Learning Android, March 2011

By Marko Gargenta, O'Reilly

A good reference book and guide on

Android application development

Our rating: 2 stars

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

439/443

Websites

Android API reference:

http://developer.android.com/reference

Android Documentation:
http://developer.android.com/guide/

A good overview on how the various parts of the system are

put together to maintain a highly secure system
http://source.android.com/tech/security/

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

440/443

Conferences
Useful conferences featuring Android topics:

Android Builders Summit:

https://events.linuxfoundation.org/events/androidbuilders-summit
Organized by the Linux Foundation in California (in the
Silicon Valley) in early Spring. Many talks about the whole
Android stack. Presentation slides are freely available on the
Linux Foundation website.
Embedded Linux Conference:
http://embeddedlinuxconference.com/
Organized by the Linux Foundation: California (Silicon Valley,
Spring), in Europe (Fall). Mostly about kernel and user space
Linux development in general, but always some talks about
Android. Presentation slides freely available
Don't miss our free conference videos on http://freeelectrons.com/community/videos/conferences/!

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

441/443

Last slides

Last slides

Embedded Linux
Experts

free electrons
Copyright 2004-2016, Free Electrons.
Creative Commons BY-SA 3.0 license.
Corrections, suggestions, contributions and translations are welcome!

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

442/443

Last slide

Thank you!
And may the Source be with you

free electrons - Embedded Linux, kernel, drivers and Android - Development, consulting, training and support. http://free-electrons.com

443/443