Using Bootchart on Android

This document describes how to use Bootchart with Android.

Introduction
Bootchart is a system designed to show a graphical display of the activity of a system during boot. The tool consists of a data collection tool and a graphic generator tool. In the original bootchart, the data collection tool is a standalone shell script. However, in Android, the data collection is built into the 'init' initialization program.

You can find out more information about Bootchart at it's home page, at: http://www.bootchart.org/

Overview of Steps to Use Bootchart
To use Bootchart on an Android System, you need to perform the following steps:


 * install the bootchart tool on your host machine
 * build 'init' with bootchart support
 * install 'init' into your system image
 * trigger bootchart functionality on system boot
 * retrieve the collected data from the system
 * generate the graphic from the data
 * view the resulting graphic
 * interpret the results

These steps will be described in detail in the following sections. This paper was written based on Android 1.5 software and SDK.

Installing the bootchart tool
Most modern distributions seem to have bootchart packages available.

On a Fedora system, you can get bootchart with yum:

$ yum install bootchart

On Ubuntu, bootchart can be installed with apt-get.

$ sudo apt-get install bootchart

Note that the bootchart tool is written in Java, and will require a java runtime system in order to run the program. (This should not be a problem for any machine configured with the Android SDK.)

Building 'init' with bootchart support
The first user-space program run on an Android system is called 'init' and it resides in the root directory (that is, at '/') of the Android file system. 'init' is executed by the Linux kernel, and it performs several initialization steps and starts all of the other user-space programs.

'init' operates by interpreting the configuration file /init.rc. This file has lists of commands to perform.

The 'init' program can be built with or without support for bootchart data collection. The default 'init' does not have support for data collection. In order to use bootchart with Android, you need to make sure you have an 'init' with bootchart support compiled into the program.

There are two ways to do this:

1. start from a clean build, and set the variable INIT_BOOTCHART=true $ cd mydroid $ export INIT_BOOTCHART=true $ make clean $ make

2. re-compile just 'init', specifying the variable INIT_BOOTCHART=true $ touch system/core/init/init.c   $ m INIT_BOOTCHART=true

Installing 'init' into your system image
There are a few alternatives, depending on whether you are using a real device or the emulator.

Copy the init program from the build area to the root directory of your device If you have ssh running, you may be able to do something like this:

scp out/target/product/generic/root/init root@device:/

Note: This requires that the root filesystem be writable, which I don't think it is, normally.

Specify to use a new initramfs image, containing 'init', with your emulator:

emulator @emu2 -ramdisk out/target/product/generic/ramdisk.img

Alternatively, you can put the new init program on your platform using adb:

adb push out/target/product/generic/root/init /init

Or, if you're using ttc, do: ttc cp out/target/product/generic/root/init target:/

Triggering bootchart functionality on system boot
Even though 'init' is compiled with bootchart functionality built-in, you must tell it, at boot time, to actually collect the bootchart data.

This is done by specifying the amount of time that 'init' should collect bootchart data.

1. The default method of telling 'init' to collect data is to place a file with a timeout value in /data. The file is named 'bootchart-start', and contains a single value which is the number of seconds to collect data.

This can be created with the adb command:

adb shell 'echo 120 > /data/bootchart-start'

When you reboot your device, bootcharting will start when /init is run, and will end after the indicated period. (120 seconds = 2 minutes)

2. If you are using an emulator, you can specify it a timeout value on the emulator command line. You use the '-bootchart ' option for this.

emulator @emu2 -ramdisk out/target/product/generic/ramdisk.img -bootchart 120

This is handy when used with the -wipe-data option with the emulator, which starts the system with an empty /data partition.

Note that internally, when the emulator sees the -bootchart option, it places "androidboot.bootchart= " on to the kernel command line. This is ignored by the kernel, but is read by 'init' from /proc/cmdline after the kernel boots. If your bootloader can pass user-specified kernel command line arguments, it can also trigger bootcharting with this option as well.

Retrieving the collected data from the system
'init' stores the collected data into a directory on the target file system. When bootcharting is activated, 'init' periodically polls the /proc filesystem for process and system-related data, and stores the data into the directory /data/bootchart in a series of files.

A command is provided to retrieve this set of files, and place it into an archive (which is the format used by the bootchart program).

$ system/core/init/grab-bootchart.sh

This will generate a file named 'bootchart.tgz' in the current directory.

Note that this command uses 'adb pull' to retrieve the files from the target, and assumes that there is only one device for adb to pull from. If you have multiple devices and/or emulators running, make sure you are getting the data from the one you expect.

Generating the graphic from the data
To generate a graphic image from the collected data, run the bootchart tool itself. Be sure to specify the file generated by the retrieval step above.

bootchart bootchart.tgz

On Ubuntu, the following command may be used to run bootchart:

$ java -jar /usr/share/bootchart/bootchart.jar ./bootchart.tgz

By default, bootchart will produce a graphic in PNG format. You can specify other graphic formats if you wish. For printing, you may wish to produce an encapsulated Postscript (EPS) file. If you wish to zoom in for details, you may wish to use the scalable vector graphics (SVG) format.

Viewing the resulting graphic
Use any image viewer to examine the graphic, if you created a png image.

If you output the graphic in SVG format, then you can open the file with an SVG-capable program (such as Inkscape or a web browser).

Interpreting the results
A bootchart graphic will have a few different sections that are of interest.

In general, there are CPU and disk utilization graphs at the top of the graphic. There is a timeline running from left to right in the image, with processes shown starting (and sometimes stopping) underneath it. The process utilization of CPU is indicated by coloration of it's process bar.

Things to look for are the start and top times of the various processes, and their CPU utilization. Long gaps with low utilization may indicate a timeout or some other problem.

Once you have identified the processes that are using time during the initialization, you can further analyze these by looking at system logs, using strace, and examining the system source code.

A note about overhead
In my own (Tim Bird's) testing of bootchart, I have found that it adds overhead of approximately 7% to the boot. That is, a system takes approximately 7% longer to boot with bootchart data collection enabled, as without it.

Here's one raw data point to support this:

For one of Tim's test systems: boot with no bootchart data collection: 35.5 seconds boot with bootchart data collection: 38 seconds

/proc files used
bootchart uses the following /proc files:

One-time use (for header):
 * /proc/version
 * /proc/cmdline
 * /proc/cpuinfo

Periodic use:
 * /proc/uptime - is saved in the file for each poll taken of the /proc data (value in file is in units of 10 ms (1 jiffy on systems with HZ=100))
 * /proc/stat - saved to /data/bootchart/proc_stat.log
 * /proc/diskstats - saved to /data/bootchart/proc_diskstats.log
 * /proc/*/cmdline saved to /data/bootchart/proc_ps.log
 * /proc/*/stat - saved to /data/bootchart/proc_ps.log

See 'man proc' for details, but here is a simple list of fields saved from /proc:

/proc/stat fields: cpu: user userlo system idle page: pagein pageout swap: swapin swapout intr: icounts... ctxt: # context switches since boot btime: # seconds since epoch at boot (doesn't change) processes: # of forks since boot procs_running: # of procs running procs_blocked: # of procs blocked

/proc/diskstats fields: reads read_merges rsectors rtime writes write_merges wsectors wtime ios_in_progress iotime weighted_iotime

/proc/ /stat fields: (see 'man proc' for details) pid (command) state ppid pgrp session tty_nr tpgid flags minflt cminflt majflt cmajflt utime stime cutime cstime priority nice num_threads itrealvalue starttime vsize rss rlim startcode endcode startstack kstkesp kstkeip signal blocked sigignore sigcatch wchan nswap cnswap exit_signal rt_priority policy delayacct_blkio_ticks

Issues on Android
diskstats are always zero on Android, since there are no block devices being used. Android uses only yaffs filesystems on boot.

other info gathered
bootchart uses process accounting (if turned on), and records information to the file /data/bootchart/kernel_pacct. This is used with the 'acct' function. (See manpage for acct)

Samples
Here are some sample bootchart images:

1.5 (cupcake) on emulator on slow machine
Cupcake running in an emulator on a rather slow desktop:

1.5 on ADP1
Below is an image of 1.5 booting up on an ADP1.

Note the many "background" applications starting.