Android Compliance Test Suite

Google provides a test suite, called the Compliance Test Suite (CTS), which can be used to validate that the behavior of a device matches what most applications expect.

It is part of their compatibility program for ensuring that new devices are compatible with the Android standard.

For an introduction to the CTS, see http://source.android.com/compatibility/cts-intro.html

There is an document describing the current "standards" for Android, called the Android Compatbility Definition Document. The CDD for Android 2.3 is in the following PDF: android-2.3.3-cdd.pdf

Finally, here are some Frequently Asked Questions about compatibility.

How to install and run the tests
Roger sent this information on the android-porting mailing list in April 2011:

I would recommend getting the official test suites which you can get at: http://source.android.com/compatibility/downloads.html

Unzip the file and you will get android-cts/ docs/ repository/ tools/

You also need to have the Android SDK installed (get it from http://developer.android.com/sdk/index.html).

Set the environment variable SDK_ROOT to the location where you install the Android SDK.

On you target device, have /sdcard mounted and readable/writable. Usually this is for an external SD card. If this is so for your device, then insert a formatted SD card before you run CTS.

Install the CTS service package, like so: adb install -r android-cts/repository/ testcases/CtsDelegatingAccessibilityService.apk

On the device, enable Settings > Accessibility > Accessibility > Delegating Accessibility Service

Launch the CTS in your shell: android-cts/tools/startcts

Run the tests: start --plan CTS

For devices I've tested, I've had to wait and ignore messages like: Exception in thread "Thread-5" com.android.ddmlib.ShellCommandUnresponsiveException

Results are output to: android-cts/repository/results/

You can stop and restart a test suite by selecting the same result set each time.