Remote Board Access Spec

This is a specification for a system for "Remote Board Access".

VERSION 0.3

Introduction
This specification describes the configuration and conventions used for a "Remote Board Access" system. "Remote Board Access" (or RBA) is the name used to describe a configuration of hardware and software which allows for an individual developer to build Linux software and test it on a target machine (usually an embedded board), which is accessed remotely.

Related to this "access specification", is the definition of an automated test framework which can be built upon it. Some aspects of this specification directly address the requirements of such a framework. However, some aspects of this specification also support the activity of individual developers working interactively with embedded development boards.

This specification creates standards for two main activities:
 * building software for a target machine
 * accessing and controlling a target machine

Taming Variability in Embedded Configurations
The key idea of this specification is to create a uniform method of access to build tools, software, and the board itself, for an embedded target board. A significant problem in the embedded space is the variety of configurations that are presented to a software developer for such boards. A large number of items in the configuration and setup of an embedded target board vary from board to board. This includes a such things as the bootloader, compiler (location, version and supported architecture), kernel version, kernel configuration, file system type, and methods of target access for software installation, console access, command invocation, file system access, etc.

It is very difficult for both a human engineer and an automated test system to deal with this large variety of factors. One of the primary purposes, then, of this specification is to create a model whereby these factors are masked and a uniform method is presented for building and installing software, and manipulating the target board.

Rationale
This specification is needed in order to:
 * to create a uniform method of building software and controlling a target board
 * to allow for automation of testing activities on a board
 * to eliminate or reduce the learning curve for using different boards


 * this specification deals with one area of interface (between a host and a target) that is part of the overall architecture of the CELF OpenTestLab
 * see TestLabArchitecture for more information.

Definitions

 * host: The machine used to control or access one or more target boards.
 * target: A development board which can be accessed remotely using the RBA system.
 * toolchain: The collection of programs used to create software for a target machine. This consists of the compiler (including preprocessor), assembler, linker and other programs used for manipulating source and machine code into a form usable on the target. This usually consists of software from the Linux packages for gcc, binutils, and glibc.

basic remote board use

 * a developer wants to build a kernel, boot it on the target, and view the kernel bootup messages
 * also, the developer wants to log in to the new kernel, and run a variety of user space applications

Steps:
 * 1) developer sets target-specific work environment
 * 2) developer gets target-specific kernel source
 * 3) developer sets the kernel configuration
 * 4) *developer gets default kernel config for $TARGET
 * 5) **or "make $TARGET_defconfig" should work
 * 6) *developer makes config modifications (by hand)
 * , or
 * 1) developer builds the kernel
 * 2) *environment provides ARCH, CROSS_COMPILE, and kimage vars
 * , or
 * 1) developer installs kernel on the target
 * 2) developer gets a target console
 * 3) developer resets target board
 * 4) developer logs into target board
 * 5) *(login on console) or
 * 6) *developer can now run commands interactively on the target
 * 1) developer resets target board
 * 2) developer logs into target board
 * 3) *(login on console) or
 * 4) *developer can now run commands interactively on the target
 * 1) *(login on console) or
 * 2) *developer can now run commands interactively on the target

testing a kernel configuration

 * a developer wants to test a particular kernel configuration option on a target
 * developer needs a baseline configuration for that target, and a way to make modifications
 * alternatively, the developer needs a way to upload their own configuration to the host

Steps:
 * 1) developer copies desired configuration file to RBA host
 * 2) developer logs into RBA host using ssh
 * 3) developer sets target-specific work environment
 * 4) developer gets target-specific kernel source
 * 5) developer sets the kernel configuration
 * 6) *developer uses default configuration with some modifications:
 * 7) **developer gets default kernel config for $TARGET
 * 8) **or "make $TARGET_defconfig" should work
 * 9) **developer makes config modifications (by hand)
 * , or
 * 1) *OR developer uses their own configuration
 * 2) developer builds the kernel
 * 3) *environment provides ARCH, CROSS_COMPILE, and kimage vars
 * 4) developer locks target for use
 * 5) developer installs kernel
 * 6) developer gets a target console
 * 7) developer resets target board
 * 8) developer logs into target board
 * 9) *(login on console) or
 * 10) developer examines messages, runs tests, etc.
 * 1) developer locks target for use
 * 2) developer installs kernel
 * 3) developer gets a target console
 * 4) developer resets target board
 * 5) developer logs into target board
 * 6) *(login on console) or
 * 7) developer examines messages, runs tests, etc.
 * 1) developer logs into target board
 * 2) *(login on console) or
 * 3) developer examines messages, runs tests, etc.

testing a new kernel patch

 * a developer wants to apply a patch to a target-specific kernel (to see if it applies) and run the new kernel on the target hardware (to see if the new feature works)

Steps:
 * 1) developer copies desired configuration file to RBA host
 * 2) developer uploads a patch to the RBA host
 * 3) developer logs into RBA host using ssh
 * 4) developer sets target-specific work environment
 * 5) developer gets target-specific kernel source
 * 6) developer applies patch to kernel source
 * 7) developer sets the kernel configuration
 * 8) *developer uses default configuration with some modifications:
 * 9) **developer gets default kernel config for $TARGET
 * 10) ***or "make $TARGET_defconfig" should work
 * 11) **developer makes config modifications (by hand)
 * 12) ***, or
 * 13) *OR developer uses their own configuration
 * 14) developer builds the kernel
 * 15) *environment provides ARCH, CROSS_COMPILE, and kimage vars
 * 16) developer locks target for use
 * 17) developer installs kernel
 * 18) developer gets a target console
 * 19) developer resets target board
 * 20) developer logs into target board
 * 21) *login on console) or
 * 22) developer examines messages, runs tests, etc.
 * 1) *environment provides ARCH, CROSS_COMPILE, and kimage vars
 * 2) developer locks target for use
 * 3) developer installs kernel
 * 4) developer gets a target console
 * 5) developer resets target board
 * 6) developer logs into target board
 * 7) *login on console) or
 * 8) developer examines messages, runs tests, etc.
 * 1) developer logs into target board
 * 2) *login on console) or
 * 3) developer examines messages, runs tests, etc.
 * 1) developer examines messages, runs tests, etc.

testing user-space software

 * a developer wants to test a user-space program on the target
 * developer needs target-specific kernel running on target
 * developer needs to be able to upload user-space program to RBA host
 * developer needs build environment for user-space program
 * developer needs a way to install program on the target
 * should be able to put program in already-available root filesystem
 * developer needs a way to run program on target, and gather results

testing a new baseline kernel
A target maintainer may want acquire target-specific patches, and apply to a new kernel.org baseline
 * this is to see which target-specific patches do not apply cleanly any more.
 * target-specific patches should be separated from baseline kernel
 * target-specific kernel source should be available as baseline kernel + patches

Steps (summary):
 * login to rba host
 * get kernel
 * get target (old/existing) baseline
 * install new baseline
 * diff old-baseline kernel >target-patch
 * patch new-baseline <target-patch
 * report problems
 * compile
 * report problems

automated unit test
A developer wants to test a particular feature of the kernel or default software, in a unit test. A series of tests are to be performed, without user interaction.


 * need to put test script on target (target cp)
 * need to initiate test on target (target reset, target run)
 * need to handle watchdogging the program/target
 * need external reset control (external power control is also desirable)
 * need to get results from target
 * need to set a config option
 * need to patch the kernel
 * need to compile, install, start the kernel

automated patch test

 * a developer wants to test that a patch can be compiled for a number of different architectures and platforms
 * the developer submits the patch to a patch server
 * the patch server iterates through a list of registered RBA sites, for each site:
 * the patch server obtains the kernel source for the target (on the RBA host, under the patch server account)
 * the patch server uploads the patch
 * the patch server compiles the kernel

=== automated regression test
 * lab software checks for update to linux kernel (or other software)
 * when new version is detected, code is downloaded and test is run automatically

run test suite on a non-lab node

 * developer accesses test server and selects a test run download and run on their local host/target combination
 * pre-requisite - user verifies that their host/target combination interoperates with the RBA system
 * user downloads an interoperability test and runs it
 * web server provides list of available tests
 * client can download and run a test from the server
 * test programs for target must be packaged somehow [need specification for format]

Supported operations (tasks)

 * verify that the target machine is available for use (target status)
 * reserve target for use (target acquire [timeout])
 * unreserve target (target release)
 * build kernel
 * obtain kernel source for target (target get_kernel)
 * set up build environment for kernel (target setenv)
 * get default kernel configuration (target get_config)
 * build root filesystem for target
 * build program for target
 * upload source for program for target (scp)
 * set up build environment for programs (target setup_build_environment)
 * install kernel (target kinstall)
 * install program (target cp)
 * install rootfs (target fsinstall)
 * get results (target cp)
 * control target
 * reset target machine ("target reset")
 * power cycle target machine ("target power_off", "target power_on", "target power")
 * access console ("target console")
 * access target remotely ("target login")

Machine configuration
Model: an RBA site consists of a single host, connected to one or more target machines.

A host has sufficient software to perform a number of development activities relative to its target machines. Specifically, the host MUST provide software and mechanisms to:
 * 1) acquire, build and install the Linux kernel software for the target
 * 2) reset and power-cycle the target
 * 3) access the boot console of the target (for interactive use)
 * 4) copy files to and from the target
 * 5) execute commands on the target

A lab host, in practice, is expected to be a normal PC machine running the Linux operating system. It should have sufficient performance and disk space to build the Linux kernel, and to host a number of copies of the Linux kernel source tree (and other source and binaries used for both host and target operation.) A minimum of 10 gigs of free hard disk space is recommenced on a lab host.

All remote board access operations must be available from the Linux command line, and runnable in a batch mode (without interaction).

Required/Recommended software for RBA host

 * toolchains - minimally capable of building a Linux kernel
 * toolchain source (for compliance with GPL)
 * toolchain binaries
 * kernel source
 * default config for target
 * "target" command, with following sub-commands:
 * target name is specified on command line
 * optional RBA_TARGET environment variable can specify the target being used
 * support the following sub-commands:
 * get_kernel [-o dir] - copy source for kernel to . or specified dir
 * kernel source should include a defconfig for the target
 * get_config [-o dir] - copy default target config to . or specified dir
 * setenv - prepare environment for building software for target
 * status - show current status of target (who is currently using it, is it operating, etc.)
 * info - show information about the target (technical information about target)
 * acquire [time[m|h|d]] - lock target board for use (default is 2 hours)
 * [-s time] specify starting time, default is now
 * release - unlock target board for use (terminate a current reservation (same as "acquire 0")
 * reset - reset target board
 * power_off - turn off power to target
 * power_on - turn on power to target
 * power - power cycle target (turn power off then on again)
 * console - access console for target (place where bootloader messages (and kernel boot messages) appear)(Only one of these is allowed)
 * login - get (non-console) login prompt for target (There can be multiple of these)
 * list - show list of targets available on this host
 * kinstall [] - kernel install - copy kernel from kernel build area to target, or to place on host where it is used to boot the target
 *  specifies the type of kernel install to use. Use target kinstall_list to see possible options.  Some examples might be: tftp, host, serial_download, eth_download, download_XIP, etc.
 * cp - copy file to/from the target
 * wait - wait for an operation to complete on the target
 * run [xxx] - run a program on the target
 * customary kernel command-line development tools
 * vi, diff, patch, make
 * toolchain specific to target: gcc, binutils, etc.
 * nice to have, but not required:
 * diffstat, cscope, ccache, ctags, diffinfo, tpm
 * ~/bin should be on the path, so users can install their own software
 * target root file system (binaries)
 * (optional) target root file system source and build system

Required Hardware support for RBA target

 * some remote console capability (usually serial)
 * ability to survice power-off/power-on cycle - in the minimum, if remote reboot and remote reset are not supported
 * a mechanism to remotely power-off/power-on the target (This might be provided by X10 control of the board power supply via software on the host)

Optional Hardware support for RBA target

 * ability to remotely reset the target by software control on the host

Required/Recommended software for RBA target
This is the absolute minimum.
 * target-specific bootloader
 * must support some remote console (serial is best)
 * must support dynamically loading the kernel (either via serial or via network, usually tftp from host is best)
 * Linux kernel
 * login shell
 * init script that can start test scripts on startup

Advanced optional features

 * networking support, and one or more of the following:
 * telnetd, rshd or sshd
 * this would support remote command execution
 * ftpd, rcpd, sshd?
 * this would support remote
 * this would support file operations (in non-NFS-rootfs mode)
 * ssh and scp availability on target
 * this would support file operations (in non-NFS-rootfs mode)
 * this would support remote command execution
 * networking supporttelnetd
 * this would support remote command execution
 * web server on host with CGI support for target status
 * web server on host with CGI-based RBA target reservation system

Notes (informational and non-normative)
additional information or clarifying comments

Related work
Remote System Framework for eclipse - proposal by MontaVista - see RSF EclipseCon Presentation.ppt

Device Software Development Platform - proposal for target development model and system for use with Eclipse. See: http://www.eclipse.org/proposals/eclipse-dsdp/index.html

Remaining Issues
[this is a placeholder section for listing issues while the spec is under development. It should be empty when the spec is completed (or the issues should be deferable to a subsequent version of the spec).]