Hottest notes

From eLinux.org
Revision as of 19:14, 26 September 2019 by Tim Bird (talk | contribs)
Jump to: navigation, search

See https://lists.linuxfoundation.org/pipermail/fuego/2019-February/002868.html

Ralphael Gago Castano said this:

Introduction

We evaluated Fuego some time ago (in the Spring-Summer 2017), because we had a LAVA setup that had become hard to work with, but at the end we didn't switch because on Fuego the GIT repository was mounted on the Jenkin's instance filesystem, this prevented multiple users to develop on the same server.

I still found Fuego superior to LAVA on a lot of aspects for our use case, so taking time from here and there I developed similar Jenkins-based framework that has a very similar philosophy: A very-thin wrapper that delegates the hard work to Jenkins, but adds some features from LAVA like dynamic board scheduling.

Luckily my company has allowed me to open-source it, so because this framework is inspired in Fuego (some code is a direct copy) I thought that it would be a good idea to share it with you here. If you think there are cool ideas/concepts you have a reference implementation. If we get some criticism from experts on the field it is very welcome.

Repository

https://github.com/HMSAB/hottest

Features

I will try to make a list of the most important features/differences:

  • Like Fuego, it has small tooling coded in Python, tests are written in shell and uses Jenkins too.
  • Test are generated by using shell script pieces (with json metadata for Jenkins consumption) that support including (copy pasting) other pieces for building a script and accumulating all the Jenkins metadata (e.g. each of these pieces can define parameters for Jenkins).
  • There is an API containing a few board functions that the user has to provide (dut_cmd, dut_get, dut_put), so board functions are abstracted and tests are shareable. All the transports on Fuego are(were) part of the core, on hottest they are a "module", so the user can implement them or select one of the transports available.
  • There are predefined milestones on the test sequence. The test writer can add functions before or after each of the milestones dynamically, e.g. you can call "add_step_before_power_on" and pass a function that crosscompiles a small executable and then call "add_step_before_test" and pass a function that copies the executable to the device, then use the executable inside a test.
  • Allows powering on and off the boards and flashing.
  • The generator supports include directories, so the board, powering, lab setup and firmware flashing implementation can be left on separate user-private repositories while still using shared tests.
  • All the jobs(tests) are self-contained, they embed the board code into the job. This avoids having to mount git repositories on the server's filesystem at the expense of code duplication on Jenkins (handled by the tool, HDD is cheap). It allows tests from older versions to coexist unmodified on the same server too.
  • It adds a mapping between boards and Jenkins nodes, so you can have some tests written for a specific board and some boards connected to the server. When you schedule a lot of tests for the same board Jenkins spreads the workload between all idle boards. This uses Jenkins labels under the hood.
  • The testplans are implemented as Groovy pipelines that schedule a series of tests in parallel. These pipelines can have some tests serially executed in a well defined order, useful for cases where e.g. there is only one hardware resource/dongle on the test PC that many tests use.
  • For now, it has no extra Jenkins dependencies, so it works with a bare modern Jenkins install with default plugins.
  • Reporting is in a minimum-viable state now, it just generates a Junit report and builds a trivial gnuplot graph as a Jenkins artifact.