Automated Testing Summit 2018
This is a public planning page for the Automated Testing Summit
Contents
2018 meeting
Coordinated by: Tim Bird and Kevin Hilman
Action items:
- Send invitations (in progress)
- Finalize sessions, presentations, and discussion topics
- Coordinate registration, signage and badging with ELCE/OSSE
Date and Venue
- Edinburgh, Scotland - Oct 25, in the Edinburgh International Conference Centre
- co-located (following) Embedded Linux Conference Europe and Open Source Summit Europe
- ELCE and OSSE run Oct 22-24
- see https://events.linuxfoundation.org/events/elc-openiot-europe-2018/
Attendees
wanted attendees
- Pawel Wieczorek (Samsung) - SLAV
- Kevin Hilman (BayLibre) - KernelCI, lab-in-a-box
- Milosz Wasilewski (Linaro) - LKFT
- Matt Hart (Linaro) - LAVA, upstream-focused testing, kernelci
- Neil Willliams (Linaro) - LAVA
- Jan Lübbe (Pengutronix) - Labgrid
- Tim Bird (Sony) - Fuego, ttc
- Andrew Murray (Witekio) - started this initiative
- Daniel Sangorrin (Toshiba) - Fuego/CIP
- Khiem Nguyen (Renesas) - LTSI testing
- Guenter Roeck - (Google) - ChromeOS testing, kerneltests.org
- Fengguang Wu - (Intel) - 0 day test service
- Agustin Benito Bethencourt (CodeThink) - board@desk testing for CIP (presentation)
- Shuah Khan - (Samsung) - kselftest
- Greg KH (Linux Foundation) - LTS maintainer
- Michael Larabel (Phoronix, phoronic-test-suites, phoromatic, openbenchmark.org, linuxbenchmark.com) - Phoronix Test Suites
- Cyril Hrubis (SUSE) - LTP
- more people from industry
- (comcast? - ask Comcast folks if they are interested)
possible invitees
- Yannick Brosseau (Facebook) -
- Dmitry Vyokov (Google) - KASAN, syskaller testing
- Kenji Tadano (Mitsubishi)
- Kengo Ibe (Mitsubishi)
- someone from yocto? (who?)
- someone from buildroot? (Thomas Petazzoni?) (is the buildroot community interested in this?)
- someone from U-Boot? (Tom Rini/Stephen Warren/Simon Glass)?
- Sjoerd Simmons (Collabora) - kernelci contributor
- Guillaume Tucker (Collabora) - kernelci contributor
interested in attending
Please add your preferred co-located conference (ELCE, Plumbers, or something else), if you have one.
- Geert Uytterhoeven - general board farm (ELCE or FOSDEM)
- Kieran Bingham - general board farm
- Jan-Simon Möller - general board farm, especially LAVA & KernelCI
- Marco Cavallini (Koan) - general board farm
- Chris Fiege (Pengutronix) - board farm hardware, special development tools (USB-SD-Mux, ...)
- Michal Simek (Xilinx) - general board farm
- Punnaiah Choudary Kalluri (Xilinx) - general board farm
- Neil Willliams (Linaro) - LAVA
- Tim Bird (Sony) - Fuego, ttc (either ELCE or Plumbers is OK)
- Carlos Hernandez (Texas Instruments) - manages board farm at TI
- Otavio Salvador (O.S. Systems) - (prefers ELCE)
- Jan Lübbe (Pengutronix) - labgrid (prefers just after ELCE)
- Milosz Wasilewski (Linaro) - LKFT (ELCE or FOSDEM are fine) - can cover LKFT if Tom can't make it
- Muhammad Shafique (Mentor) - general board farm, LAVA & KernelCI
- Mirza Krak (Endian Technologies AB) - (ELCE or FOSDEM)
- Tim Orling (Yocto Project) - (ELCE, possibly Plumbers)
- Matt Hart (Linaro) - board farm, PDUDaemon, LAVA and KernelCI
- Rouven Czerwinski (Pengutronix) - labgrid (prefers just after ELCE)
- Mark Brown (Linaro) - Board farm stuff, kernelci, consuming results
- Alan Bennett (Open Source Foundries) - (prefers ELCE)
- Tyler Baker (Open Source Foundries)
- Manuel Traut (Linuxtronix) - r4d, jenkins, ci-rt
- Anna-Maria Gleixner (Linuxtronix) - r4d
- Zhenfeng Zhao (Wind River) - (either ELCE or Plumbers is fine)
- Shuah Khan (Samsung) - kselftest (Linaro connect?)
- Steve McIntyre (Linaro) - LAVA (any)
- Matteo Vit (Starware Design) - general board farm, hardware (ELCE)
- Dan Rue (Linaro) - Linaro LKFT
- Ryan S Arnold (Linaro) - LAVA & LKFT (either ELCE, LPC, or Linaro Connect)
- Roland Hieber (Pengutronix)
- Eystein Stenberg (Mender.io) (either ELCE or Plumbers)
- Nobuhiro Iwamatsu (Cybertrust Japan / CIP / Debian) - (prefers ELCE)
- Alice Ferrazzi (Cybertrust Japan / Gentoo Kernel Project Leader) (either ELCE or Plumbers) kselftest contributor / Gentoo kernel CI
- Marc Kleine-Budde (Pengutronix) - (prefers just after ELCE)
- Tsugikazu Shibata (LTSI) - (prefers ELCE)
- Hirotaka Motai (Mitsubishi Electric) - (prefers ELCE)
Agenda (brainstorming)
Here is some brainstorming on an agenda...
- board farm survey (lightning talks)
- what are people using?
- what works?
- what's missing?
- board discovery/lab introspection?
- layers and interfaces
- what layers are supported, needed?
- any way to leverage/separate/isolate existing software?
- what tests need to be supported?
- boot-time
- run-time
- package-based (package unit tests)
- driver (hardware specific?)
- requiring specialized hardware external to board (e.g. canbus simulator, hdmi frame-grabber)
- multinode
- how to allocate/schedule multiple pieces of equipment for a test (e.g. 2 or more nodes for a network test)
- results reporting
- centralized server and API to it (kernelCI json?)
- sharing tests
- how to define standards
- de-facto only? (dominant project? (cough, LAVA))
- documents?
- survey of existing projects, and what pieces they focus on (or don't)