Difference between revisions of "Automated Testing Summit 2019"
(→Presentations: add Fuego status presentation) |
(→Summit Artifacts) |
||
Line 123: | Line 123: | ||
|- | |- | ||
|Fuego Status Update|| Tim Bird || [[File:ATS2019-Fuego-Status-Update.pdf]] | |Fuego Status Update|| Tim Bird || [[File:ATS2019-Fuego-Status-Update.pdf]] | ||
+ | |- | ||
+ | |Test Metadata Extraction|| Cyril Hrubis || [[File:Test metadata extraction.pdf]] | ||
|- | |- | ||
|} | |} |
Revision as of 03:58, 4 November 2019
This is a public planning page for the Automated Testing Summit
Please see: https://events.linuxfoundation.org/events/ats-2019/
Contents
2019 meeting
Coordinated by: Tim Bird This is now a full-blow Linux Foundation event, with sponsors, a CFP, program and everything.
See the Linux Foundation Event page for details.
Mailing list
Inter-project discussions on Automated Testing occur on the "Automated Testing" mailing list of the Yocto Project
Date and Venue
- Location: Lyon, France
- Venue: Lyon Convention Center, 50 Quai Charles de Gaulle
- Date: October 31, 2019
- Rooms: Pasteur Lounge and Rhone 3AB
- Time: 9:00 am to 17:15 (5:15 pm)
- co-located (following) Embedded Linux Conference Europe and Open Source Summit Europe
- ELCE and OSSEU run Oct 28-30
- see https://events.linuxfoundation.org/events/embedded-linux-conference-europe-2019/
Attendance is by open to the public, and we are projecting the attendee fee to be $250, with this fee waived for speakers.
Summit Pre-planning
Here is a list of action items to do to prepare for the summit:
- organize program committee (done)
- get sponsors (done)
- create sponsorship prospectus (done)
- contact candidate sponsors (done)
- hold "Call for Proposals" (done)
- need to determine CFP dates (done)
- determine agenda (done)
- organize program committee (done)
- advertise for attendees (done)
- reserve venue, catering, AV rentals (done)
- create signage, web site, registration system (done)
- hold event (done)
For private planning information, see the private planning wiki page
Call for Presentations
- CFP Open: Tuesday, May 28
- CFP Close: Friday, August 13
- Notifications Out: September 2
- Schedule Out: September 4
- Slides Due: October 24, 2019
- Event Date: October 31, 2019
Suggested Topics
- Board Farms and Test Labs - hardware, software, useful practices
- Layers and Interfaces - components of your automated CI loop and protocols between them
- Results Reporting - results aggregation, visualization, notifications, etc.
- Defining standards - propose or discuss conventions for test definitions, results formats, layer interoperability (e.g. protocols), etc.
- Existing Projects - describe your test system or framework and any challenges you overcame or that are still outstanding
- Scalability - supporting big data, or handling distributed builds or tests
- Ease of use - making testing and test development accessible to non-QA folk (developers, end-users)
Pre-meeting work
Here's what Tim thinks we need to do:
- Refine the test stack reference diagram
- Organize committees for specific APIs
- designate a head for each committee
Sponsors
Sponsorship was provided by:
- Sony - Platinum sponsor
- Core Embedded Linux Project (of the Linux Foundation) - Gold sponsor
- Linaro - Bronze sponsor
- Civil Infrastructure Platform - Bronze sponsor
Schedule
Summit Artifacts
documents and presentations
- for presentations - see below
Key decisions
- all systems to make a kcidb client (upload results)
- extend the kcidb schema
- https://github.com/kernelci/kcidb
- priority (over test definition unification work)
- use LTP metadata format as initial standard
- add meta-data convert to kselftest (Tim)
- build a prototype to move boards between LAVA and labgrid (Jan)
- want to collect more information for "Hardware Design for Testing"
- Contributions are Welcome: https://github.com/SmithChart/Designing-for-Automated-Testing/
- get something working for board management layer for poweron/poweroff/serial-console
- continue work on test definition common schema and central repository (Tim)
- add information about hardware you find useful, or have designed please add to Board_Farm
- focus on meeting at Plumbers next year
- continue using automated-testing@lists.yoctoproject.org
Action items from meeting
Action items:
- post slides to this page
- send minutes
Presentations
Session | Presenter | Slides |
---|---|---|
A Guide to CIP Testing | Chris Paterson & Michael Adler | File:2019-10-31 ATS-Lyon CIP Testing Overview v3.pdf |
Update on previous conferences, KernelCI update | Kevin Hilman | File:ATS 2019.pdf |
Bootloader testing in LAVA | Remi Duraffort | File:ATS-Lyon-Bootloader-testing-in-LAVA.pdf |
New Ways Out of the Struggle of Testing Embedded Devices | Chris Fiege | File:ATS2019-cfi-embedded-testing handout.pdf |
SLAV: Test Stack Abstraction Layers | Paweł Wieczorek | File:Test stack abstraction layers.pdf |
A Survey of Open Source Test Definitions | Tim Bird | File:ATS2019-survey-of-test-definitions.pdf |
Working together to Build a Modular CI Ecosystem | Tim Bird | File:ATS2019- buiding-a-modular-ecosystem.pdf |
Fuego Status Update | Tim Bird | File:ATS2019-Fuego-Status-Update.pdf |
Test Metadata Extraction | Cyril Hrubis | File:Test metadata extraction.pdf |