Difference between revisions of "Testing"
(fix bullets for mediawiki) |
(→Testing Standards) |
||
Line 51: | Line 51: | ||
** counts | ** counts | ||
** subtest results | ** subtest results | ||
+ | ** Candidate formats: | ||
+ | *** https://testanything.org/ | ||
* test package format | * test package format | ||
** meta-data for each test | ** meta-data for each test |
Revision as of 16:35, 9 November 2016
This page has lists of resources for Linux testing.
Contents
Presentations from events
LPC 2016 testing mini-summit (Nov 2016)
The 2016 Linux Plumbers Conference had a mini-summit on Testing and Fuzzing, as well as other sessions dedicated to open source testing issues. Here are some links related to that event:
- Testing and Fuzzing agenda
- presentations:
- Testing Kernels in the Cloud - by Theodore Ts'o
- All About the perf_fuzzer - by Vince Weaver
- Applying mutation testing for testing RCU: Progress report - by Iftekhar Ahmed
- Integrating and running all the kernel tests - by Yannick Brosseau
- Linux Kernel Testing: Where are we ? - by Guenter Roeck
- Syzkaller: Future development - by Dmitry Vyukov
- Trinity: Next directions - by Dave Jones
- Increase Test Coverage in a Linux-based OS (Google doc) - By Guillermo Ponce and Victor Rodriguez
collect presentation links above
Japan Jamboree 58-1 (Oct 2016)
- presentations:
- Fuego: Introduction, Status, and Future directions (PDF) - by Tim Bird, Sony
- Case Study (1) - by Kusakabe-san, Fujitsu TEN
- Fuego LTP Improvements (PDF) - by Danial Sangorrin, Toshiba]
- Continuous Integration and Autotest Environment using Fuego (PDF)(from ELCE 2016) - by Kengo Ibe, Kotaro Hashimoto, Mitsubishi Electric
- Current State of the Test Activities in Renesas Using Fuego (PDF) - by Masao Nomura, Renesas Electronics Corporation
Previous ELC and ELCE talks on testing
- ELCE 2010 - Embedded Software Testing (PDF) - by Arnout Vandecappelle
Testing Standards
Start collecting info for testing standards here:
- https://tools.ietf.org/html/rfc2119 - ietf MUST, SHALL, MAY, etc. wording standards
Things we'd like to standardize in open source testing:
- test nomenclature (test glossary)
- how to specify test pre-requisites
- ex: assert_define ENV_VAR_NAME
- ex: kernel_config
- test API
- host/target abstraction
- kernel installation
- file operations
- console access
- command execution
- test retrieval, build, deployment
- test execution:
- ex: 'make test'
- test execution:
- test log output format
- counts
- subtest results
- Candidate formats:
- test package format
- meta-data for each test
- test results
- baseline expected results for particular tests on particular platforms
- what tests can be skipped etc.