The Common Reporter Interface (CRI) for JavaScript Unit Testing Frameworks
| Avoid this: | Do this: |
|---|---|
 |
 |
https://github.com/js-reporters/js-reporters/issues/
We on the QUnit team have been discussing the possibility of working with other JS test frameworks, especially those that can be run client-side (e.g. Mocha, Jasmine, Intern, Buster, etc.), to agree upon a "Common Reporter Interface" so that we could hopefully share Reporter plugins between testing frameworks. This would also benefit high-level consumers of the frameworks such as Karma, BrowserStack, SauceLabs, Testling, etc.
This would most likely come in the form of:
- a common Reporter API/Interface, e.g.
- an EventEmitter interface (
.on(...)/.off(...)) OR an object with standard "hook" properties - maybe a standard-ish way to register a Reporter, e.g.
MyLib.addReporter(x),MyLib.reporter = x;, etc.
- an EventEmitter interface (
- a minimum viable set of standardly-named events
- an associated standard set of data/details provided for each event
- a minimum viable set of standard test status types (e.g. pass, fail, skip, todo, pending, etc.)
- updating all participating test frameworks to support this new common Reporter interface
Would you be interested in discussing this with us further? Please join in!
Based on the discussion in #1, this is the suggested minimum set of event names to be triggered by a testing framework, to be consumed by reporters or other testing tools.
runStart: Indicates the beginning of a testsuite, triggered just once.suiteStart: Triggered at the start of each group of tests within a testsuite.testStart: Triggered at the start of each test.testEnd: Triggered at the end of each test.suiteEnd: Triggered at the end of each group of tests within a testsuite.runEnd: Indicates the end of a testsuite, triggered just once.
These only define event names, not the data associated with each event. The data still needs to be specified via further discussion in #1.
The criteria for picking these event names included:
- These use the most common terms across a selection of frameworks, as gathered in #1
- It uses names that are valid JavaScript identifiers, which allows using those as keys in JSON and avoids the need to quote keys in regular JS objects or function calls.
- It doesn't overlap with any known existing events, so introducing these could be done in parallel to the existing API in each framework, optionally deprecating and eventually removing the previous API.