|
| 1 | +- Start Date: 2018-12-24 |
| 2 | +- RFC PR: (leave this empty) |
| 3 | +- Ember Issue: (leave this empty) |
| 4 | + |
| 5 | +# Build in ember-exam |
| 6 | + |
| 7 | +## Summary |
| 8 | + |
| 9 | +This RFC proposes to promote [Ember-exam](https://github.com/ember-cli/ember-exam)'s functionality |
| 10 | +to be a core part of the developer workflow. |
| 11 | + |
| 12 | +## Motivation |
| 13 | + |
| 14 | +Ember cares greatly about compatibility. Every addon is generated with a config file for `ember-try` |
| 15 | +to ensure the code is tested in the last two LTS version of Ember and on the upcoming _beta_ and _canary_ |
| 16 | +channels. |
| 17 | + |
| 18 | +We are doing great but we can still do better. |
| 19 | + |
| 20 | +Fast test suites are critical to the adoption of good testing practices. |
| 21 | +Testing in several browsers is, specially in addons, is key to ensure your code will run correctly |
| 22 | +for all your users and to keep the web open. |
| 23 | + |
| 24 | +Although testing in multiple browsers is already possible today out of the box, |
| 25 | +going from testing on only one browser to testing two or three doubles or triples the testing time |
| 26 | +and it is a burden many people do not care to face. |
| 27 | + |
| 28 | +[Ember-exam](https://github.com/ember-cli/ember-exam) is an fantastic addon that allows users to |
| 29 | +run their tests in parallel in different browsers or even split their suite in chunks and run it |
| 30 | +in several instances of the same browser. It allows users to run their tests in a fraction of the time |
| 31 | +or increase the number of tested browsers without worsening the feedback loop. |
| 32 | + |
| 33 | +It is just too good for an opinionated framework like Ember.js to not advocate its usage. |
| 34 | + |
| 35 | +## Detailed design |
| 36 | + |
| 37 | +There are two different approaches for this goal, listed in order of complexity: |
| 38 | + |
| 39 | +### Add ember-exam to the default blueprint for apps and addons |
| 40 | + |
| 41 | +This approach yields a good return with a relative low effort. Including it by default on newly generated |
| 42 | +projects signals it to the community as a good practice. Ember-exam is already part of the ember-cli |
| 43 | +organization so there shouldn't be a problem controlling the code and the pace of the releases. |
| 44 | + |
| 45 | +Developers can totally opt out by just removing the addon should they want to. |
| 46 | + |
| 47 | +The downside of this approach is that we cannot just assume that everybody will have ember-exam-like features. |
| 48 | +If we wanted to implement some advanced features in ember-cli like testing tailor-made per-browser builds, we'd |
| 49 | +have to account for that fragmentation. |
| 50 | + |
| 51 | +### Merge ember-exam's functionality into the testing experience. |
| 52 | + |
| 53 | +A more involved approach consists on moving the code that powers ember-exam builds into the relevant |
| 54 | +libraries (ember-cli / ember-test-loader / ember-qunit / ember-cli-mocha) and discontinuing Ember-exam as a separate tool. |
| 55 | + |
| 56 | +On one hand side ember-exam wouldn't have to monkey patch the test-loader anymore and Ember would |
| 57 | +completely embrace parallel testing, while on the other hand the complexity needed for supporting |
| 58 | +parallel testing might end divided across several projects. |
| 59 | + |
| 60 | +### Default configuration |
| 61 | + |
| 62 | +In both scenarios we can start with a parallel count of 1, meaning parallel builds are still opt-in. |
| 63 | +With this default configuration there won't be any discernible difference from how things work today out of the box. |
| 64 | + |
| 65 | +User will only have to make a change in the `/testem.js` config file to start getting the advantages of parallel testing. |
| 66 | + |
| 67 | +## How we teach this |
| 68 | + |
| 69 | +The testing section of the guides will have to be amended to include information about splitting, parallelization |
| 70 | +and other concepts that are today explained in the [Readme of Ember-exam](https://github.com/ember-cli/ember-exam/blob/master/README.md) |
| 71 | +test suites |
| 72 | + |
| 73 | +## Drawbacks |
| 74 | + |
| 75 | +Adding new features to the default Ember experience means more code to maintain and more |
| 76 | +features for newcomers to learn, who already have plenty on their plate. |
| 77 | + |
| 78 | +We can limit the teaching impact of parallel builds by defaulting to 1, but does not remove it completely. |
| 79 | + |
| 80 | +## Alternatives |
| 81 | + |
| 82 | +Leave ember-exam as the nice addon it is today so users who care the most for speed or multi-browser |
| 83 | +support install it. |
| 84 | + |
| 85 | +## Unresolved questions |
| 86 | + |
| 87 | +By default Ember addons are only tested in headless chrome. Although it's not _directly_ related parallel |
| 88 | +testing it could make multi-browser testing cheap enough to change the defaults so addons are tested in both |
| 89 | +chrome and firefox, ensuring better compatibility. |
0 commit comments