Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

prototype using c8 as a test coverage lib; try alternatives to tape-runner #2858

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

siosonel
Copy link
Member

@siosonel siosonel commented Feb 20, 2025

Description

Work-in-progress. Puppeteer seems like an excellent replacement for tape-run, and has built-in code coverage API the can generate nyc/istanbul-compatible output.

To test:

# in sjpp dir
npm i c8 -g # install globally for now, can track in `package.json` later if everything works out

npm i puppeteer --save-dev --workspace=proteinpaint/client
npm i puppeteer-to-istanbul --save-dev --workspace=proteinpaint/client
# may not need the tls reject option below
NODE_TLS_REJECT_UNAUTHORIZED=0 npx puppeteer browsers install chrome

# must have a running server for puppeteer test
npm run dev
  • in server dir, npm run test:unit, should see test coverage report like in the screenshot below
  • in client dir, npm run test:puppet, must have selenium-webdriver installed globally or in parent repo (like sjpp). The terminal logs should be the same as in a desktop browser console, when running the same tests

TODOs:

  • (DONE) generate test coverage report in browser environments (both desktop and headless)
  • add more configurations to include code files without a spec file, maybe add include/exclude filters
  • coverage report for client/dist chunk named files should use source-map file names, not chunk names
  • must run each spec file separately, and filter by matching filename pattern, to get accurate coverage results
  • have a PR-triggered CI to include coverage test requirements for changed files and line numbers (quicker), and a separate periodic CI action for coverage test across all files (takes longer to finish running)

Notes:

  • nyc/istanbul instrumentation lib does not seem to support esm import syntax, only cjs require(), so any imported code/spec files are not instrumented for coverage, as being reached during test execution
  • c8 uses V8-coverage, which uses the native coverage/profiler lib in V8 - the engine used by both Chrome browser and nodejs
  • The checking coverage readme section has command line options like c8 check-coverage --lines 95 --functions 95 --branches 95
  • for browser using testrun.html, may use Chrome Profiler from https://v8.dev/blog/javascript-code-coverage (not sure about this yet) EDIT: use puppeteer with istanbul

Screenshot of server unit test coverage, also logged in github unit test actions for this PR
Screenshot 2025-02-20 at 4 31 27 PM

Screenshot of client unit test coverage using puppeteer
Screenshot 2025-02-20 at 9 59 02 PM

Checklist

Check each task that has been performed or verified to be not applicable.

  • Tests: added and/or passed unit and integration tests, or N/A
  • Todos: commented or documented, or N/A
  • Notable Changes: updated release.txt, prefixed a commit message with "fix:" or "feat:", added to an internal tracking document, or N/A

…an alternative to tape runner, both use headless chrome under the hood
@siosonel siosonel requested review from aacic and creilly8 February 20, 2025 22:37
@siosonel siosonel force-pushed the test-coverage-runner branch from 0a00db5 to 5d1742e Compare February 21, 2025 03:51
@siosonel siosonel changed the title prototype using c8 as a test coverage lib; try selenium webdriver prototype using c8 as a test coverage lib; try alternative to tape-runner Feb 21, 2025
@siosonel siosonel force-pushed the test-coverage-runner branch from 5d1742e to 4b97d9a Compare February 21, 2025 04:09
@siosonel siosonel force-pushed the test-coverage-runner branch from 4b97d9a to 3072b6d Compare February 21, 2025 04:25
@siosonel siosonel changed the title prototype using c8 as a test coverage lib; try alternative to tape-runner prototype using c8 as a test coverage lib; try alternatives to tape-runner Feb 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant