Codecov #305
Open
Codecov #305
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Adds optional support for using nyrkio.com change detection service. Consequently gh-pages-branch and benchmark-data-dir-path are made optional parameters and if not given, the original method of storing results in github and calculating a threshold based alert is not executed. On the other hand if nyrkio-enabled is false (the default), Nyrkiö is not used. (Note: it's possible but probably not useful to enable both as this will just lead to more false positives, if anything.) In addition, this commit adds a new tool "criterion". There already was some kind of criterion support under the "cargo" tool, but it didn't work for the output.txt file I got from turso/limbo project. I don't have a lot of insight into what is the typical way of running criterion, so understand if this particular case is not representative. This implementation will work for post-commit runs. The Nyrkiö APIs for pull requests aren't used in this commit and you will probably create a mess if you enable this for pull requests. As usual, didn't add any testing but added my test file(s) under test/data. To be able to develop and test locally I added support for local git repo (which must be CWD). In fact, the branch name is often picked up from the local filesystem git repo. It is required in the Nyrkiö API but often absent in GitHub API objects.
Use branch from nyrkioJson instead. It's guaranteed to exist.
Turns out it's popular to simply use the entire SQL query as the test name. Including ? and several different quotes and apostrophes.
Chicken egg problem: How do you support a parameter while parsing parameters...
nyrkio-public nyrkio-settings-pvalue nyrkio-settings-threshold comment-always and comment-on-alert
For security reasons, to begin with...
Limbo project runs clickbench like this. And (unfortunately) this is a common way to implement simple benchmarks.
Note that tests don't actually pass yet. In nyrkio.ts a few details had drifted from the upstream data structures. (Such as using repo url vs commit url.) Here I decided to remain compatible with the original data structures. Fixes to nyrkio.ts will be in the next commit.
Plus added fields of course (repo and branch). Note, this breaks nyrkio.ts (no unit tests yet, as usual). Will fix that one in next commit.
… sanitized for URI
Remove all upstream contributors from alert-cc-on-comment
Biggest TODO: All workflows still only test the legacy graphs and threshold alerting. Original README preserved as LEGACY_README. Not sure why as it remains available upstream and we link to upstream.
Skip remaining validation (data.js) Make typescript happy...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.