Evaluate alternative provider measurement #518
Conversation
There was a problem hiding this comment.
Copilot reviewed 7 out of 8 changed files in this pull request and generated no comments.
Files not reviewed (1)
- migrations/026.do.add-daily-network-retrieval-stats-table.sql: Language not supported
Comments suppressed due to low confidence (1)
lib/preprocess.js:20
- Consider using getNetworkRetrievalResult(m) instead of getRetrievalResult(m) when setting networkRetrievalResult to ensure that network-specific logic is applied.
this.networkRetrievalResult = pointerize(getRetrievalResult(m))
There was a problem hiding this comment.
Copilot reviewed 8 out of 9 changed files in this pull request and generated no comments.
Files not reviewed (1)
- migrations/026.do.add-daily-network-retrieval-stats-table.sql: Language not supported
Comments suppressed due to low confidence (2)
test/public-stats.test.js:1173
- [nitpick] There is an extra trailing whitespace in the describe block title; consider removing it for consistency.
describe('updateDailyNetworkRetrievalStats ', () => {
lib/retrieval-stats.js:129
- [nitpick] Ensure that using 'totalForRSR' as the denominator for the network success rate accurately reflects the intended measurement subset. If only measurements with network retrieval data should be considered, you might need to filter the denominator accordingly.
telemetryPoint.floatField('network_success_rate', networkRSR)
pyropy
requested review from
bajtos,
juliangruber and
NikolasHaimerl
as code owners
pyropy
changed the title
lib/preprocess.js
Outdated
| @@ -41,6 +42,13 @@ export class Measurement { | |||
| this.stationId = pointerize(m.station_id) | |||
| this.carChecksum = pointerize(m.car_checksum) | |||
| this.carTooLarge = m.car_too_large | |||
| this.networkRetrieval = { | |||
There was a problem hiding this comment.
If we agree to use a different name than "networkRetrieval", then we need to rename this property too.
See CheckerNetwork/spark-checker#132 (comment) and CheckerNetwork/spark-checker#132 (comment)
Plus, all other places that use the name "networkRetrieval" 😢
+1 to figure out naming first. Could you please link to the relevant PRs? I found CheckerNetwork/spark-checker#132 in your comment |
pyropy
changed the title
…m:filecoin-station/spark-evaluate into add/evalute-network-retrieval-measurement
|
Should we evaluate alternative provider retrieval measurements separately using separate committees? There has been quite a few mentions of using pseudo-random number generation when picking alt provider in order to measurements into committees in CheckerNetwork/spark-checker#132. |
This pull request introduces a new metric to track and report alternative provider retrieval statistics in addition to existing retrieval metrics. New metric is calculated based on the
alternativeProviderRetrievalResultstatus. ThealternativeProviderRetrievalResultfield is evalulated using new fields introduced by CheckerNetwork/spark-checker#132. If new fields are not present in the reported measurement, evaluation falls back to using already present metric fields.Changelog
Enhancements to retrieval tracking:
alternativeProviderRetrievalResultandalternativeProviderCheckproperties to theMeasurementclass, and created thegetAlternativeProviderRetrievalResultfunction to evaluate alternative provider retrieval metrics.updateDailyAlternativeProviderRetrievalStatsfunction to update daily alternative provider retrieval statistics and included its invocation in theupdatePublicStatsfunction.buildRetrievalStatsto include alternative retrieval success rate calculations.daily_alternative_provider_retrieval_statsto store daily alternative provider retrieval statistics.Closes #517
Related to: