Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: added GtfsRiderCategoriesSchema #1989

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

qcdyx
Copy link
Contributor

@qcdyx qcdyx commented Feb 26, 2025

Summary:

Closes #1979

Expected behavior:
image

Please make sure these boxes are checked before submitting your pull request - thanks!

  • Run the unit tests with gradle test to make sure you didn't break anything
  • Add or update any needed documentation to the repo
  • Format the title like "feat: [new feature short description]". Title must follow the Conventional Commit Specification(https://www.conventionalcommits.org/en/v1.0.0/).
  • Linked all relevant issues
  • Include screenshot(s) showing how this pull request works and fixes the issue(s)

@qcdyx qcdyx linked an issue Feb 26, 2025 that may be closed by this pull request
@qcdyx qcdyx requested a review from skalexch February 26, 2025 15:43
Copy link
Contributor

📝 Acceptance Test Report

📋 Summary

❌ The rule acceptance test has failed for commit dc69b88
Download the full acceptance test report here (report will disappear after 90 days).

📊 Notices Comparison

New Errors (146 out of 1821 datasets, ~8%) ❌

Details of new errors due to code change, which is above the provided threshold of 1%.

Dataset Notice Code
us-california-glenn-transit-service-gtfs-2207 foreign_key_violation
us-california-ridgerunner-gtfs-311 foreign_key_violation
ca-quebec-societe-des-transports-de-rimouski-gtfs-760 missing_required_column
nz-waikato-bus-it-waikato-gtfs-1233 missing_required_column
nz-wellington-east-by-west-ferry-gtfs-2305 missing_required_column
us-arizona-bullhead-area-transit-system-gtfs-109 missing_required_column
us-arizona-cat-tran-shuttle-gtfs-146 missing_required_column
us-arizona-groome-transportation-arizona-shuttle-gtfs-156 missing_required_column
us-california-airport-valet-express-gtfs-27 missing_required_column
us-california-alcatraz-cruises-hornblower-gtfs-595 missing_required_column
us-california-alhambra-community-transit-gtfs-211 missing_required_column
us-california-altamont-corridor-express-gtfs-1133 missing_required_column
us-california-arcadia-transit-gtfs-215 missing_required_column
us-california-arvin-transit-gtfs-216 missing_required_column
us-california-auburn-transit-gtfs-217 missing_required_column
us-california-auburn-transit-gtfs-2275 missing_required_column
us-california-baldwin-park-transit-gtfs-219 missing_required_column
us-california-bay-area-rapid-transit-bart-gtfs-53 missing_required_column
us-california-bell-gardens-trolley-gtfs-228 missing_required_column
us-california-bellflower-bus-gtfs-227 missing_required_column
us-california-big-blue-bus-gtfs-37 missing_required_column
us-california-blue-gold-fleet-gtfs-1178 missing_required_column
us-california-calabasas-transit-system-gtfs-296 missing_required_column
us-california-caltrain-gtfs-54 missing_required_column
us-california-carson-circuit-gtfs-298 missing_required_column
us-california-catalina-flyer-gtfs-300 missing_required_column
us-california-city-of-tracy-tracer-gtfs-877 missing_required_column
us-california-compton-renaissance-gtfs-314 missing_required_column
us-california-cudahy-area-rapid-transit-gtfs-318 missing_required_column
us-california-culver-city-bus-gtfs-38 missing_required_column
us-california-downeylink-gtfs-383 missing_required_column
us-california-duarte-transit-gtfs-105 missing_required_column
us-california-el-segundo-transportation-gtfs-2210 missing_required_column
us-california-el-segundo-transportation-gtfs-458 missing_required_column
us-california-fresno-county-rural-transit-agency-fcrta-gtfs-85 missing_required_column
us-california-get-around-town-express-gtfs-606 missing_required_column
us-california-glendora-transportation-division-gtfs-609 missing_required_column
us-california-glenn-transit-service-gtfs-835 missing_required_column
us-california-go-west-shuttle-gtfs-623 missing_required_column
us-california-gtrans-gtfs-616 missing_required_column
us-california-huntington-park-express-gtfs-558 missing_required_column
us-california-kings-area-rural-transit-gtfs-42 missing_required_column
us-california-la-campana-gtfs-562 missing_required_column
us-california-la-puente-link-gtfs-2250 missing_required_column
us-california-la-puente-link-gtfs-563 missing_required_column
us-california-lassen-rural-bus-lrb-gtfs-118 missing_required_column
us-california-lynwood-trolley-breeze-gtfs-570 missing_required_column
us-california-marin-transit-gtfs-71 missing_required_column
us-california-montebello-bus-lines-gtfs-577 missing_required_column
us-california-mountain-transit-gtfs-107 missing_required_column
us-california-north-county-transit-district-nctd-gtfs-14 missing_required_column
us-california-norwalk-transit-system-nts-gtfs-102 missing_required_column
us-california-pasadena-transit-gtfs-41 missing_required_column
us-california-petaluma-transit-gtfs-72 missing_required_column
us-california-playa-vista-shuttle-gtfs-799 missing_required_column
us-california-porterville-transit-gtfs-48 missing_required_column
us-california-redding-area-bus-authority-raba-gtfs-114 missing_required_column
us-california-rosemead-explorer-gtfs-806 missing_required_column
us-california-san-diego-international-airport-metropolitan-transit-system-mts-gtfs-13 missing_required_column
us-california-san-francisco-municipal-transportation-agency-sfmta-gtfs-50 missing_required_column
us-california-san-juan-capistrano-free-weekend-trolley-gtfs-596 missing_required_column
us-california-san-leandro-links-gtfs-810 missing_required_column
us-california-san-mateo-county-transit-district-samtrans-gtfs-49 missing_required_column
us-california-santa-cruz-metro-scmtd-gtfs-1211 missing_required_column
us-california-santa-rosa-citybus-gtfs-1986 missing_required_column
us-california-sierra-madre-gateway-coach-gtfs-829 missing_required_column
us-california-simi-valley-transit-gtfs-40 missing_required_column
us-california-sonoma-county-airport-express-gtfs-66 missing_required_column
us-california-south-county-transit-link-gtfs-817 missing_required_column
us-california-stanislaus-regional-transit-start-gtfs-87 missing_required_column
us-california-tideline-water-taxi-gtfs-52 missing_required_column
us-california-tulare-county-area-transit-gtfs-646 missing_required_column
us-california-union-city-transit-gtfs-628 missing_required_column
us-california-unitrans-gtfs-82 missing_required_column
us-california-university-of-california-berkeley-shuttle-bear-transit-gtfs-226 missing_required_column
us-california-west-berkeley-shuttle-gtfs-622 missing_required_column
us-colorado-bent-county-transit-gtfs-230 missing_required_column
us-colorado-boulder-county-gtfs-880 missing_required_column
us-colorado-bustang-gtfs-169 missing_required_column
us-colorado-continuum-gtfs-2165 missing_required_column
us-colorado-garden-of-the-gods-gtfs-879 missing_required_column
us-colorado-groome-transportation-colorado-springs-shuttle-gtfs-168 missing_required_column
us-colorado-san-miguel-authority-for-regional-transportation-smart-gtfs-811 missing_required_column
us-colorado-town-of-mountain-village-gtfs-594 missing_required_column
us-colorado-town-of-telluride-gtfs-603 missing_required_column
us-connecticut-middletown-area-transit-gtfs-576 missing_required_column
us-connecticut-southeast-area-transit-district-gtfs-552 missing_required_column
us-district-of-columbia-dc-streetcar-gtfs-487 missing_required_column
us-florida-escambia-county-area-transit-ecat-gtfs-335 missing_required_column
us-florida-gowal-gtfs-610 missing_required_column
us-florida-sunshine-bus-company-gtfs-350 missing_required_column
us-georgia-gwinnett-county-transit-gtfs-369 missing_required_column
us-illinois-greater-peoria-mass-transit-citylink-gtfs-386 missing_required_column
us-indiana-terre-haute-transit-utility-gtfs-387 missing_required_column
us-louisiana-capital-area-transit-system-cats-gtfs-2065 missing_required_column
us-maryland-maryland-transit-administration-gtfs-466 missing_required_column
us-maryland-maryland-transit-administration-gtfs-469 missing_required_column
us-maryland-maryland-transit-administration-metro-subway-gtfs-470 missing_required_column
us-massachusetts-128-business-council-gtfs-440 missing_required_column
us-massachusetts-bloom-bus-gtfs-2192 missing_required_column
us-massachusetts-coach-company-gtfs-436 missing_required_column
us-massachusetts-dattco-motorcoach-gtfs-555 missing_required_column
us-massachusetts-greater-attleboro-taunton-regional-transit-authority-gatra-gtfs-418 missing_required_column
us-massachusetts-limoliner-gtfs-438 missing_required_column
us-massachusetts-logan-express-gtfs-601 missing_required_column
us-mississipi-jtran-gtfs-155 missing_required_column
us-missouri-jeffco-express-gtfs-189 missing_required_column
us-montana-butte-silver-bow-gtfs-291 missing_required_column
us-montana-capital-transit-gtfs-310 missing_required_column
us-new-jersey-patco-speedline-gtfs-505 missing_required_column
us-new-mexico-sandia-shuttle-gtfs-167 missing_required_column
us-new-york-niagara-frontier-transportation-authority-nfta-gtfs-465 missing_required_column
us-north-carolina-concord-kannapolis-area-transit-rider-transit-gtfs-372 missing_required_column
us-north-carolina-north-carolina-state-university-ncsu-gtfs-1289 missing_required_column
us-oregon-blue-star-bus-gtfs-256 missing_required_column
us-oregon-cascade-point-gtfs-639 missing_required_column
us-oregon-city-2-city-shuttle-gtfs-129 missing_required_column
us-oregon-city-of-corvallis-gtfs-136 missing_required_column
us-oregon-eastern-point-gtfs-632 missing_required_column
us-oregon-klamath-shuttle-crater-lake-trolley-gtfs-124 missing_required_column
us-oregon-klamath-shuttle-gtfs-560 missing_required_column
us-oregon-mt-bachelor-gtfs-933 missing_required_column
us-oregon-northwest-point-gtfs-638 missing_required_column
us-oregon-philomath-connection-gtfs-878 missing_required_column
us-oregon-shuttle-oregon-gtfs-1284 missing_required_column
us-oregon-southwest-point-gtfs-637 missing_required_column
us-oregon-trimet-portland-streetcar-gtfs-247 missing_required_column
us-oregon-valley-retriever-busline-gtfs-633 missing_required_column
us-oregon-water-avenue-shuttle-gtfs-831 missing_required_column
us-south-carolina-clemson-area-transit-cat-gtfs-798 missing_required_column
us-south-carolina-the-comet-gtfs-1197 missing_required_column
us-texas-gulf-coast-center-gtfs-625 missing_required_column
us-unknown-groome-transportation-gtfs-319 missing_required_column
us-unknown-groome-transportation-gtfs-614 missing_required_column
us-utah-elevated-transit-gtfs-1207 missing_required_column
us-vermont-addison-county-transit-actr-gtfs-543 missing_required_column
us-vermont-moover-gtfs-434 missing_required_column
us-vermont-stagecoach-transportation-services-sts-gtfs-449 missing_required_column
us-vermont-the-current-gtfs-428 missing_required_column
us-virginia-virginia-breeze-gtfs-1328 missing_required_column
us-virginia-williamsburg-area-transit-authority-gtfs-901 missing_required_column
us-virginia-williamsburg-area-transit-authority-wata-gtfs-474 missing_required_column
us-washington-c-tran-gtfs-258 missing_required_column
us-washington-wahkiakum-ferry-gtfs-604 missing_required_column
us-wisconsin-belle-urban-system-gtfs-395 missing_required_column
us-wisconsin-beloit-transit-gtfs-392 missing_required_column
Dropped Errors (1 out of 1821 datasets, ~0%) ✅

Details of new errors due to code change, which is less than the provided threshold of 1%.

Dataset Notice Code
us-california-north-county-transit-district-nctd-gtfs-14 duplicate_key
New Warnings (0 out of 1821 datasets, ~0%) ✅

No changes were detected due to the code change.

Dropped Warnings (0 out of 1821 datasets, ~0%) ✅

No changes were detected due to the code change.

🛡️ Corruption Check

0 out of 1821 sources (~0 %) are corrupted.

⏱️ Performance Assessment

📈 Validation Time

Assess the performance in terms of seconds taken for the validation process.

Time Metric Dataset ID Reference (s) Latest (s) Difference (s)
Average -- 3.74 3.80 ⬆️+0.07
Median -- 1.36 1.41 ⬆️+0.06
Standard Deviation -- 10.89 10.99 ⬆️+0.10
Minimum in References Reports us-michigan-detroit-people-mover-gtfs-417 0.47 0.51 ⬆️+0.04
Maximum in Reference Reports gb-unknown-uk-aggregate-feed-gtfs-2014 285.77 288.07 ⬆️+2.30
Minimum in Latest Reports us-indiana-south-shore-line-gtfs-585 0.50 0.50 ⬇️-0.01
Maximum in Latest Reports gb-unknown-uk-aggregate-feed-gtfs-2014 285.77 288.07 ⬆️+2.30
📜 Memory Consumption
Metric Dataset ID Reference (s) Latest (s) Difference (s)
Average -- 466.54 MiB 465.86 MiB ⬇️-697.48 KiB
Median -- 335.92 MiB 335.92 MiB ⬇️0 bytes
Standard Deviation -- 767.04 MiB 766.54 MiB ⬇️-514.54 KiB
Minimum in References Reports us-california-redding-area-bus-authority-raba-gtfs-114 38.48 MiB 43.17 MiB ⬆️+4.69 MiB
Maximum in Reference Reports gb-unknown-uk-aggregate-feed-gtfs-2014 10.93 GiB 11.27 GiB ⬆️+346.97 MiB
Minimum in Latest Reports us-colorado-boulder-county-gtfs-2191 403.92 MiB 38.79 MiB ⬇️-365.13 MiB
Maximum in Latest Reports gb-unknown-uk-aggregate-feed-gtfs-2014 10.93 GiB 11.27 GiB ⬆️+346.97 MiB

@emmambd
Copy link
Contributor

emmambd commented Feb 26, 2025

@Sergiodero @skalexch I did some spot checks because the error % freaked me out - this looks like another case where Trillium's empty files are the culprit. This is good timing! I'll let them know and get their thoughts

Copy link

@skalexch skalexch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@qcdyx I looked at some of the affected feeds. Things are looking good! Also the dropped Error was something we predicted would drop once these rules were implemented.
Good on the spec side.

Copy link
Contributor

📝 Acceptance Test Report

📋 Summary

❌ The rule acceptance test has failed for commit 3b65cb7
Download the full acceptance test report here (report will disappear after 90 days).

📊 Notices Comparison

New Errors (146 out of 1822 datasets, ~8%) ❌

Details of new errors due to code change, which is above the provided threshold of 1%.

Dataset Notice Code
us-california-glenn-transit-service-gtfs-2207 foreign_key_violation
us-california-ridgerunner-gtfs-311 foreign_key_violation
ca-quebec-societe-des-transports-de-rimouski-gtfs-760 missing_required_column
nz-waikato-bus-it-waikato-gtfs-1233 missing_required_column
nz-wellington-east-by-west-ferry-gtfs-2305 missing_required_column
us-arizona-bullhead-area-transit-system-gtfs-109 missing_required_column
us-arizona-cat-tran-shuttle-gtfs-146 missing_required_column
us-arizona-groome-transportation-arizona-shuttle-gtfs-156 missing_required_column
us-california-airport-valet-express-gtfs-27 missing_required_column
us-california-alcatraz-cruises-hornblower-gtfs-595 missing_required_column
us-california-alhambra-community-transit-gtfs-211 missing_required_column
us-california-altamont-corridor-express-gtfs-1133 missing_required_column
us-california-arcadia-transit-gtfs-215 missing_required_column
us-california-arvin-transit-gtfs-216 missing_required_column
us-california-auburn-transit-gtfs-217 missing_required_column
us-california-auburn-transit-gtfs-2275 missing_required_column
us-california-baldwin-park-transit-gtfs-219 missing_required_column
us-california-bay-area-rapid-transit-bart-gtfs-53 missing_required_column
us-california-bell-gardens-trolley-gtfs-228 missing_required_column
us-california-bellflower-bus-gtfs-227 missing_required_column
us-california-big-blue-bus-gtfs-37 missing_required_column
us-california-blue-gold-fleet-gtfs-1178 missing_required_column
us-california-calabasas-transit-system-gtfs-296 missing_required_column
us-california-caltrain-gtfs-54 missing_required_column
us-california-carson-circuit-gtfs-298 missing_required_column
us-california-catalina-flyer-gtfs-300 missing_required_column
us-california-city-of-tracy-tracer-gtfs-877 missing_required_column
us-california-compton-renaissance-gtfs-314 missing_required_column
us-california-cudahy-area-rapid-transit-gtfs-318 missing_required_column
us-california-culver-city-bus-gtfs-38 missing_required_column
us-california-downeylink-gtfs-383 missing_required_column
us-california-duarte-transit-gtfs-105 missing_required_column
us-california-el-segundo-transportation-gtfs-2210 missing_required_column
us-california-el-segundo-transportation-gtfs-458 missing_required_column
us-california-fresno-county-rural-transit-agency-fcrta-gtfs-85 missing_required_column
us-california-get-around-town-express-gtfs-606 missing_required_column
us-california-glendora-transportation-division-gtfs-609 missing_required_column
us-california-glenn-transit-service-gtfs-835 missing_required_column
us-california-go-west-shuttle-gtfs-623 missing_required_column
us-california-gtrans-gtfs-616 missing_required_column
us-california-huntington-park-express-gtfs-558 missing_required_column
us-california-kings-area-rural-transit-gtfs-42 missing_required_column
us-california-la-campana-gtfs-562 missing_required_column
us-california-la-puente-link-gtfs-2250 missing_required_column
us-california-la-puente-link-gtfs-563 missing_required_column
us-california-lassen-rural-bus-lrb-gtfs-118 missing_required_column
us-california-lynwood-trolley-breeze-gtfs-570 missing_required_column
us-california-marin-transit-gtfs-71 missing_required_column
us-california-montebello-bus-lines-gtfs-577 missing_required_column
us-california-mountain-transit-gtfs-107 missing_required_column
us-california-north-county-transit-district-nctd-gtfs-14 missing_required_column
us-california-norwalk-transit-system-nts-gtfs-102 missing_required_column
us-california-pasadena-transit-gtfs-41 missing_required_column
us-california-petaluma-transit-gtfs-72 missing_required_column
us-california-playa-vista-shuttle-gtfs-799 missing_required_column
us-california-porterville-transit-gtfs-48 missing_required_column
us-california-redding-area-bus-authority-raba-gtfs-114 missing_required_column
us-california-rosemead-explorer-gtfs-806 missing_required_column
us-california-san-diego-international-airport-metropolitan-transit-system-mts-gtfs-13 missing_required_column
us-california-san-francisco-municipal-transportation-agency-sfmta-gtfs-50 missing_required_column
us-california-san-juan-capistrano-free-weekend-trolley-gtfs-596 missing_required_column
us-california-san-leandro-links-gtfs-810 missing_required_column
us-california-san-mateo-county-transit-district-samtrans-gtfs-49 missing_required_column
us-california-santa-cruz-metro-scmtd-gtfs-1211 missing_required_column
us-california-santa-rosa-citybus-gtfs-1986 missing_required_column
us-california-sierra-madre-gateway-coach-gtfs-829 missing_required_column
us-california-simi-valley-transit-gtfs-40 missing_required_column
us-california-sonoma-county-airport-express-gtfs-66 missing_required_column
us-california-south-county-transit-link-gtfs-817 missing_required_column
us-california-stanislaus-regional-transit-start-gtfs-87 missing_required_column
us-california-tideline-water-taxi-gtfs-52 missing_required_column
us-california-tulare-county-area-transit-gtfs-646 missing_required_column
us-california-union-city-transit-gtfs-628 missing_required_column
us-california-unitrans-gtfs-82 missing_required_column
us-california-university-of-california-berkeley-shuttle-bear-transit-gtfs-226 missing_required_column
us-california-west-berkeley-shuttle-gtfs-622 missing_required_column
us-colorado-bent-county-transit-gtfs-230 missing_required_column
us-colorado-boulder-county-gtfs-880 missing_required_column
us-colorado-bustang-gtfs-169 missing_required_column
us-colorado-continuum-gtfs-2165 missing_required_column
us-colorado-garden-of-the-gods-gtfs-879 missing_required_column
us-colorado-groome-transportation-colorado-springs-shuttle-gtfs-168 missing_required_column
us-colorado-san-miguel-authority-for-regional-transportation-smart-gtfs-811 missing_required_column
us-colorado-town-of-mountain-village-gtfs-594 missing_required_column
us-colorado-town-of-telluride-gtfs-603 missing_required_column
us-connecticut-middletown-area-transit-gtfs-576 missing_required_column
us-connecticut-southeast-area-transit-district-gtfs-552 missing_required_column
us-district-of-columbia-dc-streetcar-gtfs-487 missing_required_column
us-florida-escambia-county-area-transit-ecat-gtfs-335 missing_required_column
us-florida-gowal-gtfs-610 missing_required_column
us-florida-sunshine-bus-company-gtfs-350 missing_required_column
us-georgia-gwinnett-county-transit-gtfs-369 missing_required_column
us-illinois-greater-peoria-mass-transit-citylink-gtfs-386 missing_required_column
us-indiana-terre-haute-transit-utility-gtfs-387 missing_required_column
us-louisiana-capital-area-transit-system-cats-gtfs-2065 missing_required_column
us-maryland-maryland-transit-administration-gtfs-466 missing_required_column
us-maryland-maryland-transit-administration-gtfs-469 missing_required_column
us-maryland-maryland-transit-administration-metro-subway-gtfs-470 missing_required_column
us-massachusetts-128-business-council-gtfs-440 missing_required_column
us-massachusetts-bloom-bus-gtfs-2192 missing_required_column
us-massachusetts-coach-company-gtfs-436 missing_required_column
us-massachusetts-dattco-motorcoach-gtfs-555 missing_required_column
us-massachusetts-greater-attleboro-taunton-regional-transit-authority-gatra-gtfs-418 missing_required_column
us-massachusetts-limoliner-gtfs-438 missing_required_column
us-massachusetts-logan-express-gtfs-601 missing_required_column
us-mississipi-jtran-gtfs-155 missing_required_column
us-missouri-jeffco-express-gtfs-189 missing_required_column
us-montana-butte-silver-bow-gtfs-291 missing_required_column
us-montana-capital-transit-gtfs-310 missing_required_column
us-new-jersey-patco-speedline-gtfs-505 missing_required_column
us-new-mexico-sandia-shuttle-gtfs-167 missing_required_column
us-new-york-niagara-frontier-transportation-authority-nfta-gtfs-465 missing_required_column
us-north-carolina-concord-kannapolis-area-transit-rider-transit-gtfs-372 missing_required_column
us-north-carolina-north-carolina-state-university-ncsu-gtfs-1289 missing_required_column
us-oregon-blue-star-bus-gtfs-256 missing_required_column
us-oregon-cascade-point-gtfs-639 missing_required_column
us-oregon-city-2-city-shuttle-gtfs-129 missing_required_column
us-oregon-city-of-corvallis-gtfs-136 missing_required_column
us-oregon-eastern-point-gtfs-632 missing_required_column
us-oregon-klamath-shuttle-crater-lake-trolley-gtfs-124 missing_required_column
us-oregon-klamath-shuttle-gtfs-560 missing_required_column
us-oregon-mt-bachelor-gtfs-933 missing_required_column
us-oregon-northwest-point-gtfs-638 missing_required_column
us-oregon-philomath-connection-gtfs-878 missing_required_column
us-oregon-shuttle-oregon-gtfs-1284 missing_required_column
us-oregon-southwest-point-gtfs-637 missing_required_column
us-oregon-trimet-portland-streetcar-gtfs-247 missing_required_column
us-oregon-valley-retriever-busline-gtfs-633 missing_required_column
us-oregon-water-avenue-shuttle-gtfs-831 missing_required_column
us-south-carolina-clemson-area-transit-cat-gtfs-798 missing_required_column
us-south-carolina-the-comet-gtfs-1197 missing_required_column
us-texas-gulf-coast-center-gtfs-625 missing_required_column
us-unknown-groome-transportation-gtfs-319 missing_required_column
us-unknown-groome-transportation-gtfs-614 missing_required_column
us-utah-elevated-transit-gtfs-1207 missing_required_column
us-vermont-addison-county-transit-actr-gtfs-543 missing_required_column
us-vermont-moover-gtfs-434 missing_required_column
us-vermont-stagecoach-transportation-services-sts-gtfs-449 missing_required_column
us-vermont-the-current-gtfs-428 missing_required_column
us-virginia-virginia-breeze-gtfs-1328 missing_required_column
us-virginia-williamsburg-area-transit-authority-gtfs-901 missing_required_column
us-virginia-williamsburg-area-transit-authority-wata-gtfs-474 missing_required_column
us-washington-c-tran-gtfs-258 missing_required_column
us-washington-wahkiakum-ferry-gtfs-604 missing_required_column
us-wisconsin-belle-urban-system-gtfs-395 missing_required_column
us-wisconsin-beloit-transit-gtfs-392 missing_required_column
Dropped Errors (1 out of 1822 datasets, ~0%) ✅

Details of new errors due to code change, which is less than the provided threshold of 1%.

Dataset Notice Code
us-california-north-county-transit-district-nctd-gtfs-14 duplicate_key
New Warnings (0 out of 1822 datasets, ~0%) ✅

No changes were detected due to the code change.

Dropped Warnings (0 out of 1822 datasets, ~0%) ✅

No changes were detected due to the code change.

🛡️ Corruption Check

0 out of 1822 sources (~0 %) are corrupted.

⏱️ Performance Assessment

📈 Validation Time

Assess the performance in terms of seconds taken for the validation process.

Time Metric Dataset ID Reference (s) Latest (s) Difference (s)
Average -- 3.85 3.95 ⬆️+0.11
Median -- 1.42 1.48 ⬆️+0.06
Standard Deviation -- 11.21 11.16 ⬇️-0.05
Minimum in References Reports us-california-flex-v2-developer-test-feed-2-gtfs-1818 0.50 0.60 ⬆️+0.10
Maximum in Reference Reports gb-unknown-uk-aggregate-feed-gtfs-2014 297.16 289.03 ⬇️-8.13
Minimum in Latest Reports us-california-flex-v2-developer-test-feed-3-gtfs-1819 0.51 0.50 ⬇️-0.01
Maximum in Latest Reports gb-unknown-uk-aggregate-feed-gtfs-2014 297.16 289.03 ⬇️-8.13
📜 Memory Consumption
Metric Dataset ID Reference (s) Latest (s) Difference (s)
Average -- 471.26 MiB 462.42 MiB ⬇️-8.84 MiB
Median -- 334.15 MiB 335.92 MiB ⬆️+1.77 MiB
Standard Deviation -- 815.91 MiB 768.12 MiB ⬇️-47.79 MiB
Minimum in References Reports us-colorado-via-mobility-gtfs-180 39.04 MiB 415.92 MiB ⬆️+376.88 MiB
Maximum in Reference Reports gb-unknown-uk-aggregate-feed-gtfs-2014 11.05 GiB 11.37 GiB ⬆️+333.17 MiB
Minimum in Latest Reports us-massachusetts-marthas-vineyard-transit-authority-gtfs-420 47.10 MiB 39.45 MiB ⬇️-7.65 MiB
Maximum in Latest Reports gb-unknown-uk-aggregate-feed-gtfs-2014 11.05 GiB 11.37 GiB ⬆️+333.17 MiB

@emmambd
Copy link
Contributor

emmambd commented Feb 26, 2025

Hey @qcdyx! The one feed that we don't understand why it's returning an error is https://mobilitydatabase.org/feeds/mdb-258. It seems to include all required fields and should not return a missing_required_column error. Can you investigate? cc @davidgamez

String riderCategoryName();

@Required
GtfsRiderCategory isDefaultRiderCategory();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@qcdyx @skalexch figured it out! It looks like this needs to be isDefaultFareCategory not isDefaultRiderCategory in order to match the spec: https://github.com/google/transit/pull/511/files#diff-3ecf0760eb54b4953728042a1e30586705dc2335807be94faae0de5829cd12a1R411 and make sure https://mobilitydatabase.org/feeds/mdb-258 doesn't trigger the error.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, it is rider_categories.is_default_rider_category in the requirement. Shall I proceed to make the change?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see what you mean from the requirements. Yes please - the spec is always the source of truth, so is_default_fare_category is the right field name.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done! image

Copy link
Contributor

📝 Acceptance Test Report

📋 Summary

❌ The rule acceptance test has failed for commit d8646a4
Download the full acceptance test report here (report will disappear after 90 days).

📊 Notices Comparison

New Errors (144 out of 1822 datasets, ~8%) ❌

Details of new errors due to code change, which is above the provided threshold of 1%.

Dataset Notice Code
us-california-glenn-transit-service-gtfs-2207 foreign_key_violation
us-california-ridgerunner-gtfs-311 foreign_key_violation
ca-quebec-societe-des-transports-de-rimouski-gtfs-760 missing_required_column
nz-waikato-bus-it-waikato-gtfs-1233 missing_required_column
nz-wellington-east-by-west-ferry-gtfs-2305 missing_required_column
us-arizona-bullhead-area-transit-system-gtfs-109 missing_required_column
us-arizona-cat-tran-shuttle-gtfs-146 missing_required_column
us-arizona-groome-transportation-arizona-shuttle-gtfs-156 missing_required_column
us-california-airport-valet-express-gtfs-27 missing_required_column
us-california-alcatraz-cruises-hornblower-gtfs-595 missing_required_column
us-california-alhambra-community-transit-gtfs-211 missing_required_column
us-california-altamont-corridor-express-gtfs-1133 missing_required_column
us-california-arcadia-transit-gtfs-215 missing_required_column
us-california-arvin-transit-gtfs-216 missing_required_column
us-california-auburn-transit-gtfs-217 missing_required_column
us-california-auburn-transit-gtfs-2275 missing_required_column
us-california-baldwin-park-transit-gtfs-219 missing_required_column
us-california-bay-area-rapid-transit-bart-gtfs-53 missing_required_column
us-california-bell-gardens-trolley-gtfs-228 missing_required_column
us-california-bellflower-bus-gtfs-227 missing_required_column
us-california-big-blue-bus-gtfs-37 missing_required_column
us-california-blue-gold-fleet-gtfs-1178 missing_required_column
us-california-calabasas-transit-system-gtfs-296 missing_required_column
us-california-caltrain-gtfs-54 missing_required_column
us-california-carson-circuit-gtfs-298 missing_required_column
us-california-catalina-flyer-gtfs-300 missing_required_column
us-california-city-of-tracy-tracer-gtfs-877 missing_required_column
us-california-compton-renaissance-gtfs-314 missing_required_column
us-california-cudahy-area-rapid-transit-gtfs-318 missing_required_column
us-california-downeylink-gtfs-383 missing_required_column
us-california-duarte-transit-gtfs-105 missing_required_column
us-california-el-segundo-transportation-gtfs-2210 missing_required_column
us-california-el-segundo-transportation-gtfs-458 missing_required_column
us-california-fresno-county-rural-transit-agency-fcrta-gtfs-85 missing_required_column
us-california-get-around-town-express-gtfs-606 missing_required_column
us-california-glendora-transportation-division-gtfs-609 missing_required_column
us-california-glenn-transit-service-gtfs-835 missing_required_column
us-california-go-west-shuttle-gtfs-623 missing_required_column
us-california-gtrans-gtfs-616 missing_required_column
us-california-huntington-park-express-gtfs-558 missing_required_column
us-california-kings-area-rural-transit-gtfs-42 missing_required_column
us-california-la-campana-gtfs-562 missing_required_column
us-california-la-puente-link-gtfs-2250 missing_required_column
us-california-la-puente-link-gtfs-563 missing_required_column
us-california-lassen-rural-bus-lrb-gtfs-118 missing_required_column
us-california-lynwood-trolley-breeze-gtfs-570 missing_required_column
us-california-marin-transit-gtfs-71 missing_required_column
us-california-montebello-bus-lines-gtfs-577 missing_required_column
us-california-mountain-transit-gtfs-107 missing_required_column
us-california-north-county-transit-district-nctd-gtfs-14 missing_required_column
us-california-norwalk-transit-system-nts-gtfs-102 missing_required_column
us-california-pasadena-transit-gtfs-41 missing_required_column
us-california-petaluma-transit-gtfs-72 missing_required_column
us-california-playa-vista-shuttle-gtfs-799 missing_required_column
us-california-porterville-transit-gtfs-48 missing_required_column
us-california-redding-area-bus-authority-raba-gtfs-114 missing_required_column
us-california-rosemead-explorer-gtfs-806 missing_required_column
us-california-san-diego-international-airport-metropolitan-transit-system-mts-gtfs-13 missing_required_column
us-california-san-francisco-municipal-transportation-agency-sfmta-gtfs-50 missing_required_column
us-california-san-juan-capistrano-free-weekend-trolley-gtfs-596 missing_required_column
us-california-san-leandro-links-gtfs-810 missing_required_column
us-california-san-mateo-county-transit-district-samtrans-gtfs-49 missing_required_column
us-california-santa-cruz-metro-scmtd-gtfs-1211 missing_required_column
us-california-santa-rosa-citybus-gtfs-1986 missing_required_column
us-california-sierra-madre-gateway-coach-gtfs-829 missing_required_column
us-california-simi-valley-transit-gtfs-40 missing_required_column
us-california-sonoma-county-airport-express-gtfs-66 missing_required_column
us-california-south-county-transit-link-gtfs-817 missing_required_column
us-california-stanislaus-regional-transit-start-gtfs-87 missing_required_column
us-california-tideline-water-taxi-gtfs-52 missing_required_column
us-california-tulare-county-area-transit-gtfs-646 missing_required_column
us-california-union-city-transit-gtfs-628 missing_required_column
us-california-unitrans-gtfs-82 missing_required_column
us-california-university-of-california-berkeley-shuttle-bear-transit-gtfs-226 missing_required_column
us-california-west-berkeley-shuttle-gtfs-622 missing_required_column
us-colorado-bent-county-transit-gtfs-230 missing_required_column
us-colorado-boulder-county-gtfs-880 missing_required_column
us-colorado-bustang-gtfs-169 missing_required_column
us-colorado-continuum-gtfs-2165 missing_required_column
us-colorado-garden-of-the-gods-gtfs-879 missing_required_column
us-colorado-groome-transportation-colorado-springs-shuttle-gtfs-168 missing_required_column
us-colorado-san-miguel-authority-for-regional-transportation-smart-gtfs-811 missing_required_column
us-colorado-town-of-mountain-village-gtfs-594 missing_required_column
us-colorado-town-of-telluride-gtfs-603 missing_required_column
us-connecticut-middletown-area-transit-gtfs-576 missing_required_column
us-connecticut-southeast-area-transit-district-gtfs-552 missing_required_column
us-district-of-columbia-dc-streetcar-gtfs-487 missing_required_column
us-florida-escambia-county-area-transit-ecat-gtfs-335 missing_required_column
us-florida-gowal-gtfs-610 missing_required_column
us-florida-sunshine-bus-company-gtfs-350 missing_required_column
us-georgia-gwinnett-county-transit-gtfs-369 missing_required_column
us-illinois-greater-peoria-mass-transit-citylink-gtfs-386 missing_required_column
us-indiana-terre-haute-transit-utility-gtfs-387 missing_required_column
us-louisiana-capital-area-transit-system-cats-gtfs-2065 missing_required_column
us-maryland-maryland-transit-administration-gtfs-466 missing_required_column
us-maryland-maryland-transit-administration-gtfs-469 missing_required_column
us-maryland-maryland-transit-administration-metro-subway-gtfs-470 missing_required_column
us-massachusetts-128-business-council-gtfs-440 missing_required_column
us-massachusetts-bloom-bus-gtfs-2192 missing_required_column
us-massachusetts-coach-company-gtfs-436 missing_required_column
us-massachusetts-dattco-motorcoach-gtfs-555 missing_required_column
us-massachusetts-greater-attleboro-taunton-regional-transit-authority-gatra-gtfs-418 missing_required_column
us-massachusetts-limoliner-gtfs-438 missing_required_column
us-massachusetts-logan-express-gtfs-601 missing_required_column
us-mississipi-jtran-gtfs-155 missing_required_column
us-missouri-jeffco-express-gtfs-189 missing_required_column
us-montana-butte-silver-bow-gtfs-291 missing_required_column
us-montana-capital-transit-gtfs-310 missing_required_column
us-new-jersey-patco-speedline-gtfs-505 missing_required_column
us-new-mexico-sandia-shuttle-gtfs-167 missing_required_column
us-new-york-niagara-frontier-transportation-authority-nfta-gtfs-465 missing_required_column
us-north-carolina-concord-kannapolis-area-transit-rider-transit-gtfs-372 missing_required_column
us-north-carolina-north-carolina-state-university-ncsu-gtfs-1289 missing_required_column
us-oregon-blue-star-bus-gtfs-256 missing_required_column
us-oregon-cascade-point-gtfs-639 missing_required_column
us-oregon-city-2-city-shuttle-gtfs-129 missing_required_column
us-oregon-city-of-corvallis-gtfs-136 missing_required_column
us-oregon-eastern-point-gtfs-632 missing_required_column
us-oregon-klamath-shuttle-crater-lake-trolley-gtfs-124 missing_required_column
us-oregon-klamath-shuttle-gtfs-560 missing_required_column
us-oregon-mt-bachelor-gtfs-933 missing_required_column
us-oregon-northwest-point-gtfs-638 missing_required_column
us-oregon-philomath-connection-gtfs-878 missing_required_column
us-oregon-shuttle-oregon-gtfs-1284 missing_required_column
us-oregon-southwest-point-gtfs-637 missing_required_column
us-oregon-valley-retriever-busline-gtfs-633 missing_required_column
us-oregon-water-avenue-shuttle-gtfs-831 missing_required_column
us-south-carolina-clemson-area-transit-cat-gtfs-798 missing_required_column
us-south-carolina-the-comet-gtfs-1197 missing_required_column
us-texas-gulf-coast-center-gtfs-625 missing_required_column
us-unknown-groome-transportation-gtfs-319 missing_required_column
us-unknown-groome-transportation-gtfs-614 missing_required_column
us-utah-elevated-transit-gtfs-1207 missing_required_column
us-vermont-addison-county-transit-actr-gtfs-543 missing_required_column
us-vermont-moover-gtfs-434 missing_required_column
us-vermont-stagecoach-transportation-services-sts-gtfs-449 missing_required_column
us-vermont-the-current-gtfs-428 missing_required_column
us-virginia-virginia-breeze-gtfs-1328 missing_required_column
us-virginia-williamsburg-area-transit-authority-gtfs-901 missing_required_column
us-virginia-williamsburg-area-transit-authority-wata-gtfs-474 missing_required_column
us-washington-wahkiakum-ferry-gtfs-604 missing_required_column
us-wisconsin-belle-urban-system-gtfs-395 missing_required_column
us-wisconsin-beloit-transit-gtfs-392 missing_required_column
us-california-culver-city-bus-gtfs-38 missing_required_field
Dropped Errors (1 out of 1822 datasets, ~0%) ✅

Details of new errors due to code change, which is less than the provided threshold of 1%.

Dataset Notice Code
us-california-north-county-transit-district-nctd-gtfs-14 duplicate_key
New Warnings (0 out of 1822 datasets, ~0%) ✅

No changes were detected due to the code change.

Dropped Warnings (0 out of 1822 datasets, ~0%) ✅

No changes were detected due to the code change.

🛡️ Corruption Check

0 out of 1822 sources (~0 %) are corrupted.

⏱️ Performance Assessment

📈 Validation Time

Assess the performance in terms of seconds taken for the validation process.

Time Metric Dataset ID Reference (s) Latest (s) Difference (s)
Average -- 3.71 3.77 ⬆️+0.06
Median -- 1.35 1.39 ⬆️+0.04
Standard Deviation -- 11.14 11.07 ⬇️-0.06
Minimum in References Reports us-california-flex-v2-developer-test-feed-3-gtfs-1819 0.46 0.57 ⬆️+0.11
Maximum in Reference Reports gb-unknown-uk-aggregate-feed-gtfs-2014 303.96 299.54 ⬇️-4.42
Minimum in Latest Reports us-oregon-high-desert-point-gtfs-636 0.49 0.50 ⬆️+0.01
Maximum in Latest Reports gb-unknown-uk-aggregate-feed-gtfs-2014 303.96 299.54 ⬇️-4.42

⚠️ Warnings

The following dataset IDs are missing validation times either in reference or latest:
ca-alberta-lethbridge-transit-gtfs-1997

📜 Memory Consumption
Metric Dataset ID Reference (s) Latest (s) Difference (s)
Average -- 470.32 MiB 462.48 MiB ⬇️-7.85 MiB
Median -- 334.88 MiB 335.92 MiB ⬆️+1.04 MiB
Standard Deviation -- 792.39 MiB 746.83 MiB ⬇️-45.56 MiB
Minimum in References Reports us-california-laguna-beach-transit-lbt-gtfs-16 39.60 MiB 411.92 MiB ⬆️+372.32 MiB
Maximum in Reference Reports gb-unknown-uk-aggregate-feed-gtfs-2014 10.94 GiB 10.99 GiB ⬆️+49.75 MiB
Minimum in Latest Reports us-california-merced-county-transit-the-bus-gtfs-86 407.92 MiB 40.29 MiB ⬇️-367.63 MiB
Maximum in Latest Reports gb-unknown-uk-aggregate-feed-gtfs-2014 10.94 GiB 10.99 GiB ⬆️+49.75 MiB

@emmambd
Copy link
Contributor

emmambd commented Feb 26, 2025

Hey @qcdyx! One more issue here re: the spec:

In the original requirements, it's specified the empty is a valid value for is_default_fare_category:
Screenshot 2025-02-26 at 4 43 10 PM

In looking at the Culver City Bus example where it's now returning a missing_required_field notice, it should be considered valid data because the column is provided, it's just empty:

Screenshot 2025-02-26 at 4 41 52 PM

Can you add empty as a valid value? Once that's complete, this should be good for QA on the spec side. cc @skalexch @Sergiodero

@davidgamez
Copy link
Member

In the original requirements, it's specified the empty is a valid value for is_default_fare_category:

In other words, 0 is the default value; is this correct?

@emmambd
Copy link
Contributor

emmambd commented Feb 26, 2025

@davidgamez Exactly, yes. The spec says :"0 or empty" or "1" are the options.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add generic rules for rider_categories
4 participants