feat(autocomplete): add type generator from the php yaml definitions

[nirinchev]

Description

Adds generators for the aggregation stages using the yaml definitions from https://github.com/mongodb-js/mongo-php-library/tree/v1.x/generator/config.

Three generators are added to process the yaml definitions:

1. schema-generator.ts generates a typescript schema from the yaml definitions which can be used to autocomplete aggregation stages. Notable limitations of the current schema include:
   • Unable to autocomplete nested fields e.g. $unset: [ 'foo.bar' ]
   • Stages following $addFields, $project, etc. will not reflect the new collection schema
   • Certain stages add variables which are not visible in the autocompletion (e.g. $lookup.let)
   • Some types don't have static schema defined (e.g. SearchScore, GeoPoint, etc.)
2. driver-schema-generator.ts generates the schema for the tests in the php yaml library. It does this by infering the schema from the documentation.
3. test-generator.ts generates tests from the yaml definitions. Those are not functional tests, but rather verify that the code compiles - that is, the schema definition does not generate false negatives. They don't attempt to verify that the autocompletion always suggests precise results though - i.e. a schema definition where everything is any would pass all the tests in this PR. Validating autocomplete results are helpful is a follow-up task that will be done when we integrate all the autocomplete tickets together.

Open Questions

We probably want to get mongodb/mongo-php-library#1659 eventually merged to ensure we don't diverge too far from upstream.

[addaleax]

Amazing work @nirinchev! ✨

[lerouxb]

We're gonna have to include mongodb in the ts language server so this can be imported by mongodb-ts-autocomplete. Which I'll just do for now, but we might want to consider using api-extractor to just include this type - the extracted bson&mongodb is a half megabyte string that we'll be parsing at runtime when we start up autocomplete:

-rw-r--r--@ 1 leroux.bodenstein  staff  416536  9 Jun 15:12 autocomplete-types.ts

I don't think that's something for this PR, though. Just thought I'd mention it.

[addaleax]

Maybe for now we could just copy this type and the subtypes that make it up into this file?

[lerouxb]

I looked into it and it looked like there was a bit of a dependency chain involved. I didn't reach the end before I realised "this is what api extractor was made for"..

[lerouxb]

Actually I just thought about this some more and I don't think we'll be including mql directly in mongodb-ts-autocomplete. We'll use it in the shell-api and probably just instruct api-extractor to include the relevant types because in the mongosh case that's the only way we'll be consuming it, same as for the bson expressions currently.

Later when we add support for other editors like a document editor or filter editor (in the compass querybar) or an aggregation editor we'll probably make something that bundles that up with whatever other shell syntax stuff we need (bson expressions?) and then re-export it from there.

So I think we don't even need "mql exported as a giant string" and there's no need to worry about importing bson and/or mongodb because that will just be handled downstream.

[lerouxb]

Great work! Like I explained I think we probably don't need 7fc9890 anymore. Up to you if you want to remove it again.

[lerouxb]

I think we need an export of schema.ts here somewhere so that we can import it from shell-api. Similar to what I started doing here and what we do in shell api here.

(Except the real thing so it can be imported, not the giant string.)