Fix/bootstrap update

[duckdoom5]

• Fixes the bootstrap code generator and updates the generated source and bindings for the new generated source
• Introduces a workflow that allows you to update the parser bindings by building with the old source and hot patching it before the source is parsed so we can live update the bindings.
  • Might be an idea to automatically build and run these now as well
• Some additional code cleanup is included

Depends on these PRs to fix the bootstrap codegen:

• Fix crash on auto return types #1898
• Fix invalid enum base types being generated in C# source #1899
• Fix incorrect tag generation for enum classes #1900
• Fix crash on nullptr TemplateTypeParm decl #1901
• Add marshalling for std::optional <=> System::Nullable #1902

Note that these are included in the current PR. I'll rebase this PR and remove the draft marker once each is merged

[duckdoom5]

@tritao I'll just leave this here since then I can provide a bit of context. I've made these changes when I was working on this and just converted them into individual PR's.
I'll add tests to those requested and some might need a little bit of love :)

[duckdoom5]

Also, I've continued using the 'old' method here, but I assume we could also update the bootstrap source generator to use the newer methods right?

[tritao]

Also, I've continued using the 'old' method here, but I assume we could also update the bootstrap source generator to use the newer methods right?

Can you clarify what you mean by this?

[duckdoom5]

Yeah I mean like the way types are mapped, ignored and renamed and such.
For example the ParserGen is using a more 'modern' API with the generator passes and ctx.RenameNamespace and other helper functions etc.

[duckdoom5]

Btw, I have a question about the requirements on generated source. What promises are we making about the language version of the generated source? I kinda want to look into modernizing the output (like inline field initialization instead of using a constructor, which can then be marked = default etc.)

[tritao]

Ah yes, I see what you mean now. I guess both approaches can have their pros/cons, but if you think the other approach is preferable and more readable here, feel free to go for it.

And overall really amazing work on this, everything looks great, think it should be ready to merge when all the dependencies are in.

[tritao]

Btw, I have a question about the requirements on generated source. What promises are we making about the language version of the generated source? I kinda want to look into modernizing the output (like inline field initialization instead of using a constructor, which can then be marked = default etc.)

There is no formal promise I guess, but if it compiles in the supported CI environments, then it should be fine. Windows CI action is on 17.12.35707.178, which seems to be pretty recent. Personally I am not on using Windows lately, but it would probably be reasonable to keep compatibility with a somewhat older release too.

[duckdoom5]

@tritao So quick question; Can I convert Parser.cpp's AST walking with clang's build-in ASTNodeTraverser?

Seems like it would avoid many of the issues we're facing now

[tritao]

@tritao So quick question; Can I convert Parser.cpp's AST walking with clang's build-in ASTNodeTraverser?

Seems like it would avoid many of the issues we're facing now

I think that would be great, I don't think it was available at the time that code was written, so we had to come up with our own traversing solution, but nowadays re-using Clang's would definitely be the way to go.

[duckdoom5]

Awesome, thought as much. I'll get on that then :)

[tritao]

Also now that we have the bootstrap tool up-to-date, I wonder if it could be used to also generate the code for using ASTNodeTraverser.

[duckdoom5]

Yeah, I was thinking the same. But I think I'll first get it working by coding it manually and then see if there's a way to generate that

[duckdoom5]

Okay, I'll change that plan a bit.

I've made a start now and have a plan of attack so I know what I need/want. I think I'll just get commit #1900 working with a small compromise and then get this PR merged as well.

That should allow me to at least work with that updated. Then I might just generate some of that code to reduce the workload. I think what I should do then, is first get the current version fully auto generated and then get to the AST node traverser update (since it's quite a bit of code to write XD)

[tritao]

Is this ready to go now?

[duckdoom5]

No not quite. I noticed there's a build error that didn't popup locally. It's because CI builds the csharp bindings while local windows uses c++/cli.

For the past 3 days I was working on getting that fixed, but I'm having trouble with getting those bindings to generate properly.

[tritao]

I would be fine with just switching everything to C# bindings, if that makes it any easier.

[duckdoom5]

No it's the other way around. The c++/cli code works great, but the c# p/invoke (std) bindings have build errors currently

D:\a\CppSharp\CppSharp\src\CppParser\Bindings\CSharp\x86_64-pc-win32-msvc-d\CppSharp.CppParser.cs(40217,42): error CS0234: The type or namespace name 'Optional' does not exist in the namespace 'Std' (are you missing an assembly reference?) [D:\a\CppSharp\CppSharp\src\CppParser\Bindings\CSharp\CppSharp.Parser.CSharp.csproj]
D:\a\CppSharp\CppSharp\src\CppParser\Bindings\CSharp\x86_64-pc-win32-msvc-d\CppSharp.CppParser.cs(40512,42): error CS0234: The type or namespace name 'Optional' does not exist in the namespace 'Std' (are you missing an assembly reference?) [D:\a\CppSharp\CppSharp\src\CppParser\Bindings\CSharp\CppSharp.Parser.CSharp.csproj]
D:\a\CppSharp\CppSharp\src\CppParser\Bindings\CSharp\x86_64-pc-win32-msvc-d\CppSharp.CppParser.cs(41103,42): error CS0234: The type or namespace name 'Optional' does not exist in the namespace 'Std' (are you missing an assembly reference?) [D:\a\CppSharp\CppSharp\src\CppParser\Bindings\CSharp\CppSharp.Parser.CSharp.csproj]