Benchmarks code gen vs reflection

[WhosNickDoglio]

I'd be curious about what this affect build times compared to using a reflection based mocking library like mockk vs having the code generated fakes like this

[anthonycr]

I would like to figure out a good way to actually benchmark this, as in this repo it's basically

:sample:kspTestKotlin + :sample:compileTestKotlin + :sample:test (unscientific average of 3 runs of this is ~2469ms)

vs

:sample:compileTestKotlin + :sample:test (unscientific average 3 runs of this is ~2930ms)

[anthonycr]

In theory the savings could be better if :sample:kspTestKotlin and :sample:compileTestKotlin are both cached but the tests still need to run, as the :sample:test run is at least 3x as long (on larger tests i have seen it be more like 5x)

[WhosNickDoglio]

Gradle profiler would be good for this. We could profile the test task for projects that use mockingbird vs mockk.

It might also be helpful to write a script to generate a large project (100+ modules) to do a more thorough benchmark. The Tangle docs have that which is nice.

[anthonycr]

I ran some preliminary basic benchmarks of clean :sample:test while disabling the tests that expect exceptions and adding in dagger code gen to normalize the difference in compile speed of not having to run ksp if mockingbird isn't enabled. The performance is basically the same. It would be better to do profiling on a larger project like you mentioned though.

mockingbird-benchmark.csv
mockk-benchmark.csv