Ability to discover test classes multiple times and load them using different classloaders

[rieske]

I want to run a set of tests where each test is executed multiple times and each time the test class would be loaded using a different classloader. That is, given test classes A, B and C, I want to create two executions for each class where I would load them with a different, custom classloader for one of those executions.

One way to achieve this is to implement a test engine that would delegate discovery and execution to an existing test engine and to intercept the discovery phase (where the test classes are loaded).
Given I have an EngineDiscoveryRequest with some class selectors, in the test engine's discover method, I can call the target engine's discover(request, id) multiple times, giving a unique id for each execution that I want to create. I can then aggregate the returned test descriptors and the tests will be executed multiple times.

For the classloading part, the test classes are loaded in ClassSelector lazily whenever getJavaClass() is called, which happens when the engine is discovering tests. If the class selectors that arrive to my custom test engine have only class names set, the classes will not have been loaded yet and there I can influence the classloader to use by setting it using Thread.currentThread().setContextClassLoader(...) before delegating the discovery. This works just for one execution though as the loaded classes will be cached in the current discovery request. From here, in order to create the second execution and have the classes loaded using a different classloader, I have to create a new EngineDiscoveryRequest based on existing one and create new ClassSelectors from existing ones using DiscoverySelectors.selectClass(selector.getClassName()). There doesn't seem to be an API to create a new discovery request based on existing one for such use case. One alternative that works is to decorate an existing EngineDiscoveryRequest by implementing EngineDiscoveryRequest and delegating everything but getSelectorsByType(...) to the existing request.

While I seem to be able to achieve what I'm after in the end, I am wondering if this can be simplified.
For example, if the ClassSelector would not cache the loaded test class itself, I would be able to reuse the existing discovery request by setting the classloader on the current thread for each discovery iteration.
Or perhaps an API could be exposed that would allow setting the classloader for a discovery request and then to create a new discovery request based on an existing one - something along the lines of DiscoveryRequests.from(existingRequest).withClassLoader(...).build().

[sbrannen]

Related to #201 and #1987.

[kriegfrj]

In Bndtools' BundleEngine, we have a workaround similar to what you have proposed. We fully resolve the classes before passing them into selectClass() so that we can handle the OSGi-related classloading issues ourselves.

One alternative that works is to decorate an existing EngineDiscoveryRequest by implementing EngineDiscoveryRequest and delegating everything but getSelectorsByType(...) to the existing request.

This is exactly what we do here:

https://github.com/bndtools/bnd/blob/cb1d669b10e38f96b3a7b374fa2cfdce9fa62ee8/biz.aQute.tester.junit-platform/src/aQute/tester/bundle/engine/discovery/BundleSelectorResolver.java#L537-L568

In fact, at a high-level, there's a high degree of overlap between what you're asking for and what BundleEngine does, because there is a very close correlation between the concept of "Bundle" in OSGi and the more generic concept of a "ClassLoader" (each Bundle has its own unique classloader). I suspect you can find a lot of what you need there, and we might even be able to modify it slightly to morph it from BundleEngine into ClassLoaderEngine.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. Given the limited bandwidth of the team, it will be automatically closed if no further activity occurs. Thank you for your contribution.

[stale]

This issue has been automatically closed due to inactivity. If you have a good use case for this feature, please feel free to reopen the issue.