Valid use of SDK causes workflow manager startup exception and failure

[metasim]

The test at the following repo and branch causes a ClassNotFoundException.

https://github.com/metasim/seahorse-test/tree/classloader-bug

To reproduce, run sbt assembly and copy the resulting jar to the $SEAHORSE/jars directory.

My suspicion is that this is due to custom operation code being loaded in a separate class loader (via the CatalogScanner) that what was used to load the SDK classes, but then a SDK class attempts to instantiate a class in that class loader from the parent ClassLoader. But I'm not 100% sure of this. It's the only reason I can think of why the class can't be found even though it is most certainly in the jar file with the Register'd operation.

The relevant part of the stack trace is this:

Caused by: java.lang.ClassNotFoundException: seahorseTest.deeplang.TestDLTransformer
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:348)
	at scala.reflect.runtime.JavaMirrors$JavaMirror.javaClass(JavaMirrors.scala:555)
	at scala.reflect.runtime.JavaMirrors$JavaMirror$$anonfun$classToJava$1.apply(JavaMirrors.scala:1211)
	at scala.reflect.runtime.JavaMirrors$JavaMirror$$anonfun$classToJava$1.apply(JavaMirrors.scala:1203)
	at scala.reflect.runtime.TwoWayCaches$TwoWayCache$$anonfun$toJava$1.apply(TwoWayCaches.scala:49)
	at scala.reflect.runtime.Gil$class.gilSynchronized(Gil.scala:19)
	at scala.reflect.runtime.JavaUniverse.gilSynchronized(JavaUniverse.scala:16)
	at scala.reflect.runtime.TwoWayCaches$TwoWayCache.toJava(TwoWayCaches.scala:44)
	at scala.reflect.runtime.JavaMirrors$JavaMirror.classToJava(JavaMirrors.scala:1203)
	at scala.reflect.runtime.JavaMirrors$JavaMirror.runtimeClass(JavaMirrors.scala:194)
	at scala.reflect.runtime.JavaMirrors$JavaMirror.runtimeClass(JavaMirrors.scala:54)
	at ai.deepsense.deeplang.TypeUtils$.typeToClass(TypeUtils.scala:35)
	at ai.deepsense.deeplang.TypeUtils$.constructorForType(TypeUtils.scala:52)
	at ai.deepsense.deeplang.TypeUtils$.instanceOfType(TypeUtils.scala:60)
	at ai.deepsense.deeplang.doperations.TransformerAsOperation.<init>(TransformerAsOperation.scala:31)
	at seahorseTest.deeplang.TestTransformerAsOp.<init>(TestTransformerAsOp.scala:10)
	... 85 more

As you can see, the class TestTransformerAsOp is loaded, the constructor gets called, and then the super type TransfomerAsOperation constructor is called. It in turn attempts to create an instance of its type parameter, as captured via a TypeTag. Through TypeUtils an instance fetched, but fails to be found by sun.misc.Launcher$AppClassLoader.loadClass, which is the root ClassLoader.

IMHO, this is a critical bug because there's no possible workaround.

[metasim]

FWIW, I've worked on projects in the past where the Reflections library was used, and it constantly caused us problems. I'd suggest replacing the use of Reflections with the standard Java SPI API, which is quite easy to use, has very low overhead, and doesn't suffer from ClassLoader issues. A simple CatalogProvider interface/trait would be all that was needed. It would address #11, and make implementing custom categories (as discussed in the community forum) easier.

I'm willing to write a PR if you're open to this approach.

Edit: custom categories is logged under #15.

[metasim]

To further malign Reflections , I point out that there are 11 outstanding pull requests for the project, a couple of which fix ClassLoader bugs, one related to #11.

[metasim]

Gripes about Reflections aside, this is part of the bug:

seahorse/seahorse-workflow-executor/deeplang/src/main/scala/ai/deepsense/deeplang/TypeUtils.scala

Line 31 in c1b2e66

private val mirror = ru.runtimeMirror(getClass.getClassLoader)

By using the ClassLoader of TypeUtils, it assumes it will only be asked to instantiate classes from the same ClassLoader as the SDK classes (clearly not the case with TransfomerAsOperation).