Library service doesn't work on Yarn cluster

[mobsy74]

Datasources we create in the library are available to our Spark jobs when running in the local cluster preset. However, when we switch to a Yarn cluster preset, those datasources are no longer available to our executors on the cluster. Is the library service intended to work in cluster mode?

[jaroslaw-osmanski]

Seahorse works only in Spark client mode (https://seahorse.deepsense.ai/reference/external_cluster.html).
Library should work using YARN cluster and this will be investigated.

[jaroslaw-osmanski]

It works on my YARN cluster. Maybe AWS security groups are the problem? I've managed to create and read dataframe using library datasource. Maybe I'm missing something in workflow? @mobsy74 @metasim

[jaroslaw-osmanski]

Closing this as it was setup issue.