Files in spark_applications_logs are empty

[metasim]

Seahorse appears to mount spark_applications_logs correctly as many files are created (even .lck files) but all of them have zero bytes. How do we enable logging?

[metasim]

FWIW: The impetus for needing these is that we can't connect to an external, stand-alone cluster, and need a means of debugging the problem:

[mteldridge]

It looks like the SparkLauncher is not redirecting to the correct session manager logger. Here is a PR for the fix: #49

FYI, the log configuration (https://github.com/deepsense-ai/seahorse/blob/master/seahorse-workflow-executor/commons/src/main/resources/log4j.xml) is creating two appenders. One is the console appender which is what gets redirected in session manager. The other is a rolling file appender that will write to the CWD of the session manager in its Docker container. This might be problematic in that it is an AuFS volume. My recommendation would be to turn off the file appender.