Releases: ByteMeDirk/pyspark3-docker
Releases · ByteMeDirk/pyspark3-docker
Official Release: PySpark3 with Python3
A single node PySpark3 docker container based on OpenJDK. Using Python 3, PySpark 3.0.3 with Spark 3.1.2 and Hadoop 2.7.
Functionality for AWS included as was-cli and boto3, with fully integrated Python3 and OpenJDK with Hadoop for extended development.
Full Changelog: v0.0.3...v1.0.0
Updated dependancies
Updated dependencies & refined docker layers.
Updated dependancies
Removes pandas and updated dependancies causing build issues.
Inital Release
Simple initial release using Spark version 3.1.2 with Hadoop version 2.7 to be used with PySpark version 3.0.3.