Skip to content

Releases: ByteMeDirk/pyspark3-docker

Official Release: PySpark3 with Python3

02 Dec 08:16
Compare
Choose a tag to compare

A single node PySpark3 docker container based on OpenJDK. Using Python 3, PySpark 3.0.3 with Spark 3.1.2 and Hadoop 2.7.

Functionality for AWS included as was-cli and boto3, with fully integrated Python3 and OpenJDK with Hadoop for extended development.

Full Changelog: v0.0.3...v1.0.0

Updated dependancies

01 Dec 14:15
Compare
Choose a tag to compare

Updated dependencies & refined docker layers.

Updated dependancies

01 Dec 12:00
Compare
Choose a tag to compare

Removes pandas and updated dependancies causing build issues.

Inital Release

01 Dec 10:57
Compare
Choose a tag to compare

Simple initial release using Spark version 3.1.2 with Hadoop version 2.7 to be used with PySpark version 3.0.3.