Releases: IBM/spark-s3-shuffle
Releases · IBM/spark-s3-shuffle
Integrate checksum support and add a configuration option for the number of prefixes.
What's Changed
- Create an NFS example. by @pspoerri in #34
- Integrate support for checksums. by @pspoerri in #35
- S3ShuffleManager: Only try to clean up shuffle files if there are no registered shuffle ids. by @pspoerri in #38
- Implement folder prefixes, sort configuration variables and improve README. by @pspoerri in #41
Full Changelog: v0.7.2...v0.8
Checksum and prefix support
Spark3.1: Fix timeouts issues with Hadoop-AWS.
What's Changed
Full Changelog: v0.7.1-spark3.1...v0.7.2-spark3.1
Make examples more consistent.
Configuration option for prefetchBatchSize and prefetchThreadPool…
Enable configuration option for prefetchBatchSize and prefetchThreadPool…
Performance improvements
v0.7-spark3.1
S3: List/Delete using futures. Signed-off-by: Pascal Spörri <psp@zurich.ibm.com>
Integrate with Github CI.
What's Changed
Full Changelog: v0.6.3...v0.6.4.1