Releases: palantir/spark
3.0.0-palantir.60
3.0.0-palantir.59
Cherry picks (#631) of the following commits from upstream:
[SPARK-27291][SQL] PartitioningAwareFileIndex: Filter out empty files on listing files
[SPARK-27627][SQL] Make option "pathGlobFilter" as a general option for all file sources
[SPARK-27990][SQL][ML] Provide a way to recursively load data from datasource
[SPARK-25348][SQL] Data source for binary files
[SPARK-27473][SQL] Support filter push down for status fields in binary file data source
[SPARK-27534][SQL] Do not load content
column in binary data source if it is not selected
[SPARK-27588] Binary file data source fails fast and doesn't attempt to read very large files
[SPARK-28030][SQL] convert filePath to URI in binary file data source
3.0.0-palantir.58
- Upgrades Maven to 3.6.3 (#629)
3.0.0-palantir.57
- Support creating conda environment with spec file (#624)
3.0.0-palantir.56
- Upgrade jackson to 2.10.0 (#627)
3.0.0-palantir.55
3.0.0-palantir.54
Improvements
- [SPARK-28938][K8S] Move to supported OpenJDK docker image for Kubernetes
3.0.0-palantir.53
3.0.0-palantir.52
- Track the attempt id when asking the driver components API if map outputs are available outside the executor (#615)
3.0.0-palantir.51
Identical in end functionality to 3.0.0-palantir.49 and 3.0.0-palantir.50, but modified the build script to pull TypeSafe sbt plugins from bintray instead of maven. Required to make the build pass.