Skip to content

Releases: palantir/spark

3.0.0-palantir.60

15 Jan 20:31
Compare
Choose a tag to compare
  • [SPARK-29874][SQL]Optimize Dataset.isEmpty() (#632)
  • [SPARK-30082][SQL] Depend on Scala type coercion when building replace query #628

3.0.0-palantir.59

10 Jan 11:35
d50bdf7
Compare
Choose a tag to compare

Cherry picks (#631) of the following commits from upstream:

[SPARK-27291][SQL] PartitioningAwareFileIndex: Filter out empty files on listing files
[SPARK-27627][SQL] Make option "pathGlobFilter" as a general option for all file sources
[SPARK-27990][SQL][ML] Provide a way to recursively load data from datasource
[SPARK-25348][SQL] Data source for binary files
[SPARK-27473][SQL] Support filter push down for status fields in binary file data source
[SPARK-27534][SQL] Do not load content column in binary data source if it is not selected
[SPARK-27588] Binary file data source fails fast and doesn't attempt to read very large files
[SPARK-28030][SQL] convert filePath to URI in binary file data source

3.0.0-palantir.58

07 Jan 02:50
Compare
Choose a tag to compare
  • Upgrades Maven to 3.6.3 (#629)

3.0.0-palantir.57

06 Jan 21:55
Compare
Choose a tag to compare
  • Support creating conda environment with spec file (#624)

3.0.0-palantir.56

04 Dec 15:49
Compare
Choose a tag to compare
  • Upgrade jackson to 2.10.0 (#627)

3.0.0-palantir.55

03 Dec 21:54
Compare
Choose a tag to compare
  • [SPARK-27805][PYTHON] Propagate SparkExceptions during toPandas with arrow enabled (#626)
  • [SPARK-27778][PYTHON] Fix toPandas conversion of empty DataFrame with Arrow enabled (#625)

3.0.0-palantir.54

07 Nov 19:28
Compare
Choose a tag to compare

Improvements

  • [SPARK-28938][K8S] Move to supported OpenJDK docker image for Kubernetes

3.0.0-palantir.53

06 Nov 01:08
af0ad25
Compare
Choose a tag to compare
  • Add more debug logging to the DAGScheduler (#617)
  • Check that the map output is not stored outside the executor before unregistering the map output we failed to fetch (#618)

3.0.0-palantir.52

24 Oct 19:37
6ca2bbb
Compare
Choose a tag to compare
  • Track the attempt id when asking the driver components API if map outputs are available outside the executor (#615)

3.0.0-palantir.51

22 Oct 08:29
Compare
Choose a tag to compare

Identical in end functionality to 3.0.0-palantir.49 and 3.0.0-palantir.50, but modified the build script to pull TypeSafe sbt plugins from bintray instead of maven. Required to make the build pass.