Skip to content

Releases: palantir/spark

Master rebase, fix timestamp inference.

02 Dec 19:28
Compare
Choose a tag to compare

Bugfixes

  • Fix broken timestamp rebase to do automatic schema inference on timestamp types correctly (#76)

Improvements

  • Merge latest from upstream master (#73)

InSet fix and timestamp pushdown

02 Dec 14:21
Compare
Choose a tag to compare

Breaking Changes

  • The Timestamp type is now written into Parquet as an INT64 column instead of the previous INT96 type. This version of Spark still supports reading data from Parquet files written in the previous INT96-serialized format, but older versions of Spark will not be able to read Parquet files with the Timestamp type written by this Spark version.

Improvements

  • Timestamp and date filter pushdown (#72)
    • Enabled due to the INT96 -> INT64 serialization change above.

Bugfixes

  • Fix inset for large queries. Disable record level filtering (#74)

Fix broken parquet splitting

28 Nov 21:04
Compare
Choose a tag to compare

Improvements

  • Resync with upstream master (#64)

Bug fixes

  • Fix createFilter for NOT queries (#68)
  • Fix for ineligible filters, use compressed block size (#70)

2.1.0-palantir18

18 Nov 13:57
Compare
Choose a tag to compare

DO NOT USE: this release contained a correctness regression that is fixed in 2.1.0-palantir19

  • SPARK-17059: Allow FileFormat to specify partition pruning strategy (#62)
  • open reader only once with correct schema (#59)
  • make dist land in maven style directory pattern (#57)

2.1.0-palantir17

14 Nov 20:44
Compare
Choose a tag to compare
  • Republish same code as 2.1.0-palantir16

2.1.0-palantir16

14 Nov 20:38
Compare
Choose a tag to compare
  • Rework the Parquet partition pruning feature (#52)

2.1.0-palantir15

01 Nov 18:03
Compare
Choose a tag to compare
  • hotfix an issue with the merge from apache master done in 2.1.0-palantir14

2.1.0-palantir14

01 Nov 17:51
Compare
Choose a tag to compare
  • toLocalIterator on CollectLimitExec doesn't shuffle (#46)
  • toJson doesn't perform toRdd conversion (#48)
  • parquet bump to 1.9.0-palantir3 (#49)
  • resync with apache master

2.1.0-palantir13

13 Oct 19:00
Compare
Choose a tag to compare

2.1.0-palantir12

10 Oct 15:24
Compare
Choose a tag to compare
Merge branch 'master' into palantir-master