Skip to content

InSet fix and timestamp pushdown
Compare
Choose a tag to compare
@pwoody pwoody released this 02 Dec 14:21
· 11046 commits to master since this release
2.1.0-palantir20
a1a17b5

Breaking Changes

  • The Timestamp type is now written into Parquet as an INT64 column instead of the previous INT96 type. This version of Spark still supports reading data from Parquet files written in the previous INT96-serialized format, but older versions of Spark will not be able to read Parquet files with the Timestamp type written by this Spark version.

Improvements

  • Timestamp and date filter pushdown (#72)
    • Enabled due to the INT96 -> INT64 serialization change above.

Bugfixes

  • Fix inset for large queries. Disable record level filtering (#74)
Assets 2
Loading