3.0.0-palantir.78

• Fix published artifact name for _2.12 Spark distribution (#704)

3.0.0-palantir.77

• Spark is now built with Scala 2.12 (#691)
• Infer 'mixed' types as strings when using Arrow serialization (#702)

3.0.0-palantir.75.1

This is a backport of

• Infer 'mixed' types as strings when using Arrow serialization (#702)

3.0.0-palantir.76.1

Same as 3.0.0-palantir.76.

3.0.0-palantir.76

• [SPARK-29376][SQL][PYTHON] Upgrade Apache Arrow to version 0.15.1
• [SPARK-29378][R] Upgrade SparkR to use Arrow 0.15 API
• [SPARK-31701][R][SQL] Bump up the minimum Arrow version as 0.15.1 in SparkR

Spark now uses Arrow version 0.15. 0.15 comes with a compatibility break (Arrow docs). Because of that, pyspark and SparkR need resp. pyarrow and r-arrow version 0.15 or higher if they want to continue using Arrow serialization.

3.0.0-palantir.75

• Add pre-installed conda configuration and use to find rlib directory (#700)
• python runner takes an optional outputstream for redirecting stderr/stdout (#701)

3.0.0-palantir.74

• Add Arrow optimizations for gappy and dapply in SparkR (#689)
• Support Arrow 0.14 in SparkR (#689)
• [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large (#686)
• [SPARK-29683][YARN]False report isAllNodeBlacklisted when RM is having issue (#688)
• Upgrade ASM to 7.2 (#697)
• [SPARK-30949][K8S][CORE] Decouple requests and parallelism on drivers (#698)

3.0.0-palantir.73

• [SPARK-27992][PYTHON] Allow Python to join with connection thread to propagate errors (#684)
• [SPARK-23961][SPARK-27548][PYTHON] Fix error when toLocalIterator goes out of scope and properly raise errors from worker (#684)
• [SPARK-26922][R] Set socket timeout consistently in Arrow optimization (#684)
• [SPARK-28003][PYTHON] Allow NaT values when creating Spark dataframe from pandas with Arrow (#684)
• [SPARK-27629][PYSPARK] Prevent Unpickler from intervening each unpickling (#684)

3.0.0-palantir.72

• [SPARK-29683][YARN] False report isAllNodeBlacklisted when RM is having issue (#688)
• [SPARK-24666][ML] Fix infinity vectors produced by Word2Vec when numIterations are large (#687)
• Build SparkR with R 3.4 (#685)

3.0.0-palantir.71

• [SPARK-27276][PYTHON][SQL] Increase minimum version of pyarrow to 0.12.1 (#649)
• [SPARK-30362][CORE] Update InputMetrics in DataSourceRDD (#682)
• Use runtimeClasspath instead of runtimeElements in Spark docker image plugin (#682)