Skip to content

Commit c7daa95

Browse files
committed
[SPARK-22128][CORE][BUILD] Add paranamer dependency to core module
## What changes were proposed in this pull request? With Scala-2.12 profile, Spark application fails while Spark is okay. For example, our documented `SimpleApp` Java example succeeds to compile but it fails at runtime because it doesn't use `paranamer 2.8` and hits [SPARK-22128](https://issues.apache.org/jira/browse/SPARK-22128). This PR aims to declare it explicitly for the Spark applications. Note that this doesn't introduce new dependency to Spark itself. https://dist.apache.org/repos/dist/dev/spark/3.0.0-SNAPSHOT-2019_01_09_13_59-e853afb-docs/_site/quick-start.html The following is the dependency tree from the Spark application. **BEFORE** ``` $ mvn dependency:tree -Dincludes=com.thoughtworks.paranamer [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) simple --- [INFO] my.test:simple:jar:1.0-SNAPSHOT [INFO] \- org.apache.spark:spark-sql_2.12:jar:3.0.0-SNAPSHOT:compile [INFO] \- org.apache.spark:spark-core_2.12:jar:3.0.0-SNAPSHOT:compile [INFO] \- org.apache.avro:avro:jar:1.8.2:compile [INFO] \- com.thoughtworks.paranamer:paranamer:jar:2.7:compile ``` **AFTER** ``` [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) simple --- [INFO] my.test:simple:jar:1.0-SNAPSHOT [INFO] \- org.apache.spark:spark-sql_2.12:jar:3.0.0-SNAPSHOT:compile [INFO] \- org.apache.spark:spark-core_2.12:jar:3.0.0-SNAPSHOT:compile [INFO] \- com.thoughtworks.paranamer:paranamer:jar:2.8:compile ``` ## How was this patch tested? Pass the Jenkins. And manually test with the sample app is running. Closes apache#23502 from dongjoon-hyun/SPARK-26583. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent b316ebf commit c7daa95

File tree

2 files changed

+5
-0
lines changed

2 files changed

+5
-0
lines changed

core/pom.xml

+4
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,10 @@
3333
<name>Spark Project Core</name>
3434
<url>http://spark.apache.org/</url>
3535
<dependencies>
36+
<dependency>
37+
<groupId>com.thoughtworks.paranamer</groupId>
38+
<artifactId>paranamer</artifactId>
39+
</dependency>
3640
<dependency>
3741
<groupId>org.apache.avro</groupId>
3842
<artifactId>avro</artifactId>

pom.xml

+1
Original file line numberDiff line numberDiff line change
@@ -1949,6 +1949,7 @@
19491949
<groupId>com.thoughtworks.paranamer</groupId>
19501950
<artifactId>paranamer</artifactId>
19511951
<version>${paranamer.version}</version>
1952+
<scope>runtime</scope>
19521953
</dependency>
19531954
<dependency>
19541955
<groupId>org.apache.arrow</groupId>

0 commit comments

Comments
 (0)