Skip to content

Commit

Permalink
Retire Java8/Scala 2.11 support in restructure
Browse files Browse the repository at this point in the history
  • Loading branch information
juripetersen committed Jan 10, 2024
1 parent e353317 commit 4c9ada7
Show file tree
Hide file tree
Showing 523 changed files with 288 additions and 2,450 deletions.
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,4 @@ hybrid/*
pom.xml.*

10e*.input
*_pb2.py
*_pb2.py
13 changes: 6 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,13 +65,13 @@ cd wayang-0.7.1-SNAPSHOT
```

In linux
```shell
```shell
echo "export WAYANG_HOME=$(pwd)" >> ~/.bashrc
echo "export PATH=${PATH}:${WAYANG_HOME}/bin" >> ~/.bashrc
source ~/.bashrc
```
In MacOS
```shell
```shell
echo "export WAYANG_HOME=$(pwd)" >> ~/.zshrc
echo "export PATH=${PATH}:${WAYANG_HOME}/bin" >> ~/.zshrc
source ~/.zshrc
Expand Down Expand Up @@ -106,12 +106,11 @@ Wayang is available via Maven Central. To use it with Maven, include the followi
Note the `***`: Wayang ships with multiple modules that can be included in your app, depending on how you want to use it:
* `wayang-core`: provides core data structures and the optimizer (required)
* `wayang-basic`: provides common operators and data types for your apps (recommended)
* `wayang-api-scala-java_2.12`: provides an easy-to-use Scala and Java API to assemble Wayang plans (recommended)
* `wayang-api-scala-java`: provides an easy-to-use Scala and Java API to assemble Wayang plans (recommended)
* `wayang-java`, `wayang-spark`, `wayang-graphchi`, `wayang-sqlite3`, `wayang-postgres`: adapters for the various supported processing platforms
* `wayang-profiler`: provides functionality to learn operator and UDF cost functions from historical execution data

> **NOTE:** The module `wayang-api-scala-java_2.12` is intended to be used with Java 11 and Scala 2.12. If you have the Java 8 version, you need to use the `wayang-api-scala-java_2.11` module.
> **NOTE:** The module `wayang-api-scala-java` is intended to be used with Java 11 and Scala 2.12.
For the sake of version flexibility, you still have to include in the POM file your Hadoop (`hadoop-hdfs` and `hadoop-common`) and Spark (`spark-core` and `spark-graphx`) version of choice.

Expand Down Expand Up @@ -163,7 +162,7 @@ If you need to rebuild Wayang, e.g., to use a different Scala version, you can s
To activate these profiles, you need to specify them when running maven, i.e.,

```shell
./mvnw clean install -DskipTests -P<profile name>
./mvnw clean install -DskipTests -P<profile name>
```

## Running the tests
Expand All @@ -177,7 +176,7 @@ You can see examples on how to start using Wayang [here](guides/wayang-examples.

## Built With

* [Java 11](https://www.oracle.com/de/java/technologies/javase/jdk11-archive-downloads.html)
* [Java 11](https://www.oracle.com/de/java/technologies/javase/jdk11-archive-downloads.html)
* [Scala 2.12](https://www.scala-lang.org/download/2.12.0.html)
* [Maven](https://maven.apache.org/)

Expand Down
14 changes: 7 additions & 7 deletions build.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,23 +21,23 @@
Apache Wayang (incubating) has different dependencies, for compiling, it needs to add some profile in the compilation to enable maven works properly.

```shell
mvn clean compile
mvn clean compile
```

The line before is because the plugin the Antlr is not needed in all the modules, as well it has happened with Scala language.

When maven compiles one or more modules using those plugins in the compilation time, it needs to add.

The modules are:
- wayang-api-scala-java_#Scala_Version#
- wayang-api-scala-java
- wayang-core <- Antlr
- wayang-iejoin_#Scala_Version#
- wayang-spark_#Scala_Version#
- wayang-profiler_#Scala_Version#
- wayang-tests-integration_#Scala_Version#
- wayang-iejoin
- wayang-spark
- wayang-profiler
- wayang-tests-integration


# Executing Coverage
# Executing Coverage

```shell
mvn clean verify jacoco:report
Expand Down
8 changes: 4 additions & 4 deletions guides/develop-with-Wayang.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,14 +37,14 @@ This tutorial shows users how to import Wayang in their Java project using the m
</dependency>
<dependency>
<groupId>org.apache.wayang</groupId>
<artifactId>wayang-spark_2.12</artifactId>
<artifactId>wayang-spark</artifactId>
<version>0.7.1</version>
</dependency>
<dependency>
<groupId>org.apache.wayang</groupId>
<artifactId>wayang-api-scala-java_2.12</artifactId>
<artifactId>wayang-api-scala-java</artifactId>
<version>0.7.1</version>
</dependency>
</dependency>
```

# Include the Spark maven dependencies in your pom
Expand Down Expand Up @@ -82,7 +82,7 @@ public static void main(String[] args) {
WayangContext wayangContext = new WayangContext(new Configuration())
.withPlugin(Java.basicPlugin())
.withPlugin(Spark.basicPlugin());

/* Get a plan builder */
JavaPlanBuilder planBuilder = new JavaPlanBuilder(wayangContext)
.withJobName("WordCount")
Expand Down
4 changes: 2 additions & 2 deletions guides/pom-example.xml
Original file line number Diff line number Diff line change
Expand Up @@ -66,12 +66,12 @@
</dependency>
<dependency>
<groupId>org.apache.wayang</groupId>
<artifactId>wayang-spark_2.12</artifactId>
<artifactId>wayang-spark</artifactId>
<version>0.6.0</version>
</dependency>
<dependency>
<groupId>org.apache.wayang</groupId>
<artifactId>wayang-api-scala-java_2.12</artifactId>
<artifactId>wayang-api-scala-java</artifactId>
<version>0.6.0</version>
</dependency>

Expand Down
6 changes: 3 additions & 3 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -711,7 +711,7 @@
<id>scala-parent-default</id>
<activation>
<file>
<exists>code</exists>
<exists>src</exists>
</file>
</activation>
<properties>
Expand Down Expand Up @@ -1231,7 +1231,7 @@
<exclude>**/.settings/**</exclude>
<exclude>**/.classpath</exclude>
<exclude>**/.factorypath</exclude>

<!--RAT files-->
<exclude>**/apache-rat-0.13/**</exclude>

Expand All @@ -1253,7 +1253,7 @@
<exclude>**/scala_1*</exclude>

<exclude>**/*pb2.py</exclude>
<exclude>**/.rat-excludes</exclude>
<exclude>**/.rat-excludes</exclude>
<exclude>**/*.csv</exclude>
<exclude>**/*.properties</exclude>
</excludes>
Expand Down
2 changes: 1 addition & 1 deletion wayang-api/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<packaging>pom</packaging>

<parent>
<artifactId>wayang</artifactId>
Expand All @@ -28,7 +29,6 @@

<artifactId>wayang-api</artifactId>
<version>0.7.1-SNAPSHOT</version>
<packaging>pom</packaging>

<name>Wayang API</name>
<description>
Expand Down
43 changes: 25 additions & 18 deletions wayang-api/wayang-api-scala-java/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,7 @@
under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<packaging>pom</packaging>
<modelVersion>4.0.0</modelVersion>

<parent>
<artifactId>wayang-api</artifactId>
Expand Down Expand Up @@ -92,6 +91,30 @@
<artifactId>hadoop-hdfs</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.wayang</groupId>
<artifactId>wayang-spark</artifactId>
<version>0.7.1-SNAPSHOT</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.12</artifactId>
<version>${spark.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-graphx_2.12</artifactId>
<version>${spark.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>${scala.version}</version>
<scope>provided</scope>
</dependency>
</dependencies>

<build>
Expand All @@ -106,20 +129,4 @@
</plugins>
</build>


<modules>
<module>wayang-api-scala-java_2.12</module>
</modules>

<profiles>
<profile>
<id>java8</id>
<activation>
<jdk>1.8</jdk>
</activation>
<modules>
<module>wayang-api-scala-java_2.11</module>
</modules>
</profile>
</profiles>
</project>
16 changes: 0 additions & 16 deletions wayang-api/wayang-api-scala-java/src/main/scala/README.md

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

Loading

0 comments on commit 4c9ada7

Please sign in to comment.