Building Apache Zeppelin

Building Apache Zeppelin

The instructions provided below specify the steps to build Apache Zeppelin version 0.8.0 on Linux on IBM Z for the following distributions:

• RHEL (7.3, 7.4, 7.5)
• SLES (12 SP3, 15)
• Ubuntu (16.04, 18.04)

General Notes:

• When following the steps below please use a standard permission user unless otherwise specified.
• A directory /<source_root>/ will be referred to in these instructions, this is a temporary writeable directory anywhere you'd like to place it.

Step 1 : Install Dependencies

1.1) Install packages

• RHEL (7.3, 7.4, 7.5)

  sudo yum install git tar wget java-1.8.0-openjdk-headless java-1.8.0-openjdk-devel
  

• SLES (12 SP3, 15)

  sudo zypper install git tar wget java-1_8_0-openjdk java-1_8_0-openjdk-devel
  

• Ubuntu (16.04, 18.04)

  sudo apt-get update
  sudo apt-get install build-essential git tar wget maven openjdk-8-jdk-headless openjdk-8-jdk
  

1.2 ) Download Maven binary executable (Only for RHEL/SLES)

cd /<source_root>/ 
wget http://www.eu.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz
sudo tar -zxf apache-maven-3.3.9-bin.tar.gz -C /usr/local/
sudo ln -s /usr/local/apache-maven-3.3.9/bin/mvn /usr/local/bin/mvn

1.3) Set environment variable

export MAVEN_OPTS="-Xmx2g -XX:MaxPermSize=1024m"
export JAVA_HOME=/usr/lib/jvm/java                   # RHEL (7.3, 7.4, 7.5)
export JAVA_HOME=/usr/lib64/jvm/java                 # SLES (12 SP3, 15)
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-s390x   # Ubuntu (16.04, 18.04)
export PATH=$JAVA_HOME/bin:$PATH

1.4) Install PhantomJS 2.1.1

• Follow the PhantomJS recipe. For Ubuntu, install using sudo apt-get install phantomjs. Make sure PhantomJS is available in PATH environment variable.

1.5) Install Protobuf 3.3.0

• Follow the ProtoBuf build instructions.

1.6) Install gRPC-Java 1.4.0

cd /<source_root>/
git clone https://github.com/grpc/grpc-java.git
cd grpc-java/
git checkout v1.4.0

• Apply patch to /<source_root>/grpc-java/compiler/build.gradle

  @@ -49,6 +49,9 @@
       gcc(Gcc) {
         target("ppcle_64")
       }
  +    gcc(Gcc) {
  +      target("s390_64")
  +    }
       clang(Clang) {
       }
     }
  @@ -63,11 +66,14 @@
       ppcle_64 {
         architecture "ppcle_64"
       }
  +    s390_64 {
  +      architecture "s390_64"
  +    }
     }
   
     components {
       java_plugin(NativeExecutableSpec) {
  -      if (arch in ['x86_32', 'x86_64', 'ppcle_64']) {
  +      if (arch in ['x86_32', 'x86_64', 'ppcle_64', 's390_64']) {
           // If arch is not within the defined platforms, we do not specify the
           // targetPlatform so that Gradle will choose what is appropriate.
           targetPlatform arch  

• Build gRPC-java

cd /<source_root>/grpc-java  --incompatible_load_argument_is_label=false 
./gradlew install -Pprotoc=/usr/local/bin/protoc  

Note: While building gRPC-Java, if the build fails with an error grpc-compiler:linkJava_pluginExecutable/bin/ld: cannot find -lstdc++, install libstdc++

Step 2 : Build and Install Apache Zeppelin

2.1) Download source

git clone https://github.com/apache/zeppelin.git
cd zeppelin/
git checkout v0.8.0

2.2) Edit /<source_root>/zeppelin/pom.xml

  @@ -95,7 +95,7 @@
       <!-- frontend maven plugin related versions-->
       <node.version>v8.9.3</node.version>
       <npm.version>5.5.1</npm.version>
  -    <plugin.frontend.version>1.4</plugin.frontend.version>
  +    <plugin.frontend.version>1.5</plugin.frontend.version>
   
       <!-- common library versions -->
       <slf4j.version>1.7.10</slf4j.version>

2.2) Build Apache Zeppelin

cd <source_root>/zeppelin/
mvn install:install-file -DgroupId=com.google.protobuf -DartifactId=protoc -Dversion=3.3.0 -Dclassifier=linux-s390_64 -Dpackaging=exe -Dfile=/<source_root>/protobuf/src/.libs/protoc
mvn install:install-file -DgroupId=io.grpc -DartifactId=protoc-gen-grpc-java -Dversion=1.4.0 -Dclassifier=linux-s390_64 -Dpackaging=exe -Dfile=/<source_root>/grpc-java/compiler/build/exe/java_plugin/protoc-gen-grpc-java
mvn clean package -DskipTests

Note: If an error of QXchConnection: Could not connect to display occurs, run export QT_QPA_PLATFORM=offscreen

Step 3 : Testing (Optional)

cd <source_root>/zeppelin/
mvn clean package -fn

Note:

• Individual test case can be executed by running command mvn clean test in the respective modules

• Test case failures in modules Zengine, Python interpeter , Spark Interpreter and Server can be ignored as they are seen on x86 system as well

• Test case failures in modules Cassandra passed on increasing server wait time in Achilles source code

Step 4 : Start Apache Zeppelin

cd /<source_root>/zeppelin/
bin/zeppelin-daemon.sh start 

Open http://<ip_address>:8080/ in your browser to access Web UI.

References

• https://zeppelin.apache.org/