Add source code

.gitignore

@@ -0,0 +1,24 @@
+**/.keep
+**/generated/
+*.avro
+*.iml
+*.ipr
+*.iws
+*.orig
+*.rej
+*.sdf
+*.suo
+*.swp
+*.vcxproj.user
+*.zip
+.DS_Store
+.classpath
+.gradle/
+.idea
+.project
+.settings
+.svn
+build/
+dependency-reduced-pom.xml
+out/
+target

.reviewboardrc

@@ -0,0 +1,2 @@
+REPOSITORY='tony'
+TRACKING_BRANCH='origin/master'

.travis.yml

@@ -0,0 +1,9 @@
+language: java
+git:
+  depth: 3
+jdk:
+  - oraclejdk8
+env:
+  - HADOOP_VERSION=3.1.1
+script: "./gradlew test --stacktrace --info"
+

CODE_OF_CONDUCT.md

@@ -0,0 +1,42 @@
+LinkedIn Open Source Code of Conduct
+-------------------------------------
+
+This code of conduct outlines expectations for participation in LinkedIn-managed open source communities, as well as steps for reporting unacceptable behavior. We are committed to providing a welcoming and inspiring community for all. People violating this code of conduct may be banned from the community.
+
+Our open source communities strive to:
+
+* **Be friendly and patient:** Remember you might not be communicating in someone else's primary spoken or programming language, and others may not have your level of understanding.
+* **Be welcoming:** Our communities welcome and support people of all backgrounds and identities. This includes, but is not limited to members of any race, ethnicity, culture, national origin, color, immigration status, social and economic class, educational level, sex, sexual orientation, gender identity and expression, age, size, family status, political belief, religion, and mental and physical ability.
+* **Be respectful:** We are a world-wide community of professionals, and we conduct ourselves professionally. Disagreement is no excuse for poor behavior and poor manners. Disrespectful and unacceptable behavior includes, but is not limited to:
+    * Violent threats or language.
+    * Discriminatory or derogatory jokes and language.
+    * Posting sexually explicit or violent material.
+    * Posting, or threatening to post, people's personally identifying information ("doxing").
+    * Insults, especially those using discriminatory terms or slurs.
+    * Behavior that could be perceived as sexual attention.
+    * Advocating for or encouraging any of the above behaviors.
+* **Understand disagreements:** Disagreements, both social and technical, are useful learning opportunities. Seek to understand the other viewpoints and resolve differences constructively.
+* This code is not exhaustive or complete. It serves to capture our common understanding of a productive, collaborative environment. We expect the code to be followed in spirit as much as in the letter.
+
+### Scope
+
+This code of conduct applies to all repos and communities for LinkedIn-managed open source projects regardless of whether or not the repo explicitly calls out its use of this code. The code also applies in public spaces when an individual is representing a project or its community. Examples include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
+
+Note: Some LinkedIn-managed communities have codes of conduct that pre-date this document and issue resolution process. While communities are not required to change their code, they are expected to use the resolution process outlined here. The review team will coordinate with the communities involved to address your concerns.
+
+### Reporting Code of Conduct Issues
+
+We encourage all communities to resolve issues on their own whenever possible. This builds a broader and deeper understanding and ultimately a healthier interaction. In the event that an issue cannot be resolved locally, please feel free to report your concerns by contacting [[email protected]](mailto:[email protected]).
+
+In your report please include:
+
+*   Your contact information.
+*   Names (real, usernames or pseudonyms) of any individuals involved. If there are additional witnesses, please include them as well.
+*   Your account of what occurred, and if you believe the incident is ongoing. If there is a publicly available record (e.g. a mailing list archive or a public chat log), please include a link or attachment.
+*   Any additional information that may be helpful.
+
+All reports will be reviewed by a multi-person team and will result in a response that is deemed necessary and appropriate to the circumstances. Where additional perspectives are needed, the team may seek insight from others with relevant expertise or experience. The confidentiality of the person reporting the incident will be kept at all times. Involved parties are never part of the review team.
+
+Anyone asked to stop unacceptable behavior is expected to comply immediately. If an individual engages in unacceptable behavior, the review team may take any action they deem appropriate, including a permanent ban from the community.
+
+_This code of conduct is based on the [Microsoft](https://opensource.microsoft.com/codeofconduct/) Open Source Code of Conduct which was based on the [template](http://todogroup.org/opencodeofconduct) established by the [TODO Group](http://todogroup.org/) and used by numerous other large communities (e.g., [Facebook](https://code.facebook.com/pages/876921332402685/open-source-code-of-conduct), [Yahoo](https://yahoo.github.io/codeofconduct), [Twitter](https://engineering.twitter.com/opensource/code-of-conduct), [GitHub](http://todogroup.org/opencodeofconduct/#[email protected])) and the Scope section from the [Contributor Covenant version 1.4](http://contributor-covenant.org/version/1/4/)._

CONTRIBUTING.md

@@ -0,0 +1,39 @@
+Contribution Agreement
+======================
+
+As a contributor, you represent that the code you submit is your original work or
+that of your employer (in which case you represent you have the right to bind your
+employer). By submitting code, you (and, if applicable, your employer) are
+licensing the submitted code to LinkedIn and the open source community subject to
+the BSD 2-Clause license.
+
+General Contribution Tips
+=========================
+
+We welcome any contributions that make TonY more reliable, accurate, usable, or
+extensible. It is generally preferred that new features and behaviors should be
+configurable unless sufficient discussion is held to determine that there is no
+situation in which the previous behavior is desirable. GitHub issues are the
+appropriate forum for such discussions.
+
+Responsible Disclosure of Security Vulnerabilities
+==================================================
+
+**Do not file an issue on Github for security issues.**  Please review
+the [guidelines for disclosure][disclosure_guidelines].  Reports should
+be encrypted using PGP ([public key][pubkey]) and sent to
+[[email protected]][disclosure_email] preferably with the title
+"Vulnerability in Github LinkedIn/tony - <short summary>".
+
+Tips for Getting Your Pull Request Accepted
+===========================================
+
+1. Make sure all new features are tested and the tests pass.
+2. Bug fixes must include a test case demonstrating the error that it fixes.
+3. Open an issue first and seek advice for your change before submitting
+   a pull request. Large features which have never been discussed are
+   unlikely to be accepted. **You have been warned.**
+
+[disclosure_guidelines]: https://www.linkedin.com/help/linkedin/answer/62924
+[pubkey]: https://gist.github.com/chriseppstein/3f45d3a8e6fb42f24cb7b3f77f21381e
+[disclosure_email]: mailto:[email protected]?subject=Vulnerability%20in%20Github%20LinkedIn/tony%20-%20%3Csummary%3E

LICENSE

@@ -0,0 +1,28 @@
+BSD 2-CLAUSE LICENSE
+
+Copyright 2018 LinkedIn Corporation.
+All Rights Reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are
+met:
+
+1. Redistributions of source code must retain the above copyright
+   notice, this list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright
+   notice, this list of conditions and the following disclaimer in the
+   documentation and/or other materials provided with the
+   distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

NOTICE

@@ -0,0 +1,22 @@
+Copyright 2018 LinkedIn Corporation
+All Rights Reserved.
+
+Licensed under the BSD 2-Clause License (the "License").
+See LICENSE in the project root for license information.
+
+================================================================================
+
+This product includes/uses TensorFlow (https://github.com/tensorflow)
+Copyright 2018 The TensorFlow Authors
+License: Apache-2.0
+
+This product includes/uses TensorFlowOnYARN (https://github.com/Intel-bigdata/TensorFlowOnYARN)
+Copyright 2018 Intel-bigdata
+License: Apache-2.0
+
+================================================================================
+
+In addition, this product automatically loads third party code from an external repository
+using the Gradle build system. Such third party code is subject to other license
+terms than as set forth above. In addition, such third party code may also
+depend on and load multiple tiers of dependencies.

build.gradle

@@ -0,0 +1,149 @@
+group "com.linkedin"
+version "0.1.0"
+
+apply plugin: "java"
+apply plugin: "eclipse"
+apply plugin: "idea"
+
+sourceCompatibility = 1.8
+
+buildscript {
+  repositories {
+    maven {
+      url "https://plugins.gradle.org/m2/"
+    }
+  }
+  dependencies {
+    classpath "com.github.jengelman.gradle.plugins:shadow:2.0.4"
+    classpath "com.google.protobuf:protobuf-gradle-plugin:0.8.1"
+    classpath "gradle.plugin.nl.javadude.gradle.plugins:license-gradle-plugin:0.14.0"
+  }
+}
+
+def hadoopVersion = "3.1.1"
+
+ext.deps = [
+    hadoop: [
+        "common": "org.apache.hadoop:hadoop-common:${hadoopVersion}",
+        "common_test": "org.apache.hadoop:hadoop-common:${hadoopVersion}:tests",
+        "hdfs": "org.apache.hadoop:hadoop-hdfs:${hadoopVersion}",
+        "hdfs_client": "org.apache.hadoop:hadoop-hdfs-client:${hadoopVersion}",
+        "hdfs_test": "org.apache.hadoop:hadoop-hdfs:${hadoopVersion}:tests",
+        "mapreduce_client_core": "org.apache.hadoop:hadoop-mapreduce-client-core:${hadoopVersion}",
+        "minicluster": "org.apache.hadoop:hadoop-minicluster:${hadoopVersion}",
+        "yarn_api": "org.apache.hadoop:hadoop-yarn-api:${hadoopVersion}",
+        "yarn_client": "org.apache.hadoop:hadoop-yarn-client:${hadoopVersion}",
+        "yarn_common": "org.apache.hadoop:hadoop-yarn-common:${hadoopVersion}",
+        "yarn_server_test": "org.apache.hadoop:hadoop-yarn-server-tests:${hadoopVersion}:tests"
+    ],
+    external: [
+        "avro": "org.apache.avro:avro:1.8.2",
+        "guava": "com.google.guava:guava:16.0.1",
+        "jackson_databind": "com.fasterxml.jackson.core:jackson-databind:2.8.3",
+        "jackson_dataformat_yaml": "com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.9.6",
+        // Only needed by Hadoop test classes
+        "junit": "junit:junit:4.12",
+        "objenesis": "org.objenesis:objenesis:2.6",
+        "py4j": "net.sf.py4j:py4j:0.8.2.1",
+        "sshd": "org.apache.sshd:sshd-core:1.1.0",
+        "testng": "org.testng:testng:6.4",
+        "text": "org.apache.commons:commons-text:1.4",
+        "zip4j": "net.lingala.zip4j:zip4j:1.3.2"
+    ]
+]
+
+allprojects {
+  project.version= "0.1.0"
+  group = "com.linkedin.tony"
+}
+
+subprojects {
+  apply plugin: "license"
+  apply plugin: "com.google.protobuf"
+  apply plugin: "java"
+  apply plugin: "eclipse"
+  apply plugin: "idea"
+  repositories {
+    mavenCentral()
+  }
+  plugins.withType(JavaPlugin) {
+    sourceCompatibility = 1.8
+    dependencies {
+      // dependency defined in product_spec.json
+      testCompile deps.external.testng
+    }
+
+    test {
+      useTestNG()
+    }
+  }
+
+  license {
+    header rootProject.file('license_header')
+    // Set the year in the license
+    ext.year = Calendar.getInstance().get(Calendar.YEAR)
+    skipExistingHeaders = false
+    excludes(["com/linkedin/tony/rpc/proto/", "**/*.properties"])
+  }
+  configurations {
+    hadoopRuntime.extendsFrom(runtime)
+    hadoopRuntime {
+      exclude group: "org.apache.hadoop"
+    }
+  }
+}
+
+apply plugin: 'distribution'
+
+// Generates a closure which is used to set up the contents
+// for a distribution; parametrized by the name of the
+// configuration to include in the lib directory.
+def generateDistContents(configurationName) {
+  return {
+    into('.') {
+      from rootProject.fileTree('.') {
+        include 'README.md'
+        include 'LICENSE'
+        include 'NOTICE'
+        include 'CONTRIBUTING.md'
+      }
+    }
+    into('bin') {
+      def bashFiles = []
+      rootProject.subprojects.each {
+        bashFiles << it.fileTree("src/main/bash") {
+          include "*.sh"
+        }
+      }
+      from bashFiles
+    }
+    into('lib') {
+      def dependencies = files()
+      def jars = []
+      rootProject.subprojects.each {
+        // Use subtraction to eliminate duplicates
+        dependencies = dependencies + (it.configurations[configurationName] - dependencies)
+        jars << it.jar
+      }
+      from dependencies
+      from jars
+    }
+  }
+}
+
+distributions {
+  // main distribution does not include Hadoop JARs; this is the one
+  // typically expected to be used on a system properly set up with
+  // an existing Hadoop installation.
+  main {
+    baseName = rootProject.name
+    contents generateDistContents('hadoopRuntime')
+  }
+  // fat distribution includes all dependencies.
+  fat {
+    baseName = rootProject.name + '-fat'
+    contents generateDistContents('runtime')
+  }
+}
+
+build.dependsOn(distZip)

gradle/wrapper/gradle-wrapper.properties

@@ -0,0 +1,6 @@
+#Mon Aug 13 14:46:40 PDT 2018
+distributionBase=GRADLE_USER_HOME
+distributionPath=wrapper/dists
+zipStoreBase=GRADLE_USER_HOME
+zipStorePath=wrapper/dists
+distributionUrl=https\://services.gradle.org/distributions/gradle-3.5-rc-2-all.zip