Skip to content

Commit 52614ad

Browse files
junraojunrao
committed
kafka-1171; Gradle build for Kafka; patched by David Arthur, Joe Stein, Chris Freeman and Jun Rao; reviewed by Guozhang Wang, Joel Koshy and Neha Narkhede
1 parent 8c9a781 commit 52614ad

13 files changed

+695
-46
lines changed

‎HEADER

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
Licensed to the Apache Software Foundation (ASF) under one
2+
or more contributor license agreements. See the NOTICE file
3+
distributed with this work for additional information
4+
regarding copyright ownership. The ASF licenses this file
5+
to you under the Apache License, Version 2.0 (the
6+
"License"); you may not use this file except in compliance
7+
with the License. You may obtain a copy of the License at
8+
9+
http://www.apache.org/licenses/LICENSE-2.0
10+
11+
Unless required by applicable law or agreed to in writing,
12+
software distributed under the License is distributed on an
13+
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
KIND, either express or implied. See the License for the
15+
specific language governing permissions and limitations
16+
under the License.

‎README-sbt.md

+60
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
# Apache Kafka #
2+
3+
See our [web site](http://kafka.apache.org) for details on the project.
4+
5+
## Building it ##
6+
1. ./sbt update
7+
2. ./sbt package
8+
3. ./sbt assembly-package-dependency
9+
10+
To build for a particular version of Scala (either 2.8.0, 2.8.2, 2.9.1, 2.9.2 or 2.10.1), change step 2 above to:
11+
2. ./sbt "++2.8.0 package"
12+
13+
To build for all supported versions of Scala, change step 2 above to:
14+
2. ./sbt +package
15+
16+
## Running it ##
17+
Follow instuctions in http://kafka.apache.org/documentation.html#quickstart
18+
19+
## Running unit tests ##
20+
./sbt test
21+
22+
## Building a binary release zip or gzipped tar ball ##
23+
./sbt release-zip
24+
./sbt release-tar
25+
The release file can be found inside ./target/RELEASE/.
26+
27+
## Other Build Tips ##
28+
Here are some useful sbt commands, to be executed at the sbt command prompt (./sbt). Prefixing with "++<version> " runs the
29+
command for a specific Scala version, prefixing with "+" will perform the action for all versions of Scala, and no prefix
30+
runs the command for the default (2.8.0) version of Scala. -
31+
32+
tasks : Lists all the sbt commands and their descriptions
33+
clean : Deletes all generated files (the target directory).
34+
compile : Compile all the sub projects, but not create the jars
35+
test : Run all unit tests in all sub projects
36+
release-zip : Create all the jars, run unit tests and create a deployable release zip
37+
release-tar : Create all the jars, run unit tests and create a deployable release gzipped tar tall
38+
package: Creates jars for src, test, docs etc
39+
projects : List all the sub projects
40+
project sub_project_name : Switch to a particular sub-project. For example, to switch to the core kafka code, use "project core-kafka"
41+
42+
The following commands can be run only on a particular sub project -
43+
test-only package.test.TestName : Runs only the specified test in the current sub project
44+
run : Provides options to run any of the classes that have a main method. For example, you can switch to project java-examples, and run the examples there by executing "project java-examples" followed by "run"
45+
46+
For more details please see the [SBT documentation](https://github.com/harrah/xsbt/wiki)
47+
48+
## Contribution ##
49+
50+
Kafka is a new project, and we are interested in building the community; we would welcome any thoughts or [patches](https://issues.apache.org/jira/browse/KAFKA). You can reach us [on the Apache mailing lists](http://kafka.apache.org/contact.html).
51+
52+
To contribute follow the instructions here:
53+
* http://kafka.apache.org/contributing.html
54+
55+
We also welcome patches for the website and documentation which can be found here:
56+
* https://svn.apache.org/repos/asf/kafka/site
57+
58+
59+
60+

‎README.md

+67-43
Original file line numberDiff line numberDiff line change
@@ -1,49 +1,77 @@
1-
# Apache Kafka #
1+
# Licensed to the Apache Software Foundation (ASF) under one or more
2+
# contributor license agreements. See the NOTICE file distributed with
3+
# this work for additional information regarding copyright ownership.
4+
# The ASF licenses this file to You under the Apache License, Version 2.0
5+
# (the "License"); you may not use this file except in compliance with
6+
# the License. You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
215

16+
# Apache Kafka #
317
See our [web site](http://kafka.apache.org) for details on the project.
418

5-
## Building it ##
6-
1. ./sbt update
7-
2. ./sbt package
8-
3. ./sbt assembly-package-dependency
19+
## Building a jar and running it ##
20+
1. ./gradlew copyDependantLibs
21+
2. ./gradlew jar
22+
3. Follow instuctions in http://kafka.apache.org/documentation.html#quickstart
923

10-
To build for a particular version of Scala (either 2.8.0, 2.8.2, 2.9.1, 2.9.2 or 2.10.1), change step 2 above to:
11-
2. ./sbt "++2.8.0 package"
24+
## Running unit tests ##
25+
./gradlew test
1226

13-
To build for all supported versions of Scala, change step 2 above to:
14-
2. ./sbt +package
27+
## Forcing re-running unit tests w/o code change ##
28+
./gradlew cleanTest test
1529

16-
## Running it ##
17-
Follow instuctions in http://kafka.apache.org/documentation.html#quickstart
30+
## Running a particular unit test ##
31+
./gradlew -Dtest.single=RequestResponseSerializationTest core:test
1832

19-
## Running unit tests ##
20-
./sbt test
21-
22-
## Building a binary release zip or gzipped tar ball ##
23-
./sbt release-zip
24-
./sbt release-tar
25-
The release file can be found inside ./target/RELEASE/.
26-
27-
## Other Build Tips ##
28-
Here are some useful sbt commands, to be executed at the sbt command prompt (./sbt). Prefixing with "++<version> " runs the
29-
command for a specific Scala version, prefixing with "+" will perform the action for all versions of Scala, and no prefix
30-
runs the command for the default (2.8.0) version of Scala. -
31-
32-
tasks : Lists all the sbt commands and their descriptions
33-
clean : Deletes all generated files (the target directory).
34-
compile : Compile all the sub projects, but not create the jars
35-
test : Run all unit tests in all sub projects
36-
release-zip : Create all the jars, run unit tests and create a deployable release zip
37-
release-tar : Create all the jars, run unit tests and create a deployable release gzipped tar tall
38-
package: Creates jars for src, test, docs etc
39-
projects : List all the sub projects
40-
project sub_project_name : Switch to a particular sub-project. For example, to switch to the core kafka code, use "project core-kafka"
41-
42-
The following commands can be run only on a particular sub project -
43-
test-only package.test.TestName : Runs only the specified test in the current sub project
44-
run : Provides options to run any of the classes that have a main method. For example, you can switch to project java-examples, and run the examples there by executing "project java-examples" followed by "run"
45-
46-
For more details please see the [SBT documentation](https://github.com/harrah/xsbt/wiki)
33+
## Building a binary release gzipped tar ball ##
34+
./gradlew clean
35+
./gradlew releaseTarGz
36+
The release file can be found inside ./core/build/distributions/.
37+
38+
## Cleaning the build ##
39+
./gradlew clean
40+
41+
## Running a task on a particular version of Scala (either 2.8.0, 2.8.2, 2.9.1, 2.9.2 or 2.10.1) ##
42+
## (If building a jar with a version other than 2.8.0, the scala version variable in bin/kafka-run-class.sh needs to be changed to run quick start.) ##
43+
./gradlew -PscalaVersion=2.9.1 jar
44+
./gradlew -PscalaVersion=2.9.1 test
45+
./gradlew -PscalaVersion=2.9.1 releaseTarGz
46+
47+
## Running a task for a specific project in 'core', 'perf', 'contrib:hadoop-consumer', 'contrib:hadoop-producer', 'examples', 'clients' ##
48+
./gradlew core:jar
49+
./gradlew core:test
50+
51+
## Listing all gradle tasks ##
52+
./gradlew tasks
53+
54+
# Building IDE project ##
55+
./gradlew eclipse
56+
./gradlew idea
57+
58+
# Building the jar for all scala versions and for all projects ##
59+
./gradlew jarAll
60+
61+
## Running unit tests for all scala versions and for all projects ##
62+
./gradlew testAll
63+
64+
## Building a binary release gzipped tar ball for all scala versions ##
65+
./gradlew releaseTarGzAll
66+
67+
## Publishing the jar for all version of Scala and for all projects to maven (To test locally, change mavenUrl in gradle.properties to a local dir.) ##
68+
./gradlew uploadArchivesAll
69+
70+
## Building the test jar ##
71+
./gradlew testJar
72+
73+
## Determining how transitive dependencies are added ##
74+
./gradlew core:dependencies --configuration runtime
4775

4876
## Contribution ##
4977

@@ -54,7 +82,3 @@ To contribute follow the instructions here:
5482

5583
We also welcome patches for the website and documentation which can be found here:
5684
* https://svn.apache.org/repos/asf/kafka/site
57-
58-
59-
60-

‎bin/kafka-run-class.sh

+29-2
Original file line numberDiff line numberDiff line change
@@ -32,13 +32,40 @@ if [ -z "$SCALA_VERSION" ]; then
3232
SCALA_VERSION=2.8.0
3333
fi
3434

35+
# TODO: remove when removing sbt
3536
# assume all dependencies have been packaged into one jar with sbt-assembly's task "assembly-package-dependency"
3637
for file in $base_dir/core/target/scala-${SCALA_VERSION}/*.jar;
3738
do
3839
CLASSPATH=$CLASSPATH:$file
3940
done
4041

41-
for file in $base_dir/perf/target/scala-${SCALA_VERSION}/kafka*.jar;
42+
# run ./gradlew copyDependantLibs to get all dependant jars in a local dir
43+
for file in $base_dir/core/build/dependant-libs-${SCALA_VERSION}/*.jar;
44+
do
45+
CLASSPATH=$CLASSPATH:$file
46+
done
47+
48+
for file in $base_dir/perf/build/libs//kafka-perf_${SCALA_VERSION}*.jar;
49+
do
50+
CLASSPATH=$CLASSPATH:$file
51+
done
52+
53+
for file in $base_dir/clients/build/libs//kafka-clients*.jar;
54+
do
55+
CLASSPATH=$CLASSPATH:$file
56+
done
57+
58+
for file in $base_dir/examples/build/libs//kafka-examples*.jar;
59+
do
60+
CLASSPATH=$CLASSPATH:$file
61+
done
62+
63+
for file in $base_dir/contrib/hadoop-consumer/build/libs//kafka-hadoop-consumer*.jar;
64+
do
65+
CLASSPATH=$CLASSPATH:$file
66+
done
67+
68+
for file in $base_dir/contrib/hadoop-producer/build/libs//kafka-hadoop-producer*.jar;
4269
do
4370
CLASSPATH=$CLASSPATH:$file
4471
done
@@ -49,7 +76,7 @@ do
4976
CLASSPATH=$CLASSPATH:$file
5077
done
5178

52-
for file in $base_dir/kafka*.jar;
79+
for file in $base_dir/core/build/libs/kafka_${SCALA_VERSION}*.jar;
5380
do
5481
CLASSPATH=$CLASSPATH:$file
5582
done

0 commit comments

Comments
 (0)