KAFKA-18229: Move configs out of "kraft" directory (apache#18389)

Reviewers: Mickael Maison <[email protected]>, Ismael Juma <[email protected]>, José Armando García Sancio <[email protected]>

README.md

@@ -110,8 +110,8 @@ fail due to code changes. You can just run:
 Using compiled files:
 
     KAFKA_CLUSTER_ID="$(./bin/kafka-storage.sh random-uuid)"
-    ./bin/kafka-storage.sh format --standalone -t $KAFKA_CLUSTER_ID -c config/kraft/reconfig-server.properties
-    ./bin/kafka-server-start.sh config/kraft/reconfig-server.properties
+    ./bin/kafka-storage.sh format --standalone -t $KAFKA_CLUSTER_ID -c config/server.properties
+    ./bin/kafka-server-start.sh config/server.properties
 
 Using docker image:
 

docker/docker_official_images/3.7.0/jvm/Dockerfile

@@ -78,7 +78,7 @@ RUN set -eux ; \
     chmod -R ug+w /etc/kafka /var/lib/kafka /etc/kafka/secrets; \
     cp /opt/kafka/config/log4j.properties /etc/kafka/docker/log4j.properties; \
     cp /opt/kafka/config/tools-log4j.properties /etc/kafka/docker/tools-log4j.properties; \
-    cp /opt/kafka/config/kraft/reconfig-server.properties /etc/kafka/docker/server.properties; \
+    cp /opt/kafka/config/kraft/server.properties /etc/kafka/docker/server.properties; \
     rm kafka.tgz kafka.tgz.asc KEYS; \
     apk del wget gpg gpg-agent; \
     apk cache clean;

docker/docker_official_images/3.7.0/jvm/jsa_launch

@@ -17,9 +17,9 @@
 KAFKA_CLUSTER_ID="$(opt/kafka/bin/kafka-storage.sh random-uuid)"
 TOPIC="test-topic"
 
-KAFKA_JVM_PERFORMANCE_OPTS="-XX:ArchiveClassesAtExit=storage.jsa" opt/kafka/bin/kafka-storage.sh format --standalone -t $KAFKA_CLUSTER_ID -c opt/kafka/config/kraft/reconfig-server.properties
+KAFKA_JVM_PERFORMANCE_OPTS="-XX:ArchiveClassesAtExit=storage.jsa" opt/kafka/bin/kafka-storage.sh format --standalone -t $KAFKA_CLUSTER_ID -c opt/kafka/config/kraft/server.properties
 
-KAFKA_JVM_PERFORMANCE_OPTS="-XX:ArchiveClassesAtExit=kafka.jsa" opt/kafka/bin/kafka-server-start.sh opt/kafka/config/kraft/reconfig-server.properties &
+KAFKA_JVM_PERFORMANCE_OPTS="-XX:ArchiveClassesAtExit=kafka.jsa" opt/kafka/bin/kafka-server-start.sh opt/kafka/config/kraft/server.properties &
 
 check_timeout() {
     if [ $TIMEOUT -eq 0 ]; then

docker/jvm/Dockerfile

@@ -78,7 +78,7 @@ RUN set -eux ; \
     chmod -R ug+w /etc/kafka /var/lib/kafka /etc/kafka/secrets; \
     cp /opt/kafka/config/log4j2.yaml /etc/kafka/docker/log4j2.yaml; \
     cp /opt/kafka/config/tools-log4j2.yaml /etc/kafka/docker/tools-log4j2.yaml; \
-    cp /opt/kafka/config/kraft/reconfig-server.properties /etc/kafka/docker/server.properties; \
+    cp /opt/kafka/config/server.properties /etc/kafka/docker/server.properties; \
     rm kafka.tgz kafka.tgz.asc KEYS; \
     apk del wget gpg gpg-agent; \
     apk cache clean;

docker/jvm/jsa_launch

@@ -17,9 +17,9 @@
 KAFKA_CLUSTER_ID="$(opt/kafka/bin/kafka-storage.sh random-uuid)"
 TOPIC="test-topic"
 
-KAFKA_JVM_PERFORMANCE_OPTS="-XX:ArchiveClassesAtExit=storage.jsa" opt/kafka/bin/kafka-storage.sh format --standalone -t $KAFKA_CLUSTER_ID -c opt/kafka/config/kraft/reconfig-server.properties
+KAFKA_JVM_PERFORMANCE_OPTS="-XX:ArchiveClassesAtExit=storage.jsa" opt/kafka/bin/kafka-storage.sh format --standalone -t $KAFKA_CLUSTER_ID -c opt/kafka/config/server.properties
 
-KAFKA_JVM_PERFORMANCE_OPTS="-XX:ArchiveClassesAtExit=kafka.jsa" opt/kafka/bin/kafka-server-start.sh opt/kafka/config/kraft/reconfig-server.properties &
+KAFKA_JVM_PERFORMANCE_OPTS="-XX:ArchiveClassesAtExit=kafka.jsa" opt/kafka/bin/kafka-server-start.sh opt/kafka/config/server.properties &
 
 check_timeout() {
     if [ $TIMEOUT -eq 0 ]; then

docker/native/Dockerfile

@@ -63,7 +63,7 @@ RUN apk update ; \
     chmod -R ug+w /etc/kafka /opt/kafka /mnt/shared/config ;
 
 COPY --chown=appuser:root --from=build-native-image /app/kafka/kafka.Kafka /opt/kafka/
-COPY --chown=appuser:root --from=build-native-image /app/kafka/config/kraft/reconfig-server.properties /etc/kafka/docker/
+COPY --chown=appuser:root --from=build-native-image /app/kafka/config/server.properties /etc/kafka/docker/
 COPY --chown=appuser:root --from=build-native-image /app/kafka/config/log4j2.yaml /etc/kafka/docker/
 COPY --chown=appuser:root --from=build-native-image /app/kafka/config/tools-log4j2.yaml /etc/kafka/docker/
 COPY --chown=appuser:root resources/common-scripts /etc/kafka/docker/

docs/quickstart.html

@@ -52,10 +52,10 @@ <h5>Using downloaded files</h5>
         <pre><code class="language-bash">$ KAFKA_CLUSTER_ID="$(bin/kafka-storage.sh random-uuid)"</code></pre>
 
         <p>Format Log Directories</p>
-        <pre><code class="language-bash">$ bin/kafka-storage.sh format --standalone -t $KAFKA_CLUSTER_ID -c config/kraft/reconfig-server.properties</code></pre>
+        <pre><code class="language-bash">$ bin/kafka-storage.sh format --standalone -t $KAFKA_CLUSTER_ID -c config/server.properties</code></pre>
 
         <p>Start the Kafka Server</p>
-        <pre><code class="language-bash">$ bin/kafka-server-start.sh config/kraft/reconfig-server.properties</code></pre>
+        <pre><code class="language-bash">$ bin/kafka-server-start.sh config/server.properties</code></pre>
 
         <p>Once the Kafka server has successfully launched, you will have a basic Kafka environment running and ready to use.</p>
 

docs/security.html

@@ -825,7 +825,7 @@ <h3 class="anchor-heading"><a id="security_sasl" class="anchor-link"></a><a href
                         Client credentials may be created and updated dynamically and updated credentials will be used to authenticate new connections.
                         <code>kafka-configs.sh</code> can be used to create and update credentials after Kafka brokers are started.</p>
                     <p>Create initial SCRAM credentials for user <i>admin</i> with password <i>admin-secret</i>:
-                    <pre><code class="language-bash">$ bin/kafka-storage.sh format -t $(bin/kafka-storage.sh random-uuid) -c config/kraft/server.properties --add-scram 'SCRAM-SHA-256=[name="admin",password="admin-secret"]'</code></pre>
+                    <pre><code class="language-bash">$ bin/kafka-storage.sh format -t $(bin/kafka-storage.sh random-uuid) -c config/server.properties --add-scram 'SCRAM-SHA-256=[name="admin",password="admin-secret"]'</code></pre>
                     <p>Create SCRAM credentials for user <i>alice</i> with password <i>alice-secret</i> (refer to <a href="#security_sasl_scram_clientconfig">Configuring Kafka Clients</a> for client configuration):
                     <pre><code class="language-bash">$ bin/kafka-configs.sh --bootstrap-server localhost:9092 --alter --add-config 'SCRAM-SHA-256=[iterations=8192,password=alice-secret]' --entity-type users --entity-name alice --command-config client.properties</code></pre>
                     <p>The default iteration count of 4096 is used if iterations are not specified. A random salt is created if it's not specified.

docs/streams/quickstart.html

@@ -106,13 +106,13 @@ <h4><a id="quickstart_streams_startserver" href="#quickstart_streams_startserver
   Format Log Directories
 </p>
 
-<pre><code class="language-bash">$ bin/kafka-storage.sh format --standalone -t $KAFKA_CLUSTER_ID -c config/kraft/reconfig-server.properties</code></pre>
+<pre><code class="language-bash">$ bin/kafka-storage.sh format --standalone -t $KAFKA_CLUSTER_ID -c config/server.properties</code></pre>
 
 <p>
   Start the Kafka Server
 </p>
 
-<pre><code class="language-bash">$ bin/kafka-server-start.sh config/kraft/reconfig-server.properties</code></pre>
+<pre><code class="language-bash">$ bin/kafka-server-start.sh config/server.properties</code></pre>
 
 <h4><a id="quickstart_streams_prepare" href="#quickstart_streams_prepare">Step 3: Prepare input topic and start Kafka producer</a></h4>
 

docs/upgrade.html

@@ -82,6 +82,9 @@ <h5><a id="upgrade_400_notable" href="#upgrade_400_notable">Notable changes in 4
                         </li>
                         <li>The function <code>onNewBatch</code> in <code>org.apache.kafka.clients.producer.Partitioner</code> class was removed.
                         </li>
+                        <li>The default properties files for KRaft mode are no longer stored in the separate <code>config/kraft</code> directory since Zookeeper has been removed. These files have been consolidated with other configuration files.
+                            Now all configuration files are in <code>config</code> directory.
+                        </li>
                     </ul>
                 </li>
                 <li><b>Broker</b>

trogdor/README.md

@@ -12,8 +12,8 @@ Running Kafka in Kraft mode:
 
 ```
 KAFKA_CLUSTER_ID="$(./bin/kafka-storage.sh random-uuid)"
-./bin/kafka-storage.sh format --standalone -t $KAFKA_CLUSTER_ID -c config/kraft/reconfig-server.properties
-./bin/kafka-server-start.sh config/kraft/reconfig-server.properties  &> /tmp/kafka.log &
+./bin/kafka-storage.sh format --standalone -t $KAFKA_CLUSTER_ID -c config/server.properties
+./bin/kafka-server-start.sh config/server.properties  &> /tmp/kafka.log &
 ```
 Then, we want to run a Trogdor Agent, plus a Trogdor Coordinator.