getting-started

Getting started with Application Services

Important

Application Services is currently available for Development Preview. Development Preview releases provide early access to a limited set of features that might not be fully tested and that might change in the final GA version. Users should not use Development Preview software in production or for business-critical workloads. Limited documentation is available for Development Preview releases and is typically focused on fundamental user goals.

As a developer of applications and services, you can use Application Services to create and set up Kafka instances and connect your applications and services to these instances. App Services is a managed cloud service that enables you to add Kafka data-streaming functionality in your applications without having to install, configure, run, and maintain your own Kafka clusters.

Prerequisites

• JDK 11 or later is installed.

• For Windows, the latest version of Oracle JDK is installed.

Creating a Kafka instance in App Services

Use the App Services web console to create and configure a Kafka instance for your applications or services. A Kafka instance in App Services includes a Kafka cluster, bootstrap server, and the configurations needed to connect to producer and consumer services.

Prerequisites

• You’re logged in to the App Services web console at https://console.redhat.com/beta/application-services/streams/.

Procedure

1. In the App Services web console, go to Streams for Apache Kafka > Kafka Instances and click Create Kafka instance.

2. Enter a unique Name for the Kafka instance, such as my-first-kafka-instance. For this release of App Services, all other fields are preconfigured.

3. Click Create instance to start the creation process for your Kafka instance.

   

   Figure 1. Kafka instance configuration details

   The new Kafka instance is listed in the instances table. When the instance Status is Ready, you can start using the Kafka instance. You can use the options icon (three vertical dots) to view instance details, connect to the instance, or delete the instance.

   

   Figure 2. Kafka instance options menu

   Note	Although you can see Kafka instances created by other users in your organization, you can’t manage or connect to those instances. Only the user who creates an instance can edit or delete the instance, access the associated service account and topics, or connect to the instance.

Verification

1. Verify that the new Kafka instance is listed in the instances table.

2. Verify that the state of the new Kafka instance is shown as Ready.

Creating a service account to connect to a Kafka instance in App Services

To connect your applications or services to a Kafka instance in the App Services web console, you need to create a service account that’s associated with the instance. You also need to save the bootstrap server endpoint for the instance, the generated service account credentials, and the authentication token endpoint to a secure location. You’ll use the service account and Kafka instance information later when you configure your application.

Prerequisites

• You’ve created a Kafka instance and the instance is in Ready state.

Procedure

1. In the Kafka Instances page of the web console, for the relevant Kafka instance that you want to connect to, select the options icon (three vertical dots) and click Connection.

2. In the Connection page, copy the Bootstrap server endpoint to a secure location. This is the bootstrap server endpoint that you’ll need for connecting to this Kafka instance.

3. Click Create service account to set up the account that you’ll use to access this Kafka instance.

4. Enter a unique service account name, such as my-service-account, add an optional description, and click Create.

5. Copy the generated Client ID and Client Secret to a secure location. These are the credentials that you’ll use to connect to this Kafka instance.

   Important

   The generated credentials are displayed only one time, so ensure that you’ve successfully and securely saved the copied credentials before closing the credentials window.

6. After you save the generated credentials to a secure location, select the confirmation check box in the credentials window and close the window.

7. For the Authentication method, copy the SASL/OAUTHBEARER Token endpoint URL to a secure location. This is the endpoint that you’ll use with your service account credentials to authenticate the connection to this Kafka instance.

   Note	SASL/PLAIN authentication is also available for tools and libraries that don’t support SASL/OAUTHBEARER, but SASL/OAUTHBEARER is recommended whenever possible. With SASL/PLAIN authentication, you use only the service account credentials to authenticate the connection to the Kafka instance.

   You’ll use the service account information that you saved to configure your application to connect to your Kafka instances when you’re ready. For example, if you plan to use Kafkacat to interact with your Kafka instance, you’ll use this information to set your bootstrap server and client environment variables.

8. To review your service account information, reset your credentials, or delete the service account, go to the Streams for Apache Kafka > Service Accounts page in the web console.

Verification

1. Verify that the bootstrap server, client credentials, and authentication token endpoint are saved to a secure location.

2. Verify that your service account was successfully created in the Streams for Apache Kafka > Service Accounts page.

Creating a Kafka topic in App Services

After you create a Kafka instance, you can create Kafka topics to start producing and consuming messages in your services.

Prerequisites

• You’ve created a Kafka instance and the instance is in Ready state.

Procedure

1. In the Kafka Instances page of the web console, click the name of the Kafka instance that you want to add a topic to.

2. Select the Topics tab, click Create topic, and follow the guided steps to define the topic details. Click Next to complete each step and click Finish to complete the setup.

   

   Figure 3. Guided steps to define topic details

   • Topic name: Enter a unique topic name, such as my-first-kafka-topic.

   • Partitions: Set the number of partitions for this topic. This example sets the partition to 1 for a single partition. Partitions are distinct lists of messages within a topic and enable parts of a topic to be distributed over multiple brokers in the cluster. A topic can contain one or more partitions, enabling producer and consumer loads to be scaled.

   • Message retention: Set the message retention time and size to the relevant value and increment. This example sets the retention time to A week and the retention size to Unlimited. Message retention time is the amount of time that messages are retained in a topic before they are deleted or compacted, depending on the cleanup policy. Retention size is the maximum total size of all log segments in a partition before they are deleted or compacted.

   • Replicas: For this release of App Services, the replicas are preconfigured. The number of partition replicas for the topic is set to 3 and the minimum number of follower replicas that must be in sync with a partition leader is set to 2. Replicas are copies of partitions in a topic. Partition replicas are distributed over multiple brokers in the cluster to ensure topic availability if a broker fails. When a follower replica is in sync with a partition leader, the follower replica can become the new partition leader if needed.

   After you complete the topic setup, the new Kafka topic is listed in the topics table. You can now start producing and consuming messages to and from this topic using services that you connect to this instance.

3. In the topics table, on the right side of the Kafka topic, use the options icon (three vertical dots) to edit or delete the topic as needed.

   

   Figure 4. Edit or delete Kafka topic

Verification

• Verify that the new Kafka topic is listed in the topics table.

Viewing metrics for a Kafka instance in App Services

After you create Kafka topics in your Kafka instance and start producing and consuming messages in your services, use the Dashboard tab in the Kafka instance to view metrics for the instance and topics. These metrics help you understand the performance and data usage for your Kafka instance and topics.

Prerequisites

• You’ve created a Kafka instance and the instance is in Ready state.

• You’ve created topics in your Kafka instance and you’ve started producing and consuming messages in your services.

Procedure

1. In the Kafka Instances page of the web console, click the name of the Kafka instance for which you want to view metrics.

2. Select the Dashboard tab and review the metrics for your Kafka instance and topics. Use the drop-down filters to refine the displayed metrics as needed.

   Note	In some cases, you might need to wait several minutes for the latest metrics to appear.

   The following metrics are available:

   • Kafka instance metrics

     • Used disk space: Used disk space is the amount of disk space used by the Kafka brokers in the instance. This metric enables you to assess available disk space relative to the limit. To reduce used disk space, you can adjust topic retention time or other topic properties as needed.

   • Topic metrics

     • Total bytes: Total bytes is the total incoming and outgoing bytes for all topics or for a selected topic in the Kafka instance. This metric enables you to assess data transfer in and out of your Kafka instance. To modify incoming and outgoing bytes, you can adjust topic message size or other topic properties as needed.

Verification

• Verify that metrics are displayed for your Kafka instance and topics. You might need to wait several minutes for the latest metrics to appear.

Additional resources

• Configuration in Kafka