This project brings multiple components to build and run a fast-data application, messaging broker, streaming service, indexed DB, and real-time analytics dashboard

You gonna need to build the image of the event-producer. To handle this, go to application's repository and follow the steps to do it (JVM mode).

Architecture

Layer/Role	Component
Application	Event Producer (Quarkus)
Messaging	Apache Kafka (1 broker / 3 partitions)
Streaming	Apache Nifi (Kafka to Elastic)
Database	Elasticsearch 7.9.1 (2 nodes)
Analytics/DataViz	Kibana 7.9.1

Build'n'Run

Docker Compose

Very straight forward, just run the command to start the containers

$ docker-compose -f devops/docker-compose.yml up -d

It takes a couple minutes to start everything :)

Here are the URLs of the components:

Event Producer

• http://localhost:8080/health - health-check
• http://localhost:8080/metrics/application - metrics (prometheus model)

Apache Kafka

• http://localhost:9000/topic/matches - Kafdrop (Kafka Web UI) - topic "matches"

Apache Nifi

• http://localhost:8081/nifi/ - Nifi Web UI

Elasticsearch

• http://localhost:9200/_cat/nodes?v&pretty - Elasticsearch (check nodes)

Kibana (the last one in the starting phase)

• http://localhost:5601/ - Kibana

To turn the things off:

$ docker-compose -f devops/docker-compose.yml down -v

Setup

Apache Nifi

Import the xml template (setup/nifi/kafka-to-elastic.xml) of the streaming processors (kafka-to-elastic) and start the processors.

Drag the template to the middle of the screen

and a selection box will appear.

This is what you gonna have if did everything properly :)

Just start and you're good to go!

Kibana

Import the "json" file in the "Saved Objects" section. This will allow you to access the dashboard.

Select the file

Dashboard

For a better experience, turn the dark theme ON.

To do this, access the Advanced Settings page and toggle it on.

Now just look for the Dashboard name as "Matches" in the Kibana panel (in the left)

This is your prize :) enjoy, explore, go beyond!