Kafka Producer / Consumer connected to Twitter API

In this tutorial, we will play with the new Kafka service offered by OVHcloud.

We will use the Twitter v2 API to produce data and then send it to the Kafka cluster.

We will finish by creating a small monitoring tool in Bash.

Note: In a future release, we will develop a web page that will expose the Kafka cluster stream.

To make it easier for you, you can run this tutorial entirely via Gitpod ❤️, just click on the Open in Gitpod button provided.

Let's go, have fun with OVHcloud managed Kafka

The tutorial is organized in several successive steps. You can start at any stage, each stage contains the necessary content required.

STEP 01 - Subscribe and connect to a new OVHcloud managed Kafka service

👟 Jump to Step01

STEP 02 - Get access to the Twitter API

🤿 Dive in Step02

STEP 03 - Create and launch the producer daemon

🥊 Knock the Step03

STEP 04 - Create a Bash consumer

👉 Follow Step04

STEP 05 - Push producer logs to Opensearch

🛝 Slide to Step05