Monitoring Components:

• Prometheus
• Kafka
• Logstash
• Grafana
• Alert Manager
• Node Exporter
• Push Gateway
• Slack

Logger

• Python Application Used To Create And Produce Logs To kafka

Kafka

• Kafka Consumer will receive logs in 'logs' topic.

LogStash

• It will receive logs from Kafka in plain text, reformat them, and send them to Loki.

Loki

• It will start indexing the logs and ship them to Grafana.

Grafana

• Finally, the logs are visualized in Grafana.

Push Gateway and Node Exporter

• To collect system metrics and cron jobs or small processes, and ship them to Prometheus.

Alert Manager

• Wait for a condition to occur, then send a notification to Slack or any other app.