📊

Kafka Metrics Monitoring Overview

Jul 29, 2024

Kafka Metrics Monitoring

Introduction

  • Importance of Kafka metrics monitoring
  • Huge variety of Kafka metrics can be overwhelming

Key Metrics to Monitor

  1. Number of Active Controllers

    • Should always be 1.
    • One broker acts as the active controller.
    • Manages leader election, assignments, etc.
    • Zero indicates cluster not working.
    • More than one implies a significant bug.
  2. Number of Under Replicated Partitions (URP)

    • Should always be 0.
    • Indicates partitions are lagging in replication.
    • High write loads and slow replication can cause under-replication.
    • Be cautious; may indicate overloaded brokers or network issues.
  3. Number of Offline Partitions

    • Should be 0.
    • Offline partitions mean the topic is partially down.

Importance of Monitoring and Alerts

  • Monitor these metrics actively and set alerts.
  • Alerts help in identifying issues quickly (e.g., via email notifications).
  • More metrics monitored can simplify troubleshooting.

Documentation Reference

  • Kafka documentation contains extensive metrics information.
  • Key section: 6.6 Monitoring.
    • Explains how Kafka uses Yammer metrics exposed by JMX.
  • Metrics include error rates, request rates, bytes in/out, etc.
  • Set a normal value for metrics to know when something’s wrong.

Examples of Metrics

  • Under Replicated Partitions:
    • Use Prometheus to check metrics at server and partition levels.
    • Two metrics available: one at server level and one at partition level.

Additional Resources

  • Explore Kafka monitoring documentation for comprehensive metrics list.
  • Breakdown of metrics:
    • Broker Metrics:
      • Under replicated partitions, offline partition counts, active controller count, etc.
    • Consider both producer and consumer metrics as well.

Conclusion

  • Next lecture will cover adding these metrics to Grafana.
  • Explore documentation further to identify additional metrics relevant to your needs.