telefonica/kafka-prometheus repository overview

⁠kafka with prometheus exporter

The image is based on confluentinc/cp-kafka⁠, check its Docker Hub page⁠ for more information regarding the base.

This image includes the following extensions:

• Adds the following environment variables:

  • CMD_KAFKA_ADVERTISED_LISTENERS: includes a command that will be executed at run time and whose output will be assigned to the variable KAFKA_ADVERTISED_LISTENERS. This is useful when the advertised host name is only known at run time.

      services:
        kafka:
          image: telefonica/kafka-prometheus
          environment:
            # Use the AWS metadata service to get the public IP and use it to build the Kafka advertised listeners
            CMD_KAFKA_ADVERTISED_LISTENERS: "echo PLAINTEXT://`curl -s http://169.254.169.254/latest/meta-data/public-ipv4`:9092"
    

    Copy

  • CMD_KAFKA_BROKER_RACK: includes a command that will be executed at run time and whose output will be assigned to the variable KAFKA_BROKER_RACK. This is useful when the rack of the broker is only known at run time.

      services:
        kafka:
          image: telefonica/kafka-prometheus
          environment:
            # Use the AWS metadata service to get the availability zone and use it to set the broker rack id
            CMD_KAFKA_BROKER_RACK: "curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone"
    

    Copy

  • CMD_KAFKA_BROKER_ID: includes a command that will be executed at run time and whose output will be assigned to the variable KAFKA_BROKER_ID. This is useful when the identifier of the broker is only known at run time.

      services:
        kafka:
          image: telefonica/kafka-prometheus
          environment:
            # Use part or an environment variable (the HOSTNAME in the example) to get the broker id
            CMD_KAFKA_BROKER_ID: "echo ${HOSTNAME##*-}"
    

    Copy

  • CUSTOM_INIT_SCRIPT: includes a command that will be executed at run time, before Kafka starts.

• Loads the Prometheus JMX exporter⁠ as a -javaagent. It exposes a /metrics HTTP endpoint on port 7071.

• Installs the jq tool⁠.

⁠build

$ docker build -t telefonica/kafka-prometheus:<tag> .

Tags must follow the convention x.y.z-N where x.y.z matches the Confluent base image tag and N is our build number.

⁠override the filter

Mount a config or replace the file in the following path:

/etc/kafka/kafka-jmx-exporter.yml