Running Kafka Exporter in OpenShift
4 min readMar 15, 2022
In my last blog, I run KEDA to manage the autoscaling based on the consumer group’s lag. Ideally, if we can see this through a Grafana Dashboard, that will be the best. However, a little bit of effort is required for this…
Turn on Kafka Exporter
Kafka exporter is a Prometheus exporter that can monitor Kafka topic, throughputs, consumer groups, and so on. It’s included in the Strimizi Operator and so does it for IBM event streams. To enable it for event streams, configure the CR with the following settings:
...
strimziOverrides:
kafkaExporter: {}
...
You will see an extra POD is running,
$ oc -n es get pods | grep exporter
es-kafka-exporter-69fc97684c-bkccn 1/1 Running 0 2d23h
If we check the POD logs, we can find many broker not connected error message as shown below.
time="2022-03-15T06:31:07Z" level=error msg="Cannot get offset of group eventstreams-apicurio-registry: kafka: broker not connected" source="kafka_exporter.go:384"
time="2022-03-15T06:31:07Z" level=error msg="Cannot get offset of group __strimzi-topic-operator-kstreams: kafka: broker not connected" source="kafka_exporter.go:384"
time="2022-03-15T06:32:07Z" level=error msg="Cannot get offset of group…