Running Kafka Exporter in OpenShift

Zhimin Wen
4 min readMar 15, 2022
Image by Karsten Bergmann from Pixabay

In my last blog, I run KEDA to manage the autoscaling based on the consumer group’s lag. Ideally, if we can see this through a Grafana Dashboard, that will be the best. However, a little bit of effort is required for this…

Turn on Kafka Exporter

Kafka exporter is a Prometheus exporter that can monitor Kafka topic, throughputs, consumer groups, and so on. It’s included in the Strimizi Operator and so does it for IBM event streams. To enable it for event streams, configure the CR with the following settings:

...
strimziOverrides:
kafkaExporter: {}
...

You will see an extra POD is running,

$ oc -n es get pods | grep exporter
es-kafka-exporter-69fc97684c-bkccn 1/1 Running 0 2d23h

If we check the POD logs, we can find many broker not connected error message as shown below.

time="2022-03-15T06:31:07Z" level=error msg="Cannot get offset of group eventstreams-apicurio-registry: kafka: broker not connected" source="kafka_exporter.go:384"
time="2022-03-15T06:31:07Z" level=error msg="Cannot get offset of group __strimzi-topic-operator-kstreams: kafka: broker not connected" source="kafka_exporter.go:384"
time="2022-03-15T06:32:07Z" level=error msg="Cannot get offset of group eventstreams-apicurio-registry: kafka: broker not connected" source="kafka_exporter.go:384"
time="2022-03-15T06:32:07Z" level=error msg="Cannot get offset of group test-consumer-group: kafka: broker not connected" source="kafka_exporter.go:384"
time="2022-03-15T06:32:07Z" level=error msg="Cannot get offset of group __strimzi-topic-operator-kstreams: kafka: broker not connected" source="kafka_exporter.go:384"
time="2022-03-15T06:32:08Z" level=error msg="Cannot get describe groups: kafka: broker not connected" source="kafka_exporter.go:373"

However, it seems like the exporter function is not affected with the following curl command,

oc -n es exec -it es-kafka-exporter-69fc97684c-bkccn -- curl localhost:9404/metrics
# HELP go_gc_duration_seconds A summary of the GC invocation durations.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 0.000100557
go_gc_duration_seconds{quantile="0.25"} 0.000185327
...
# TYPE kafka_brokers gauge
kafka_brokers…

--

--