Exploring IBM EventStreams without Zookeeper

Since the v3.3 release of Apache kafka, Zookeeper can be replaced by the Kraft consensus protcol. This feature is now available in IBM Event Streams. Let’s explore it.

Enable the Kraft Feature

EventStreams is based on the Strimizi operator. After the subscribtion of the operator, lets update the CSV

oc edit csv ibm-eventstreams.v3.2.1

Locate to the place of the json path of,

.spec.install.spec.deployments[0].spec.template.spec.containers[0].env

Update the enviroment variable, STRIMZI_FEATURE_GATES to the value of “+UseKRaft”

Deploy the Kafka

We can then create the CR for the EventStreams, focus on the Kafka overrides settings as below,

strimziOverrides:
  kafkaExporter: {}
  kafka:
    replicas: 3
    authorization:
      type: custom
      supportsAdminApi: true
      authorizerClass: com.ibm.eventstreams.runas.authorizer.RunAsKRaftAuthorizer 
      # must use KRaft authorizer
    config:
      inter.broker.protocol.version: '3.4'
      log.cleaner.threads: 6
      num.io.threads: 24
      num.network.threads: 9
      num.replica.fetchers: 3
      offsets.topic.replication.factor: 3
      default.replication.factor: 3
      min.insync.replicas: 2
      auto.create.topics.enable: "false"
    listeners:
      - name: external
        port: 9094
        type: route
        tls: true
        authentication:
          type: tls
      - name: tls
        port: 9093
        type: internal
        tls: true
        authentication:
          type: tls
    metricsConfig:
      type: jmxPrometheusExporter
      valueFrom:
        configMapKeyRef:
          key: kafka-metrics-config.yaml
          name: prod-3-brokers-metrics-config
    storage:
      type: persistent-claim
      size: 10Gi
      class: {{ .sc }}
  zookeeper:
    replicas: 1
    storage:
      class: {{ .sc }}
      size: 10Gi
      type: persistent-claim

  entityOperator: #add entity operators for topic and users automation by YAML
    # topicOperator: {} #topic operator is not supported in kraft yet
    userOperator: {}

Some points to take note.