Run the Apache Kafka Passthrough Benchmark

This page reproduces the Apache Kafka Passthrough Benchmark with the Aklivity fork of the OpenMessaging Benchmark (OMB) on AWS. The run sends the same workload to Apache Kafka directly and then through Zilla Plus.

Provision the Environment

Clone and build the OMB fork:

git clone https://github.com/aklivity/openmessaging-benchmark.git
cd openmessaging-benchmark
mvn clean install -Dlicense.skip=true

Follow the prerequisites and Terraform steps in the deployment README with the 2cku setup. It provisions 3 Zilla Plus instances, 6 benchmark clients, 6 Kafka brokers, 3 controllers, and 1 Prometheus instance, all on m5.xlarge.

Run the Workloads

From the OMB root directory, load your SSH key and generate the workloads:

eval $(ssh-agent)
ssh-add ~/.ssh/omb
./confluent-bin/create-blog-workloads.sh

Run the 2 KB, 100 MB/s workload against Apache Kafka directly:

./confluent-bin/run-workloads.sh ak workloads/confluent-blog-workloads-2cku-1p-1c-2Kb-45topic-200part-100MBps.yaml

Run the same workload through Zilla Plus:

./confluent-bin/run-workloads.sh zpak workloads/confluent-blog-workloads-2cku-1p-1c-2Kb-45topic-200part-100MBps.yaml

Each run uses a 30-minute warm-up and a 60-minute measurement.

Chart the Results

When a run completes, OMB writes output.txt to the results/ directory. Generate publish and end-to-end latency charts from it:

python3 ./bin/create_charts.py results/<your-result-folder>/output.txt

Tear Down

From the deployment directory, destroy the AWS resources:

cd driver-kafka/deploy/aklivity-deployment
terraform destroy