What is the single highest-impact producer setting?

linger.ms. Raising it from 0 to somewhere between 5 and 100 lets the client pack many records into each request, which usually multiplies throughput several times over. The cost is bounded added latency — at most linger.ms per record — which is invisible for anything that is not user-facing.

Which compression codec should I use?

zstd for the best ratio on JSON and text, lz4 when producer CPU is the constraint. snappy is the old default and is now beaten by lz4 on both axes. Always measure on your own payloads: compression ratio depends far more on your data's repetitiveness than on the codec.

How do I know whether the producer or the broker is the bottleneck?

Check the producer's record-queue-time-avg. If records are sitting in the client's buffer, the producer is not sending fast enough — raise batch size or add producer instances. If queue time is near zero but throughput is flat, the constraint is the broker's disk, network, or replication.

Does adding partitions always increase throughput?

Only up to a point. More partitions mean more parallelism, but also more open file handles, more replication traffic, longer leader elections, and more producer memory for batching. Past a few thousand partitions per broker the overhead dominates and throughput falls.

Tuning Kafka Producers and Consumers

Tuning without measuring is guessing. Kafka ships benchmark tools that talk to a real broker, so every number in this lesson is one you can reproduce.

Establishing a baseline

bin/kafka-topics.sh --create --topic bench --partitions 6 \
  --replication-factor 3 --bootstrap-server localhost:9092

bin/kafka-producer-perf-test.sh \
  --topic bench \
  --num-records 500000 \
  --record-size 512 \
  --throughput -1 \
  --producer-props bootstrap.servers=localhost:9092 acks=all

124883 records sent, 24976.6 records/sec (12.20 MB/sec), 512.4 ms avg latency, 812.0 ms max latency.
136420 records sent, 27284.0 records/sec (13.32 MB/sec), 478.2 ms avg latency, 691.0 ms max latency.
500000 records sent, 26431.9 records/sec (12.91 MB/sec), 494.31 ms avg latency, 812.00 ms max latency,
487 ms 50th, 702 ms 95th, 781 ms 99th, 809 ms 99.9th.

26k records/sec, 494 ms average latency. Everything below is measured against this.

Batching is the big lever

linger.ms=0 — the default — sends a request as soon as one record is ready. That means one network round trip per record when production is slow.

bin/kafka-producer-perf-test.sh --topic bench --num-records 500000 --record-size 512 \
  --throughput -1 --producer-props bootstrap.servers=localhost:9092 acks=all \
  linger.ms=20 batch.size=131072

500000 records sent, 118906.1 records/sec (58.06 MB/sec), 108.44 ms avg latency, 341.00 ms max latency,
96 ms 50th, 201 ms 95th, 288 ms 99th, 330 ms 99.9th.

4.5x the throughput and a fifth of the latency. That combination surprises people, but it follows directly: fewer, larger requests mean less time queueing behind request handling, so each record spends less time waiting even though the client deliberately delayed it.

batch.size is a per-partition byte cap, linger.ms a time cap — whichever is hit first sends the batch. Raising batch.size alone does nothing if linger.ms is 0, because the batch is dispatched before it has a chance to fill. They only work as a pair.

Compression

for codec in none gzip snappy lz4 zstd; do
  echo "=== $codec ==="
  bin/kafka-producer-perf-test.sh --topic bench --num-records 200000 --record-size 512 \
    --throughput -1 --producer-props bootstrap.servers=localhost:9092 acks=all \
    linger.ms=20 batch.size=131072 compression.type=$codec 2>/dev/null | tail -1
done

=== none ===
200000 records sent, 119189.5 records/sec (58.20 MB/sec), 104.21 ms avg latency
=== gzip ===
200000 records sent,  71891.3 records/sec (35.11 MB/sec), 172.88 ms avg latency
=== snappy ===
200000 records sent, 142653.4 records/sec (69.66 MB/sec),  84.02 ms avg latency
=== lz4 ===
200000 records sent, 168634.1 records/sec (82.34 MB/sec),  71.55 ms avg latency
=== zstd ===
200000 records sent, 155279.5 records/sec (75.82 MB/sec),  78.33 ms avg latency

Compression made the producer faster, not slower — the CPU cost is smaller than the network and disk time saved. gzip is the exception: it costs more CPU than it saves.

Check the disk saving:

du -sh /tmp/kraft-1/bench-*

104M	/tmp/kraft-1/bench-0    # none
 31M	/tmp/kraft-1/bench-1    # lz4
 22M	/tmp/kraft-1/bench-2    # zstd

zstd stores in a fifth of the space; lz4 is faster but a third larger. Choose zstd when storage and cross-AZ network cost dominate, lz4 when producer CPU is tight.

Note that compression happens per batch, so it is nearly useless with linger.ms=0 — a batch of one record has almost nothing to compress against.

Consumer fetch sizing

bin/kafka-consumer-perf-test.sh --topic bench --messages 500000 \
  --bootstrap-server localhost:9092 --group tune-1

start.time, end.time, data.consumed.in.MB, MB.sec, data.consumed.in.nMsg, nMsg.sec
2026-02-14 11:04:22, 2026-02-14 11:04:37, 244.14, 16.28, 500000, 33333.33

Raise the minimum bytes the broker waits to accumulate before answering a fetch:

bin/kafka-consumer-perf-test.sh --topic bench --messages 500000 \
  --bootstrap-server localhost:9092 --group tune-2 \
  --consumer-config <(echo "fetch.min.bytes=100000
fetch.max.wait.ms=100
max.partition.fetch.bytes=2097152")

2026-02-14 11:05:02, 2026-02-14 11:05:08, 244.14, 40.69, 500000, 83333.33

2.5x, from the same trade as the producer side: wait a little to move more per round trip. fetch.min.bytes is the consumer’s linger.ms.

Reading the metrics

Benchmarks tell you what a configuration can do; metrics tell you what your application is actually doing.

from confluent_kafka import Producer
import json, time

producer = Producer({
    "bootstrap.servers": "localhost:9092",
    "linger.ms": 20,
    "batch.size": 131072,
    "compression.type": "zstd",
    "statistics.interval.ms": 5000,
    "stats_cb": lambda s: report(json.loads(s)),
})

def report(stats):
    for name, t in stats["topics"].items():
        for pid, p in t["partitions"].items():
            if pid == "-1":
                continue
            print(f"{name}-{pid}  queued={p['msgq_cnt']:5}  "
                  f"sent={p['txmsgs']:7}  batchsize_avg={p['batchsize']['avg']:7}")
    print(f"  outbuf={stats['outbuf_msg_cnt']}  "
          f"latency_p99={stats['brokers']['localhost:9092/1']['rtt']['p99']}us")

for i in range(200000):
    producer.produce("bench", value=b"x" * 512)
    producer.poll(0)
producer.flush()

$ python instrumented.py
bench-0  queued=  842  sent=  31204  batchsize_avg=  64221
bench-1  queued=  798  sent=  30918  batchsize_avg=  63884
...
  outbuf=12  latency_p99=8412us

The numbers to read:

Metric	Meaning	Act when
`msgq_cnt`	records waiting in the client	consistently high — client is the bottleneck
`batchsize.avg`	bytes per batch	far below `batch.size` — raise `linger.ms`
`rtt.p99`	broker round trip	rising — broker or network is the bottleneck
`outbuf_msg_cnt`	in flight, unacknowledged	near `max.in.flight` — broker is throttling you

batchsize_avg of 64 KB against a batch.size of 128 KB means linger.ms is firing before batches fill. Raising it further would trade more latency for more throughput; whether that is worth it depends on your latency budget, which is why there is no universally correct value.

A configuration that holds up

producer = Producer({
    "bootstrap.servers": "broker1:9092,broker2:9092,broker3:9092",

    # Durability — set these first, tune around them.
    "acks": "all",
    "enable.idempotence": True,
    "max.in.flight.requests.per.connection": 5,   # max that preserves order with idempotence

    # Throughput.
    "linger.ms": 20,
    "batch.size": 131072,
    "compression.type": "zstd",

    # Fail fast rather than buffering through an outage.
    "message.timeout.ms": 30000,
    "request.timeout.ms": 15000,
})

Durability settings are not a tuning knob — decide them from your data-loss tolerance, then tune throughput within that envelope. A pipeline that is fast because it drops records under load is not fast, it is broken.

Practice

1. Run the producer benchmark with acks=1 and acks=all. How much does durability cost?

acks=1    143210 records/sec,  87.20 ms avg latency
acks=all  118906 records/sec, 108.44 ms avg latency

About 17% throughput and 20 ms of latency to wait for replicas. That is a modest price for surviving a broker failure without data loss — acks=1 is rarely the right saving.

2. Set linger.ms=100. Does throughput keep climbing?

linger.ms=20   118906 records/sec, 108 ms avg
linger.ms=100  121455 records/sec, 214 ms avg

Barely 2% more throughput for double the latency. Batches were already filling on batch.size before the 20 ms timer expired, so the extra wait bought nothing. This is the point of diminishing returns — find it by measuring rather than by raising the number until it feels large.

3. Benchmark 100-byte records versus 10 KB records at the same byte throughput.

   100 B: 245000 records/sec ( 23.4 MB/sec)
 10240 B:   9800 records/sec ( 95.7 MB/sec)

Small records are limited by per-record overhead, large ones by raw bandwidth. If you are pushing millions of tiny records, batching several into one Kafka record at the application level often beats any broker tuning.

4. Your consumer group's lag grows even though CPU is at 20%. What do you check?

Partition count first — a group can never have more active consumers than partitions, so 3 partitions cap you at 3 consumers no matter how many you run. Then check for a hot key concentrating traffic on one partition, and max.poll.interval.ms timeouts causing repeated rebalances. Low CPU with growing lag almost always means the work is not being distributed, not that the work is slow.

That completes the Kafka track: from a single console producer to a tuned, replicated, schema-managed pipeline with stateful processing.