benchmark-producer

Command Info

This tool is used to run a Kafka Producer Benchmark. For more information on benchmarking see Benchmark.

Command Usage

Usage of benchmark-producer:
  -bootstrap-host string
      kafka bootstrap host (default "localhost")
  -bootstrap-port int
      kafka bootstrap port (default 9092)
  -client-id string
      client-id to pass along to kafka (default "warpstream-cli")
  -disable-idempotent-write
      disables idempotent write, WARNING: this may cause poor performance as this sets the maximum in-flight requests to 1
  -enable-tls
      dial with TLS or not
  -kafka-log-level string
      the log level to set on the kafka client, accepted values are DEBUG, INFO, WARN, ERROR (default "WARN")
  -max-records-per-second int
      maximum number of records per second to produce per kafka client (default 10000)
  -num-clients int
      number of kafka clients (default 3)
  -num-records int
      number of messages to produce, -1 for unlimited. (default 1000000)
  -produce-record-timeout duration
      maximum amount of time to wait for a record to be produced (default 10s)
  -producer-max-bytes int
      upper bounds the size of a record batch, this mirrors Kafka's max.message.bytes. (default 16000000)
  -prometheus-port int
      the port to serve promethes metrics on, -1 to disable (default 8081)
  -record-size int
      message size in bytes (default 10000)
  -sasl-password string
      password for SASL authentication
  -sasl-scram
      uses sasl scram authentication (sasl plain by default)
  -sasl-username string
      username for SASL authentication
  -tls-client-cert-file string
      path to the X.509 certificate file in PEM format for the client
  -tls-client-key-file string
      path to the X.509 private key file in PEM format for the client
  -tls-disable-hostname-verification
      whether to disable hostname verification or not
  -tls-server-ca-cert-file string
      path to the X.509 certificate file in PEM format for the server certificate authority. If not specified, the host's root certificate pool will be used for server certificate verification.
  -topic string
      the topic to produce to

Example

This is an example of running the producer benchmark tool against a local playground WarpStream Cluster with a single client.

$ warpstream cli benchmark-producer -topic ws-benchmark -num-clients 1

46798 records sent (446.00 MiB), 9359.60 records/sec (89.26 MiB/sec), 186.476ms min latency, 300.85209ms avg latency, 557.667ms max latency, 2842 buffered records.
49441 records sent (471.00 MiB), 9888.20 records/sec (94.30 MiB/sec), 192.584ms min latency, 257.174298ms avg latency, 392.739ms max latency, 2926 buffered records.
49734 records sent (474.00 MiB), 9946.80 records/sec (94.86 MiB/sec), 189.176ms min latency, 250.716986ms avg latency, 316.025ms max latency, 2914 buffered records.
49613 records sent (473.00 MiB), 9922.60 records/sec (94.63 MiB/sec), 186.951ms min latency, 280.346658ms avg latency, 480.603ms max latency, 2890 buffered records.
49032 records sent (467.00 MiB), 9806.40 records/sec (93.52 MiB/sec), 182.87ms min latency, 265.414339ms avg latency, 403.456ms max latency, 2925 buffered records.
49597 records sent (472.00 MiB), 9919.40 records/sec (94.60 MiB/sec), 187.95ms min latency, 265.404274ms avg latency, 484.618ms max latency, 2892 buffered records.
49626 records sent (473.00 MiB), 9925.20 records/sec (94.65 MiB/sec), 185.409ms min latency, 250.42801ms avg latency, 311.784ms max latency, 2970 buffered records.