warpstream cli

The warpstream cli subcommand is the command-line tool for interacting with a WarpStream cluster. It provides functionality similar to the Apache Kafka command-line tools for administering, producing to, and consuming from a Kafka cluster.

Usage:

warpstream cli <sub-command> [options]

Run warpstream cli with no sub-command to print the list of available sub-commands, or run warpstream cli <sub-command> --help to see the flags a specific sub-command accepts.

Note

warpstream cli-beta is a deprecated alias for warpstream cli and will be removed in a future release. warpstream cli-old (and its equivalent warpstream kcmd) refers to the previous generation of the CLI; see warpstream-kcmd.md for its reference.

Common flags

Every sub-command that talks to a Kafka broker accepts the following flags. They can also be supplied via the corresponding environment variables where noted.

Flag	Environment variable	Description
-bootstrap-host	BOOTSTRAP_HOST	Kafka bootstrap host. Default: localhost.
-bootstrap-port	BOOTSTRAP_PORT	Kafka bootstrap port. Default: 9092.
-client-id		Client ID to pass along to Kafka. Default: warpstream-cli.
-enable-tls		Dial with TLS.
-tls-client-cert-file		Path to the X.509 client certificate file (PEM).
-tls-client-key-file		Path to the X.509 client private key file (PEM).
-tls-server-ca-cert-file		Path to the X.509 server CA certificate file (PEM). If unset, the host’s root certificate pool is used.
-tls-disable-hostname-verification		If set to true, hostname verification is disabled (with agent v849+).
-sasl-scram		Use SASL SCRAM authentication (SASL PLAIN by default).
-sasl-username	SASL_USERNAME	Username for SASL authentication.
-sasl-password	SASL_PASSWORD	Password for SASL authentication.
-kafka-log-level		Log level of the underlying Kafka client. One of DEBUG, INFO, WARN, ERROR. Default: WARN.

Sub-commands accept additional flags of their own — see each command’s page or run warpstream cli <sub-command> --help.

Sub-commands

Topic administration

Sub-command	Description
create-topic	Create a new topic.
delete-topics	Delete one or more topics.
alter-topic-config	Add, update, or delete topic-level configuration values. Incremental by default; pass --full-state to replace the entire config set.
describe-topic-configs	Print the current configuration values for a topic.

Broker & cluster administration

Sub-command	Description
alter-broker-config	Add, update, or delete cluster-level (broker) configuration values. Use --delete-config to reset a value back to its WarpStream default. Incremental by default; pass --full-state to replace the entire config set.
describe-broker-configs	Print the current cluster-level (broker) configuration values.
broker-metadata	Print broker metadata as returned by the Kafka Metadata API.
api-versions	Print the Kafka API versions supported by the broker.

Consumer groups

Sub-command	Description
describe-groups	Print information about one or more consumer groups.
consumer-group-lag	Print consumer-group lag per partition.
commit-offsets	Commit offsets for a consumer group.

Access control (ACLs)

Sub-command	Description
create-acls	Create Kafka ACLs.
describe-acls	Print the ACLs currently defined on the cluster.
delete-acls	Delete Kafka ACLs.

Producing & consuming

Sub-command	Description
console-producer	Read records from stdin and produce them to a topic.
console-consumer	Consume records from a topic and print them to stdout.
benchmark-producer	Run a producer benchmark against the cluster.
benchmark-consumer	Run a consumer benchmark against the cluster.

Diagnostics & tooling

Sub-command	Description
diagnose-connection	Diagnose connectivity from this machine to a WarpStream/Kafka cluster.
diagnose-record	Print diagnostic information about a specific record at a given topic/partition/offset.
describe-log-dirs	Print the log-directory information returned by the broker.
certificate-to-dn	Print the distinguished name (DN) that a given X.509 client certificate resolves to.
file-reader	Read a WarpStream Agent segment file directly from object storage and print its contents.
file-scrubber	Validate the integrity of a WarpStream Agent segment file in object storage.
validate-pipeline-config	Validate a managed data pipeline configuration YAML file locally without contacting the control plane.

Altering topic and broker configuration

alter-topic-config and alter-broker-config use the same flags as Kafka’s kafka-configs tool:

Flag	Description
--add-config	Comma-separated key=value pairs to set. Square brackets group values that contain commas, e.g. k1=v1,k2=[v1,v2].
--delete-config	Comma-separated config names to delete (reset back to their WarpStream default), e.g. retention.ms,segment.ms.
--full-state	Replace the entire config set with --add-config instead of altering only the specified configs. Any config not listed is reset to its default.

By default these commands alter only the configs you specify (an incremental alter); every other config on the topic or cluster is left untouched.

# Set one or more configs on a topic (leaves other configs unchanged).
warpstream cli alter-topic-config \
  --topic my-topic \
  --add-config "retention.ms=259200000,max.message.bytes=1048576"

# Reset a topic config back to its default.
warpstream cli alter-topic-config \
  --topic my-topic \
  --delete-config "retention.ms"

# Replace the entire config set; any config not listed is reset to its default.
warpstream cli alter-topic-config \
  --topic my-topic \
  --add-config "retention.ms=259200000" \
  --full-state

# Set a cluster-level (broker) config.
warpstream cli alter-broker-config \
  --add-config "warpstream.default.topic.type=lightning"