# Configure Clients to Eliminate AZ Networking Costs

How to configure your Kafka clients to keep all traffic zone-local.

With WarpStream, there are no Availability Zone (AZ) networking costs between Agents. This means you can produce and consume data from different AZs without incurring additional networking expenses. The same applies to Agent \<--> client communication: WarpStream eliminates AZ networking costs, allowing you to connect clients only to agents within the same AZ.

## Requirements for Zonal Alignment of Kafka clients

:::caution
WarpStream's zone-aware service discovery system is incompatible with standard Kafka client's rack-aware consumer strategy. When using WarpStream, do **not** configure the rack field on your Kafka clients or enable the rack-aware consumer strategy. If you enable those setting, your Kafka consumers may experience a large amount of unnecessary rebalances. Instead, unset those fields and follow the instructions on this page.
:::

To ensure your Kafka clients connect to Agents within the same availability zone, you need to ensure there is at least one Agent in the same availability zone as your clients and provide the Kafka client’s availability zone. There are two ways to provide the availability zone information:

1. Specifying the availability zone in your client ID
2. Mapping subnets to availability zones in the Agent configuration

### Specifying the availability zone in your client ID

Append the following value to your Kafka client's `ClientID`: `ws_az=<your-az>`. This flag indicates the AZ in which the client is operating.

**Diagram: Zone-aligned Kafka clients.** In each of three availability zones, a producer and a consumer set ws_az to their zone in their client ID, so they only connect to the Agent in the same zone and no traffic crosses zones.

#### Example

Here are examples using various kafka libraries of how to set up the `clientID` with the AZ flag

<details>

<summary>librdkafka/confluent-kafka</summary>

Generic Configuration

```ini
client.id=application-foo,ws_az=us-east-1a
```

Python

```python
availability_zone = lookup_az()
client_id = "application-foo"

p = Producer({
    'client.id': f"{client_id},ws_az={availability_zone}",
})
```

Golang

```go
availabilityZone := lookupAZ()
clientID = "application-foo"

p, err := kafka.NewProducer(&kafka.ConfigMap{
	"client.id": fmt.Sprintf("%s,ws_az=%s", clientID, availabilityZone)
})
if err != nil {
	panic(err)
}
```

Javascript

```javascript
availabilityZone = lookupAZ()
clientID = "application-foo"

const producer = new Kafka().producer({
    'client.id': `${clientID},ws_az=${availabilityZone}`
});
```

</details>

<details>

<summary>Java</summary>

```java
String availabilityZone = lookupAZ()
String clientID = "application-foo"

Properties properties = new Properties();
properties.setProperty(ConsumerConfig.CLIENT_ID_CONFIG, String.format("%s,ws_az=%s", clientID, availabilityZone));
KafkaConsumer<String, String> consumer = new KafkaConsumer<>(properties);
```

</details>

<details>

<summary>Franz-go</summary>

```go
availabilityZone := lookupAZ()
clientID = "application-foo"

cl, err := kgo.NewClient(
    kgo.SeedBrokers("localhost:9092"),
    kgo.ClientID(fmt.Sprintf("%s,ws_az=%s", clientID, availabilityZone)),
)
```

</details>

<details>

<summary>Sarama</summary>

```go
availabilityZone := lookupAZ()
clientID = "application-foo"

config := sarama.NewConfig()
config.ClientID = fmt.Sprintf("%s,ws_az=%s", clientID, availabilityZone)

client, err := sarama.NewConsumerGroup(brokers, groupID, config)
```

</details>

<details>

<summary>KafkaJS</summary>

```javascript
availabilityZone = lookupAZ()
clientID = "application-foo"

const kafka = new Kafka({
  clientId: `${clientID},ws_az=${availabilityZone}`
});
```

</details>

To find the AZ that your application is in it is recommended to use your Cloud Provider's Metadata API, for example in AWS querying the following URL: `http://169.254.169.254/latest/meta-data/placement/availability-zone` will give you the availability zone. Alternatively, if you're [using availability zone IDs](/warpstream/kafka/configure-kafka-client/configure-clients-to-eliminate-az-networking-costs#mapping-availability-zones-across-aws-accounts) then you would query: `http://169.254.169.254/latest/meta-data/placement/availability-zone-id`

Our [warpstream-go library](https://github.com/warpstreamlabs/warpstream-go/blob/main/pkg/cloudmetadata/cloud_metadata.go) has sample code that demonstrates how to query for your application's availability zone in every major cloud.

To learn more about the WarpStream-specific client ID features (like `ws_si` and `ws_az`) check out [our documentation on configuring Kafka Client ID features](/warpstream/kafka/configure-kafka-client/configuring-kafka-client-id-features).

### Mapping subnets to availability zones in the Agent configuration

Pass a subnet mapping to the agents via the `-zonedCIDRBlocks`command-line flag or the `WARPSTREAM_ZONED_CIDR_BLOCKS` environment variable. This mapping allows the agents to determine which zone the Kafka client is sending traffic from. The value needs to be a `<>` delimited list of availability zone to CIDR range pairs, where each pair starts with an AZ, a `@`, and a comma separated list of CIDR blocks representing the range of IPs used by the Kafka clients in that given AZ. For example, `us-east-1a@10.0.0.0/19,10.0.32.0/19<>us-east-1b@10.0.64.0/19<>us-east-1c@10.0.96.0/19` indicates that the Kafka clients with IPs that match `10.0.0.0/19` and `10.0.32.0/19` belong to `us-east-1a`, those with IPs that match `10.0.64.0/19` belong to `us-east-1b`, and those with IPs that match `10.0.96.0/19` belong to `us-east-1c`.

Note that if an AZ is appended to the Kafka client's `ClientID` and a subnet mapping is also provided to the agents but the values conflict with each other, the AZ from the `ClientID` will be used as the source of truth.

### Mapping availability zones across AWS accounts using IDs

Availability zone names are not consistent between different AWS accounts. If you're deploying a WarpStream cluster where Agents and Clients will be deployed in different AWS accounts, you'll want the Agents to advertise their **availability zone ID** instead of their **availability zone name**. This can be accomplished by setting the `WARPSTREAM_LOOKUP_AVAILABILITY_ZONE_ID=true` (Agent v706+) environment variable on the Agents.

Similarly, if you're using the client ID strategy, your Kafka clients will need to advertise their availability zone ID instead of availability zone name. Alternatively, if you're using the subnet-mapping strategy, your mapping should refer to avialability zone IDs instead of names.

## Dealing with Zonal Load Imbalances

WarpStream will always route client connections to an Agent in the same zone as the client (when zone-alignment is properly configured) as long as there is at **least one Agent** in the desired zone. WarpStream will only allow the client to establish a cross-zone connection if there are **zero Agents** in the desired zone.

This means that enabling zone alignment can result in load-imbalances between availability zones if your client traffic is not equally balanced across all zones. This is not a problem in of itself, but it can lead to problems if your Agent auto-scaler is not configured to scale each zone independently.

For example, imagine a scenario where the WarpStream Agents are deployed across zones A, B, and C, with an equal number of Agents in each zone, but zone A has three times as much client traffic as zones B and C. In this scenario, the _average_ CPU utilization of all the Agents across all the zones may not be enough for Kubernetes to trigger an up-scale, but the Agents in zone A may be overloaded resulting in degradation of client performance in that zone.

This problem is easily resolved by configuring your auto-scaler to scale each zone independently. For more details on this, see the [zone-specific scaling section](/warpstream/agent-setup/deploy#zone-specific-scaling) of the HPA documentation.

## Monitoring that Zonal Alignment is Correctly Configured

WarpStream's [automated diagnostics system](/warpstream/agent-setup/monitor-the-warpstream-agents/diagnostics) will automatically detect if it thinks Kafka client zonal alignment is misconfigured warn you via a cost diagnostic in the UI, as well as emit a metric for the diagnostic itself. The diagnostic will include a sample client ID for the misconfigured client.

In addition, WarpStream Agents emit two metrics that are tagged by the detected availability zone of the Kafka client:

1. `warpstream_agent_kafka_produce_compressed_bytes_counter`
2. `warpstream_agent_kafka_fetch_compressed_bytes_counter`

If either of these metrics are being emitted where the `client_az` value doesnt not match the availability zone of the Agent itself, cross-az traffic is occurring.