# Consumer Group Protocol (KIP-848) Migration Guide [KIP-848](https://cwiki.apache.org/confluence/display/KAFKA/KIP-848%3A+The+Next+Generation+of+the+Consumer+Rebalance+Protocol) is the next generation of Kafka's consumer group protocol. WarpStream has full support for this protocol including migration of groups on the older protocol to the newer protocol. ## Configuration ### Broker configs | Setting | Default | Description | | -------------------------------------- | --------------- | ------------------------------------------------------------------------------------------------------- | | `group.consumer.session.timeout.ms` | `45000` | How long a member can go without heartbeating before it's removed from the group. | | `group.consumer.heartbeat.interval.ms` | `5000` | How often members send heartbeats. | | `group.consumer.max.size` | `32000` | Maximum number of members in a group. | | `group.consumer.migration.policy` | `bidirectional` | Whether a group with active members can switch protocols. See [Migration policy](#migration-policy). | These can be altered using the alter config or incremental alter config kafka apis. ### Group configs You can override the session timeout and heartbeat interval for an individual group with `consumer.session.timeout.ms` and `consumer.heartbeat.interval.ms` consumer group configs using the incremental alter configs kafka api. ## Migrating from the older classic protocol Migrated consumers will resume from the last committed offset for a partition. ### 1. Upgrade your clients Upgrade to a client version that supports KIP-848. Opting in is a client-side setting: for the Java client and librdkafka, set `group.protocol=consumer`. See [Client migration guides](#client-migration-guides) for each client's minimum version and instructions. ### 2. Update your consumer configuration Remove settings that only apply to the classic protocol: `session.timeout.ms`, `heartbeat.interval.ms`, and `partition.assignment.strategy`. Some clients, including the Java client, refuse to start if they're set together with `group.protocol=consumer`. Use the cluster-wide or per-group settings in [Configuration](#configuration) instead. ### 3. Rolling restart your consumers **Rolling migration.** The first client which connects with WarpStream on the new protocol will cause the group to automatically convert to the modern protocol. Existing members still heartbeating on the old protocol are supported. As all clients finish rotating, all members of the group will be on the new protocol. This requires the [migration policy](#migration-policy) to be `upgrade` or `bidirectional`. WarpStream won't convert a group that still has classic members if any of the following are true: * The migration policy doesn't allow upgrades. * The group uses a custom assignor which isn't supported in the old protocol. ### Migration policy `group.consumer.migration.policy` controls whether a group with active members can switch protocols. Empty groups can always switch. | Value | Classic to KIP-848 | KIP-848 to classic | | --------------- | ------------------ | ------------------ | | `disabled` | Offline only | Offline only | | `upgrade` | Offline or rolling | Offline only | | `downgrade` | Offline only | Offline or rolling | | `bidirectional` | Offline or rolling | Offline or rolling | The default is `bidirectional`. ### Rolling back To roll back, switch clients back to the classic protocol, for example with `group.protocol=classic`. The group will only roll back once every member on the newer protocol has left the group. ## Client migration guides Each client library documents its own KIP-848 migration steps. This table covers the clients in [Tuning for Performance](/warpstream/kafka/configure-kafka-client/tuning-for-performance#client-settings). | Client | KIP-848 support | Migration guide | | -------------------------- | ------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Java client | 4.0 and later | [Consumer Rebalance Protocol: Upgrade & Downgrade](https://kafka.apache.org/43/operations/consumer-rebalance-protocol/#upgrade--downgrade) | | librdkafka | 2.12.0 and later | [Next Generation Consumer Group Protocol (KIP-848)](https://docs.confluent.io/platform/current/clients/librdkafka/html/md_INTRODUCTION.html#next-generation-consumer-group-protocol-kip-848) | | confluent-kafka-go | 2.12.0 and later | [KIP-848 Migration Guide](https://github.com/confluentinc/confluent-kafka-go/blob/HEAD/kafka/kip-848-migration-guide.md) | | confluent-kafka-python | 2.12.0 and later | [KIP-848 Migration Guide](https://docs.confluent.io/platform/current/clients/confluent-kafka-python/html/index.html#kip-848-migration-guide) | | confluent-kafka-javascript | 1.6.0 and later | Follow the [librdkafka guide](https://docs.confluent.io/platform/current/clients/librdkafka/html/md_INTRODUCTION.html#next-generation-consumer-group-protocol-kip-848). Set `group.protocol` outside the `kafkaJS` config block. | | franz-go | 1.22.0 and later | Enable [`kgo.ServerSideBalancer`](https://pkg.go.dev/github.com/twmb/franz-go/pkg/kgo#ServerSideBalancer), then follow the [upgrade and downgrade flow](https://kafka.apache.org/43/operations/consumer-rebalance-protocol/#upgrade--downgrade). | | Segment kafka-go | Not supported | None | | Sarama | Not supported | [Tracking issue](https://github.com/IBM/sarama/issues/3818) | | KafkaJS | Not supported | [Migrate to confluent-kafka-javascript](https://docs.confluent.io/kafka-clients/javascript/current/migration.html#migrate-to-javascript-client-from-kafkajs), then follow the [KIP-848 guide](https://docs.confluent.io/platform/current/clients/librdkafka/html/md_INTRODUCTION.html#next-generation-consumer-group-protocol-kip-848). | | Kafka Connect | Not supported. Connect workers use their own group protocol. | None | | Spark | Not applicable. Spark tracks offsets in its own checkpoints. | [Kafka-specific configurations](https://spark.apache.org/docs/latest/streaming/structured-streaming-kafka-integration.html#kafka-specific-configurations) | ## See also * [Broker Configuration Reference](/warpstream/kafka/reference/protocol-and-feature-support/broker-configuration-reference) * [Tuning for Performance](/warpstream/kafka/configure-kafka-client/tuning-for-performance) * [Monitoring Consumer Groups](/warpstream/agent-setup/monitor-the-warpstream-agents/monitoring-consumer-groups) * [Apache Kafka KIP-848](https://cwiki.apache.org/confluence/display/KAFKA/KIP-848%3A+The+Next+Generation+of+the+Consumer+Rebalance+Protocol)