We cache offset commits along with their commit record offset.
Group contains the following metadata:
Group contains the following metadata:
Membership metadata:
State metadata:
Case class used to represent group metadata for the ListGroups API
Member metadata contains the following metadata:
Member metadata contains the following metadata:
Heartbeat metadata: 1. negotiated heartbeat session timeout 2. timestamp of the latest heartbeat
Protocol metadata: 1. the list of supported protocols (ordered by preference) 2. the metadata associated with each protocol
In addition, it also contains the following state information:
1. Awaiting rebalance callback: when the group is in the prepare-rebalance state, its rebalance callback will be kept in the metadata if the member has sent the join group request 2. Awaiting sync callback: when the group is in the awaiting-sync state, its sync callback is kept in metadata until the leader provides the group assignment and the group transitions to stable
Messages stored for the group topic has versions for both the key and value fields.
Messages stored for the group topic has versions for both the key and value fields. Key version is used to indicate the type of the message (also to differentiate different types of messages from being compacted together if they have the same field values); and value version is used to evolve the messages within their data types:
key version 0: group consumption offset -> value version 0: [offset, metadata, timestamp]
key version 1: group consumption offset -> value version 1: [offset, metadata, commit_timestamp, expire_timestamp]
key version 2: group metadata -> value version 0: [protocol_type, generation, protocol, leader, members]
We cache offset commits along with their commit record offset. This enables us to ensure that the latest offset commit is always materialized when we have a mix of transactional and regular offset commits. Without preserving information of the commit record offset, compaction of the offsets topic it self may result in the wrong offset commit being materialized.