Updating FAQ/Notes when seeing rebalancing due to frequent consumer leave group.
This commit is contained in:
Eric Lam (MSFT) 2021-03-25 23:33:53 -07:00 коммит произвёл GitHub
Родитель 0fd45bebf0
Коммит 7b289d61f3
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: 4AEE18F83AFDEB23
1 изменённых файлов: 1 добавлений и 0 удалений

Просмотреть файл

@ -71,3 +71,4 @@ Symptoms | Problem | Solution
----|---|-----
Offset commit failures due to rebalancing | Your consumer is waiting too long in between calls to poll() and the service is kicking the consumer out of the group. | You have several options: <ul><li>increase session timeout</li><li>decrease message batch size to speed up processing</li><li>improve processing parallelization to avoid blocking consumer.poll()</li></ul> Applying some combination of the three is likely wisest.
Network exceptions at high produce throughput | Are you using Java client + default max.request.size? Your requests may be too large. | See Java configs above.
Seeing frequent rebalancing becaues of frequent consumer leave group | Check your client side logs, and you should find the log saying "Member [some member-id] sending LeaveGroup request to coordinator [xyz] due to consumer poll timeout has expired". This means the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time processing messages. | There are serveral settings you can tweak: <ul><li>Either increase max.poll.interval.ms (but then rebalance may take longer)</li><li>or speed up processing by reducing the maximum size of batches returned in poll() with max.poll.records (which may impact performance due to less batching)</li><li>or improve processing parallelization to avoid blocking consumer.poll() for too long</li></ul> Applying some combination of the three is likely necessary to get the best balance for your scenario.