[go: up one dir, main page]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: make kafka producer usable in cluster setup #3189

Merged
merged 1 commit into from
Jan 24, 2021
Merged

fix: make kafka producer usable in cluster setup #3189

merged 1 commit into from
Jan 24, 2021

Conversation

Reasno
Copy link
Member
@Reasno Reasno commented Jan 23, 2021 •
edited
Loading

简单解释一下这次变更；

  1. 在多副本集群环境下，生产消息必须发送给分区的leader，而原API要求用户手写broker，否则就随机，那么当随机到的broker不是leader是就会报错。修改为：无需用户指定broker，自动根据分区和meta信息推断。作为一个生产者，如何确定正确的brokerId？ swoole/phpkafka#15
  2. 原API发送信息时默认指定分区0，我认为不妥，修改为默认按照key或value取模。
  3. 原API不能当做单例使用，在server环境下每次使用都需要重新拉取meta信息，消耗较大。修改为单例模式。
  4. 原SwooleSocket会在关闭连接时抛出致命错误，这里fork并处理了一下。socket关闭时程序异常退出。 swoole/phpkafka#16

@limingxinleo limingxinleo merged commit d8a4b1f into hyperf:master Jan 24, 2021
@Reasno Reasno deleted the kafka branch January 24, 2021 10:34
@Yurunsoft
Copy link
Contributor

现在 longyan/phpkafka 已经支持了多 broker 下生产和消费，并且内置支持了分区策略。

参数可以看这边：https://github.com/longyan/phpkafka/blob/master/doc/producer.md

默认分区策略：

如果指定了分区，则使用指定的分区；

如果没有指定分区，但指定了 key，会根据 key 的哈希值（crc32）选择分区；

如果没有指定分区，也没有指定 key，会使用轮询策略。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@limingxinleo limingxinleo limingxinleo approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@Reasno @Yurunsoft @limingxinleo