E528 fix(kafka_producer): send messages to wolff producer to buffer even when connector is in `connecting` state (sync query mode) by thalesmg · Pull Request #11722 · emqx/emqx · GitHub
[go: up one dir, main page]

Skip to content

Conversation

@thalesmg
Copy link
Contributor
@thalesmg thalesmg commented Oct 6, 2023

Fixes https://emqx.atlassian.net/browse/EMQX-11085

Messages would not be sent to wolff if the connection was down, so they were effectively lost.

Summary

🤖 Generated by Copilot at ea170df

Added a new module emqx_bridge_kafka_impl_producer to implement the Kafka bridge producer logic using the emqx_resource behaviour. Improved the query handling and resource information in the emqx_resource_buffer_worker module. Bumped the version of the emqx_resource application and renamed a variable for clarity.

PR Checklist

Please convert it to a draft if any of the following conditions are not met. Reviewers may skip over until all the items are checked:

  • Added tests for the changes
  • Added property-based tests for code which performs user input validation
  • Changed lines covered in coverage report
  • Change log has been added to changes/(ce|ee)/(feat|perf|fix)-<PR-id>.en.md files
  • For internal contributor: there is a jira ticket to track this change
  • Created PR to emqx-docs if documentation update is required, or link to a follow-up jira ticket
  • Schema changes are backward compatible

Checklist for CI (.github/workflows) changes

  • If changed package build workflow, pass this action (manual trigger)
  • Change log has been added to changes/ dir for user-facing artifacts update

…hen connector is in `connecting` state

Fixes https://emqx.atlassian.net/browse/EMQX-11085

Messages would not be sent to wolff if the connection was down, so they were effectively lost.
@thalesmg thalesmg force-pushed the fix-bridge-buffer-supported-m-20231006 branch from ea170df to 34186fc Compare October 6, 2023 14:43
@thalesmg thalesmg marked this pull request as ready for review October 6, 2023 16:14
@thalesmg thalesmg requested a review from a team as a code owner October 6, 2023 16:14
@coveralls
Copy link
Collaborator

Pull Request Test Coverage Report for Build 6433089882

  • 1 of 1 (100.0%) changed or added relevant line in 1 file are covered.
  • 17 unchanged lines in 7 files lost coverage.
  • Overall coverage increased (+0.01%) to 82.74%

Files with Coverage Reduction New Missed Lines %
apps/emqx_connector/src/emqx_connector_jwt_worker.erl 1 89.47%
apps/emqx_ft/src/emqx_ft_storage_fs.erl 1 82.09%
apps/emqx_gateway_exproto/src/emqx_exproto_gcli.erl 1 64.15%
apps/emqx/src/emqx_banned.erl 1 89.86%
apps/emqx/src/emqx_connection.erl 1 83.42%
apps/emqx/src/emqx_os_mon.erl 1 91.78%
apps/emqx/src/emqx_reason_codes.erl 11 89.71%
Totals Coverage Status
Change from base Build 6430958153: 0.01%
Covered Lines: 33786
Relevant Lines: 40834

💛 - Coveralls

@thalesmg thalesmg changed the title fix(kafka_producer): send messages to wolff producer to buffer even when connector is in connecting state fix(kafka_producer): send messages to wolff producer to buffer even when connector is in connecting state (sync query mode) Oct 6, 2023
@thalesmg thalesmg merged commit 2fe6e8e into emqx:master Oct 6, 2023
@thalesmg thalesmg deleted the fix-bridge-buffer-supported-m-20231006 branch October 6, 2023 18:32
end.

do_call_query(QM, Id, Index, Ref, Query, #{is_buffer_supported := true} = QueryOpts, Resource) ->
do_call_query(QM, Id, Index, Ref, Query, QueryOpts, #{query_mode := ResQM} = Resource) when
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do the authn authz resources end up calling this path?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, they end up calling this function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here I introduce a new query mode type: #11724
Perhaps it would be more appropriate to restrict this behavior to that more specific type.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I think we would also need to introduce a new query mode for the async version of a buffered connector, to have the correct semantics as well. 🤔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

0