enh(tmq): optimize update tablelist for topic of stable with filter when alter tag #34417
Conversation
Summary of ChangesHello @wangmm0220, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request primarily focuses on enhancing the efficiency and precision of table list management and filtering, particularly for Time-Series Message Queue (TMQ) operations and when altering table tags. By introducing the ability to filter based on specific tag column IDs, the system can now perform more targeted updates and reduce unnecessary processing, leading to improved performance. Additionally, a correction to the TMQ consumer connection type ensures proper functionality. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request introduces an optimization for updating the table list for topics on stable tables when a tag is altered. The core idea is to pass the column IDs of the altered tags down to the filtering logic, allowing it to skip re-evaluation if the altered tags are not part of the filter conditions. The changes span across several files to plumb this information through. While the optimization logic is sound, I've found a couple of issues related to error handling and correctness in the implementation that need to be addressed.
There was a problem hiding this comment.
Pull request overview
This PR aims to optimize table list updates for TMQ topics with stable (super table) filters when tag values are altered. The optimization passes a list of changed column IDs (cidList) through the call chain to enable more selective filtering, avoiding unnecessary re-evaluation of tag conditions that don't involve the changed columns.
Changes:
- Extended function signatures to pass
cidListparameter through the executor and TMQ subsystems - Refactored tag condition filtering logic in
executil.cto support column-specific filtering optimization - Modified vnode tag alteration handling to construct and pass the list of altered column IDs
- Changed TMQ consumer connection type from
CONN_TYPE__TMQtoCONN_TYPE__QUERY
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| include/libs/executor/executor.h | Added cidList parameter to qSubFilterTableList function signature |
| source/libs/executor/inc/executil.h | Updated doFilterByTagCond signature to accept cidList and removed addUid and listAdded parameters |
| source/libs/executor/src/executor.c | Removed doFilterTableByTagCond wrapper function and updated to call doFilterByTagCond directly with new signature |
| source/libs/executor/src/executil.c | Major refactoring: inverted checkTagCond logic, extracted getTableListInInOperator function, modified tag filtering to check if altered columns are involved |
| source/dnode/vnode/src/vnd/vnodeSvr.c | Added logic to construct cidList from altered tag information and pass it to tqUpdateTbUidList |
| source/dnode/vnode/src/inc/vnodeInt.h | Updated tqUpdateTbUidList function signature |
| source/dnode/vnode/src/tq/tqRead.c | Updated to accept and forward cidList parameter |
| source/dnode/vnode/src/meta/metaQuery.c | Added check to skip tables that already have tag values populated |
| source/client/src/clientTmq.c | Changed connection type from CONN_TYPE__TMQ to CONN_TYPE__QUERY |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…en alter tag
Description
Issue(s)
Checklist
Please check the items in the checklist if applicable.