[code_review] Rank comments based on importance #5019
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
547b883 to
825f4e3
Compare
There was a problem hiding this comment.
Pull Request Overview
This PR introduces a ranking mechanism for code review comments by including an order field to specify comment priority, along with related adjustments in both the evaluation tool and comment model. Key changes include enabling verbose mode in the evaluator for debugging, adding the new_comment_order field in the processed output, and updating the inline comment structure and review prompt instructions to support ranking.
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| scripts/code_review_tool_evaluator.py | Enabled verbose logging and updated diff output to include new_comment_order; changed tool variant string. |
| bugbug/tools/code_review.py | Added the order field to InlineComment and updated review generation instructions to expect a ranking order. |
Comments suppressed due to low confidence (2)
marco-c
reviewed
May 21, 2025
marco-c
reviewed
May 21, 2025
marco-c
reviewed
May 21, 2025
marco-c
reviewed
May 21, 2025
Co-authored-by: Marco Castelluccio <mcastelluccio@mozilla.com>
marco-c
approved these changes
May 26, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #5018
I will report the experiment results here: https://docs.google.com/spreadsheets/d/1CIskQot4TVxHFXxOpXpYMReERuOobqeYQh25bmPHjxQ