8000 Overlay: Add manual Java overlay annotations & discard predicates by kaspersv · Pull Request #19813 · github/codeql · GitHub
[go: up one dir, main page]

Skip to content

Overlay: Add manual Java overlay annotations & discard predicates #19813

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: kaspersv/overlay-java-annotations
Choose a base branch
from

Conversation

kaspersv
Copy link
Contributor
@kaspersv kaspersv commented Jun 18, 2025

This PR builds on top of #19779, adding a few additional manual overlay annotations and defining entity discard predicates for Java.

The overlay[local?] annotations in the experimental queries are needed to ensure that virtual dispatch (which depends indirectly on ActiveExperimentalModels) becomes local. In addition to the manual overlay[local?] annotations, the PR adds an overlay[global] annotation in DataFlowImplCommon.qll to ensure lambda flow is global.

The entity discard predicates defined in this PR are not exhaustive and additional discarding may be needed. The PR adds entity discard predicates to discard the following:

  • base locations, expressions, javadocs, statements and local scoped variables in files fully extracted in the overlay
  • base methods on anonymous classes in files fully extracted in the overlay
  • base methods in files fully extracted in the overlay that are not used in the overlay

@kaspersv kaspersv force-pushed the kaspersv/overlay-java-discarding branch from 6f5bc41 to 8672313 Compare June 19, 2025 06:34
@kaspersv kaspersv force-pushed the kaspersv/overlay-java-discarding branch from 8672313 to db52a3c Compare June 19, 2025 07:13
@kaspersv kaspersv force-pushed the kaspersv/overlay-java-discarding branch from db52a3c to 26896ea Compare June 20, 2025 11:39
@kaspersv kaspersv force-pushed the kaspersv/overlay-java-annotations branch from ea40677 to 052023e Compare June 20, 2025 11:59
@kaspersv kaspersv force-pushed the kaspersv/overlay-java-discarding branch 3 times, most recently from bdf1bdd to 3c2c871 Compare June 23, 2025 13:00
@kaspersv kaspersv changed the title Java: Add manual overlay annotations & discard predicates Overlay: Add manual Java overlay annotations & discard predicates Jun 24, 2025
@kaspersv kaspersv force-pushed the kaspersv/overlay-java-annotations branch from 052023e to 81b677a Compare June 24, 2025 08:26
@kaspersv kaspersv force-pushed the kaspersv/overlay-java-discarding branch from 3c2c871 to 0ee6a78 Compare June 24, 2025 08:38
@kaspersv kaspersv marked this pull request as ready for review June 25, 2025 06:03
@kaspersv kaspersv requested a review from a team as a code owner June 25, 2025 06:03
@kaspersv kaspersv requested a review from aschackmull June 25, 2025 06:03
@kaspersv kaspersv added the no-change-note-required This PR does not need a change note label Jun 25, 2025
@kaspersv
Copy link
Contributor Author

The Compile all queries CI check should succeed once the next CLI is released.

Copy link
Contributor
@jbj jbj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy with this if @aschackmull is happy

Copy link
Contributor
@aschackmull aschackmull left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can simplify the boilerplate somewhat.

10000
// numlines is used to restrict attention to fully extracted files and
// ignore skeleton extracted files in the overlay
exists(@locatable l | numlines(l, _, _, _) and file = getRawFile(l))
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about adding the following:

Suggested change
}
}
/**
* A `@locatable` in the base variant that should be discarded if its file is
* extracted in the overlay variant.
*/
overlay[local]
abstract class DiscardableLocatable extends @locatable {
string getRawFileInBase() { not isOverlay() and result = getRawFile(this) }
}
overlay[discard_entity]
private predicate discardLocatable(@locatable el) {
extractedInOverlay(el.(DiscardableLocatable).getRawFileInBase())
}

I think that can reduce a lot of boilerplate.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarly, for things like @method we can use:

/**
 * A `@locatable` in the base variant that should be discarded if its file is
 * extracted in the overlay variant and it is itself not extracted in the
 * overlay, that is, it is deleted in the overlay.
 */
overlay[local]
abstract class DiscardableReferableLocatable extends @locatable {
  string getRawFileInBase() { not isOverlay() and result = getRawFile(this) }

  predicate existsInOverlay() { isOverlay() and exists(this) }
}

overlay[discard_entity]
private predicate discardReferableLocatable(@locatable el) {
  exists(DiscardableReferableLocatable drl | drl = el |
    extractedInOverlay(drl.getRawFileInBase()) and
    not drl.existsInOverlay()
  )
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. I've added the suggested abstraction for locatables in commit 4. I've left the locations case as is, as there is only one discard predicate for locations currently.

Comment on lines 2706 to 2716
overlay[local]
private predicate discardableExpr(string file, @expr e) {
not isOverlay() and
file = getRawFile(e)
}

/** Discard base expressions in files fully extracted in the overlay. */
overlay[discard_entity]
private predicate discardExpr(@expr e) {
exists(string file | discardableExpr(file, e) and extractedInOverlay(file))
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the suggested DiscardableLocatable, this could simply become:

Suggested change
overlay[local]
private predicate discardableExpr(string file, @expr e) {
not isOverlay() and
file = getRawFile(e)
}
/** Discard base expressions in files fully extracted in the overlay. */
overlay[discard_entity]
private predicate discardExpr(@expr e) {
exists(string file | discardableExpr(file, e) and extractedInOverlay(file))
}
overlay[local]
private class DiscardableExpr extends DiscardableLocatable, @expr { }

I'm unsure whether overlay[local] is needed here or whether it's inherited from DiscardableLocatable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overlay[local] is not inherited from DiscardableLocatable, but the overlay[local?] annotation from the top-level module declaration is inherited, so overlay[local] isn't strictly needed, but does clarify the intend.

overlay[local]
private predicate discardableJavadoc(string file, @javadoc d) {
not isOverlay() and
exists(@member m | file = getRawFile(m) and hasJavadoc(m, d))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe @javadoc is a @locatable, so the indirection through hasJavadoc is unnecessary, I think. Also, with the abstract class this would simply be

private class DiscardableJavadoc extends DiscardableLocatable, @javadoc { }

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point; the indirection is unnecessary.

@kaspersv kaspersv requested a review from aschackmull June 25, 2025 14:59
@kaspersv kaspersv force-pushed the kaspersv/overlay-java-discarding branch from 54589c8 to bcadf31 Compare June 25, 2025 15:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
DataFlow Library Java no-change-note-required This PR does not need a change note
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
0