8000 PC invalidates previous work on new run [ci: last-only] by som-snytt · Pull Request #10867 · scala/scala · GitHub
[go: up one dir, main page]

Skip to content

PC invalidates previous work on new run [ci: last-only] #10867

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 3 commits into
from

Conversation

som-snytt
Copy link
Contributor
@som-snytt som-snytt commented Sep 20, 2024

The getDocComment command creates a newTyperRun as of #2387. Then checkForMoreWork notices there is a new run after pollForWork and then will demandNewCompilerRun.

However, a second execution of the test would fail because the unit is deemed up-to-date. This commit updates the minRunId which governs whether the unit is stale when the run is created.

I verified that the test would always fail the second time under Linux but occasionally the first time under Windows.

I don't see a race condition, but the code uses currentTimeMillis instead of nanoTime for timeouts, so maybe it's a Windows clock issue. Or maybe a timing issue (what the PC thread completes before the next ask) causes a different code path. (Two guesses.)

The test comment somewhere says "check you can get a comment one time". The original PR was #2156.

My previous fix was to reset all sources at demandNewCompilerRun, which would likely be heavy-handed.

Probably the change at demandNewCompilerRun can be refined, as it matters just for doc comments.

askParse is tweaked to submit one file at a time, since otherwise the parser callback on the first job will look for work and start the next parse. (In this test, the chain stopped there because the second file was trivial and invoked no callback.)

Fixes scala/scala-dev#840

@scala-jenkins scala-jenkins added this to the 2.13.16 milestone Sep 20, 2024
Copy link
Member
@lrytz lrytz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good / safe to me, I don't fully follow (what are the "parser callbacks"?)

@@ -301,6 +301,7 @@ class Global(settings: Settings, _reporter: Reporter, projectName: String = "")
def isOutOfDate: Boolean = outOfDate

def demandNewCompilerRun() = {
if (!lastWasReload) allSources.foreach(getUnit(_).foreach(reset(_)))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would it be equivalent to add a boolean param to demandNewCompilerRun and set it true at the callsite in reload?

@som-snytt
Copy link
Contributor Author

There is signalParseProgress and then signalDone when a tree is typed (TIL).

@som-snytt
Copy link
Contributor Author

BTW I was curious (or annoyed) about the test; it's not my intention to muck with (or break) the PC. I assumed incorrectly that it would be a mechanism problem that required no special knowledge of PC semantics and operations.

@som-snytt
Copy link
Contributor Author

I was following the "outdated symbols in the IDE" comment to collectImplicitImports which leads to the infamous flaky PC test, i.e., back to here. (I'm considering the extra imported test in isQualifyingImplicit, whereby an import would always be shadowed by an entry in the current scope. That had an exists to combat staleness. I was able to make the stale object test fail on the shadowing PR, but I've forgotten the precious test number, a dir for separate compilation; and I've forgotten how I fixed the broken lookup. Oh I left a code comment, how clever.

 // if lookup fails, it's a stale symbol problem, not our problem (pos/t5639)

which is why that doesn't drop on failed lookup, only on successful lookup of the wrong (shadowing) symbol.

(I was recently reminded to look again at the 2.13.15 PR to warn about "phantom object" from the classpath: the class got a module, so the same module defined in source is hidden; the method of detection is to look in the package owner.)

The test in isQualifyingImiplicit looks benign and correct, except for the staleness caveat. (Not sure it's much more efficient than lookup test.)

The change in collectImplicitImports to use import's qual.tpe (the package) instead of the pre (the package object). Actually the new shadowed check says

8000
            // ii.pre or skipPackageObject
            val pre = if (ii.pre.typeSymbol.isPackageObjectClass) ii.pre.typeSymbol.owner.module.info else ii.pre

@som-snytt
Copy link
Contributor Author
som-snytt commented May 21, 2025

I changed that counter to 1000 and ran ok, then back to 2 and it

reload: Base.scala, Derived.scala
Unexpected foo method (method foo) comment: None
Unexpected Derived class comment:None
Unexpected Derived class comment:None

for

!!  7 - presentation/dock                         [output differs][duration 5.99s]

That is the infamous flaky half of the doc test, transplanted to dock.

Edit: just noticed that 1000 is not a stress test, it's a retry! I flipped the boolean so it runs to failure, so I can actually test it.

I'm pretty sure Homer Simpson works with booleans at the power plant. He may even be a "boolean specialist".

@som-snytt
Copy link
Contributor Author

I think the test is just taunting me, trying to draw me out into the open ocean, like a scene from Moby Dick. Maybe there is a reason I called the second test dock, where it is dry and safe.

@som-snytt som-snytt changed the title PC invalidates previous work on new run PC invalidates previous work on new run [ci: last-only] May 22, 2025
@som-snytt som-snytt force-pushed the sd/840-flaky-presentation-doc branch from 11a33e4 to 4bbd250 Compare May 22, 2025 00:50
@som-snytt
Copy link
Contributor Author

Before and after log for the "sequential parsing" change.

[doctest] new typer run
[doctest] added unit for dock/src/p/Base.scala
[doctest] added unit for dock/src/p/Derived.scala
[doctest] responded, delay = 8ms
[doctest] done with work item: reload List(Base.scala, Derived.scala)
[doctest] quitting work item: reload List(Base.scala, Derived.scala), delay = 0ms
[doctest][Starting new presentation compiler type checking pass]
[doctest] parsing: Base.scala
[doctest] some work at node 2 current = 2
[doctest] picked up work item at RangePosition(dock/src/p/Base.scala, 131, 135, 146): getParsedEntered Base.scala, keepLoaded = true, delay = 16ms
[doctest] parsing: Base.scala
[doctest] some work at node 3 current = 3
[doctest] picked up work item at RangePosition(dock/src/p/Base.scala, 131, 135, 146): getParsedEntered Derived.scala, keepLoaded = true, delay = 2ms
[doctest] parsing: Derived.scala
[doctest] responded, delay = 18ms
[doctest] done with work item: getParsedEntered Derived.scala, keepLoaded = true
[doctest] quitting work item: getParsedEntered Derived.scala, keepLoaded = true, delay = 0ms
[doctest] responded, delay = 2ms
[doctest] done with work item: getParsedEntered Base.scala, keepLoaded = true
[doctest] quitting work item: getParsedEntered Base.scala, keepLoaded = true, delay = 0ms
[doctest] type checking: Base.scala
[doctest] Typer signal done, checking for more work...


[doctest][reload: List(Base.scala, Derived.scala)]
[doctest] new typer run
[doctest] added unit for dock/src/p/Base.scala
[doctest] added unit for dock/src/p/Derived.scala
[doctest] responded, delay = 4ms
[doctest] done with work item: reload List(Base.scala, Derived.scala)
[doctest] quitting work item: reload List(Base.scala, Derived.scala), delay = 0ms
[doctest][Starting new presentation compiler type checking pass]
[doctest] parsing: Base.scala
[doctest] some work at node 2 current = 2
[doctest] picked up work item at RangePosition(dock/src/p/Base.scala, 131, 135, 146): getParsedEntered Base.scala, keepLoaded = true, delay = 12ms
[doctest] parsing: Base.scala
[doctest] responded, delay = 12ms
[doctest] done with work item: getParsedEntered Base.scala, keepLoaded = true
[doctest] quitting work item: getParsedEntered Base.scala, keepLoaded = true, delay = 0ms
[doctest] parsing: Derived.scala
[doctest] some work at node 4 current = 4
[doctest] picked up work item at RangePosition(dock/src/p/Derived.scala, 93, 97, 105): getParsedEntered Derived.scala, keepLoaded = true, delay = 2ms
[doctest] parsing: Derived.scala
[doctest] responded, delay = 1ms
[doctest] done with work item: getParsedEntered Derived.scala, keepLoaded = true
[doctest] quitting work item: getParsedEntered Derived.scala, keepLoaded = true, delay = 0ms
[doctest] type checking: Base.scala
[doctest] Typer signal done, checking for more work...

@som-snytt som-snytt marked this pull request as ready for review May 22, 2025 03:17
@som-snytt
Copy link
Contributor Author
som-snytt commented May 22, 2025

Never mind. 10K rep did not complete on Windows 10

reload: Base.scala, Derived.scala
reload: Base.scala, Derived.scala
Missing foo method (method foo) comment
done at 1214

I'll check the other two ideas (brittle wait and not using nanoTime).

@som-snytt som-snytt marked this pull request as draft May 22, 2025 05:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

The most infamous flaky test
4 participants
0