8000 The most infamous flaky test · Issue #840 · scala/scala-dev · GitHub
[go: up one dir, main page]
Skip to content

The most infamous flaky test #840

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
som-snytt opened this issue Apr 24, 2023 · 5 comments · May be fixed by scala/scala#10867
Open

The most infamous flaky test #840

som-snytt opened this issue Apr 24, 2023 8000 · 5 comments · May be fixed by scala/scala#10867

Comments

@som-snytt
Copy link

scala/scala#10105 (comment)

presentation/doc sees Unexpected foo method comment: None.

Personally, I am surprised that foo requires comment.

Parking it here as internal.
@som-snytt
Copy link
Author

scala/scala#10732 (comment)

and

scala/scala#10732 (comment)

@lrytz
Copy link
Member
lrytz commented Aug 20, 2024

i saw it locally 👻

@som-snytt
Copy link
Author
som-snytt commented Aug 22, 2024 •
edited
Loading

Duplicates #72 !

I corrected the title.

@som-snytt som-snytt changed the title The less famous flaky test The most infamous flaky test Aug 22, 2024
@lrytz
Copy link
Member
lrytz commented Aug 23, 2024

Looking at the code, the None probably originates here: https://github.com/scala/scala/blob/2d9e6acce8c4246e6cba5a22ff9d965ec3bd117c/src/interactive/scala/tools/nsc/interactive/Response.scala#L68

response.get is called here https://github.com/scala/scala/blob/53e1571071bf75ce6ddd6e7b4f0b84ef1bdc32f8/test/files/presentation/doc/doc.scala#L65

@som-snytt som-snytt linked a pull request Sep 20, 2024 that will close this issue
Draft
@som-snytt
Copy link
Author
som-snytt commented May 22, 2025 •
edited
Loading

I forgot to investigate the suggestion from August. (Also at this comment #72 (comment))

This test completes on Windows 10, which does the loadSources (of 2 files), one run of the test (2 askDocComment), shutdown, many times.

object Test {
  def main(args: Array[String]): Unit = {
    def run(): Boolean = {
      val it = new ITest {
        loadSources()
      }
      val res =
        try it.dorun()
        finally it.askShutdown()
      res
    }
    var n = 10000
    while (n > 0 && run()) {
      n -= 1
    }
    println(s"done at $n")
  }
}

The test was turned off in 2013 scala/scala@8986ee4

then back on in 2014 scala/scala@1122e9e

then back off in 2016 for Windows scala/scala@8e52acd

I see from the OP and a year later that it persisted on Linux and also both getDocComments were susceptible to failure; I had only seen the last fail.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

PC invalidates previous work on new run [ci: last-only] som-snytt/scala
2 participants
@lrytz @som-snytt
0