8000 Microsoft.DotNet.Watch.UnitTests.FileWatcherTests.DeleteSubfolder has flaky test assertion · Issue #49233 · dotnet/sdk · GitHub
[go: up one dir, main page]

Skip to content
Microsoft.DotNet.Watch.UnitTests.FileWatcherTests.DeleteSubfolder has flaky test assertion #49233
@ericstj

Description

@ericstj

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=1056990
Build error leg or test failing: dotnet-watch.Tests.dll.3.WorkItemExecution
Pull request: #49092

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "Failed Microsoft.DotNet.Watch.UnitTests.FileWatcherTests.DeleteSubfolder",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Sample failing log: https://helixr1107v0xdeko0k025g8.blob.core.windows.net/dotnet-sdk-refs-pull-49092-merge-89284afdacb94b978c/dotnet-watch.Tests.dll.3/3/console.3de535af.log?helixlogtype=result

  Failed Microsoft.DotNet.Watch.UnitTests.FileWatcherTests.DeleteSubfolder(usePolling: True) [1 s]
  Error Message:
   Expected:
ChangedPath { Path = /tmp/dotnetSdkTests.igDIQiVU/True/subdir, Kind = Delete },
ChangedPath { Path = /tmp/dotnetSdkTests.igDIQiVU/True/subdir/foo1, Kind = Delete },
ChangedPath { Path = /tmp/dotnetSdkTests.igDIQiVU/True/subdir/foo2, Kind = Delete },
ChangedPath { Path = /tmp/dotnetSdkTests.igDIQiVU/True/subdir/foo3, Kind = Delete }
Actual:
ChangedPath { Path = /tmp/dotnetSdkTests.igDIQiVU/True/subdir, Kind = Update },
ChangedPath { Path = /tmp/dotnetSdkTests.igDIQiVU/True/subdir, Kind = Delete },
ChangedPath { Path = /tmp/dotnetSdkTests.igDIQiVU/True/subdir/foo1, Kind = Delete },
ChangedPath { Path = /tmp/dotnetSdkTests.igDIQiVU/True/subdir/foo2, Kind = Delete }

  Stack Trace:
     at Microsoft.DotNet.Watch.UnitTests.AssertEx.Fail(String message) in /_/test/dotnet-watch.Tests/Utilities/AssertEx.cs:line 214
   at Microsoft.DotNet.Watch.UnitTests.AssertEx.SequenceEqual[T](IEnumerable`1 expected, IEnumerable`1 actual, IEqualityComparer`1 comparer, String message, String itemSeparator, Func`2 itemInspector) in /_/test/dotnet-watch.Tests/Utilities/AssertEx.cs:line 169
   at Microsoft.DotNet.Watch.UnitTests.FileWatcherTests.TestOperation(String dir, ChangedPath[] expectedChanges, Boolean usePolling, Action operation) in /_/test/dotnet-watch.Tests/FileWatcherTests.cs:line 63
   at Microsoft.DotNet.Watch.UnitTests.FileWatcherTests.DeleteSubfolder(Boolean usePolling) in /_/test/dotnet-watch.Tests/FileWatcherTests.cs:line 353
--- End of stack trace from previous location ---
  Standard Output Messages:

Have a look at the assertion the test is making --

await TestOperation(
dir,
expectedChanges: usePolling ?
[
new(subdir, ChangeKind.Delete),
new(f1, ChangeKind.Delete),
new(f2, ChangeKind.Delete),
new(f3, ChangeKind.Delete),
]
: RuntimeInformation.IsOSPlatform(OSPlatform.OSX) ?
[
new(subdir, ChangeKind.Add),
new(subdir, ChangeKind.Delete),
new(f1, ChangeKind.Update),
new(f1, ChangeKind.Add),
new(f1, ChangeKind.Delete),
new(f2, ChangeKind.Update),
new(f2, ChangeKind.Add),
new(f2, ChangeKind.Delete),
new(f3, ChangeKind.Update),
new(f3, ChangeKind.Add),
new(f3, ChangeKind.Delete),
]
: RuntimeInformation.IsOSPlatform(OSPlatform.Windows) ?
[
new(subdir, ChangeKind.Update),
new(subdir, ChangeKind.Delete),
new(f1, ChangeKind.Delete),
new(f2, ChangeKind.Delete),
new(f3, ChangeKind.Delete),
]
:
[
new(subdir, ChangeKind.Delete),
new(f1, ChangeKind.Delete),
new(f2, ChangeKind.Delete),
new(f3, ChangeKind.Delete),
],

It's asserting it will see a series of events exactly on different operating systems. FileSystemWatcher (and file notification APIs in the platform for that matter) don't garuntee that. You can't count on seeing all changed/updated events. Especially in the case of polling you cannot garuntee you'll see an update event if it's followed by a delete since a polling implementation can only see a file or directory has been removed, and not see any changes that might have happened between the last poll and removal.

This test should be changed to just assert on seeing a minimum set of events, not exact set. In other words - a contains check rather than equals.

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=1056990
Error message validated: [Failed Microsoft.DotNet.Watch.UnitTests.FileWatcherTests.DeleteSubfolder]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/2/2025 8:08:17 PM UTC

Report

Build Definition Test Pull Request
1091826 dotnet/sdk dotnet-watch.Tests.dll.3.WorkItemExecution #49739
1089135 dotnet/sdk dotnet-watch.Tests.dll.3.WorkItemExecution #49589
1088888 dotnet/sdk dotnet-watch.Tests.dll.3.WorkItemExecution #49589
1088082 dotnet/sdk dotnet-watch.Tests.dll.3.WorkItemExecution #49503
1081242 dotnet/sdk dotnet-watch.Tests.dll.3.WorkItemExecution #49596

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 0 5

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      0