8000 GenerateToolsSettingsFile causes incremental build issues · Issue #17454 · dotnet/sdk · GitHub
[go: up one dir, main page]
Skip to content

GenerateToolsSettingsFile causes incremental build issues #17454 8000

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
alexrp opened this issue May 5, 2021 · 8 comments · Fixed by #47788
Closed

GenerateToolsSettingsFile causes incremental build issues #17454

alexrp opened this issue May 5, 2021 · 8 comments · Fixed by #47788
Assignees
@dsplaisted
Milestone
10.0.1xx

Comments

@alexrp
Copy link
alexrp commented May 5, 2021

The GenerateToolsSettingsFile task unconditionally writes DotnetToolSettings.xml on every build. This causes issues with incremental builds; specifically, dotnet pack will always generate a fresh package because the GenerateNuspec target considers this file as one of its inputs.

I'm not sure what the proper fix is here. Maybe GenerateToolsSettingsFile should compare the generated document with the existing file and only write the document if it's different?
@ghost ghost added the untriaged Request triage from a team member label May 5, 2021
@ghost
Copy link
ghost commented May 5, 2021

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

@marcpopMSFT marcpopMSFT self-assigned this May 5, 2021
@marcpopMSFT marcpopMSFT removed the untriaged Request triage from a team member label May 10, 2021
@marcpopMSFT marcpopMSFT removed their assignment May 10, 2021
@marcpopMSFT marcpopMSFT added this to the Backlog milestone May 10, 2021
@KalleOlaviNiemitalo KalleOlaviNiemitalo mentioned this issue Jul 18, 2022
Closed
@Forgind Forgind mentioned this issue Jan 30, 2024
Merged
ViktorHofer pushed a commit that referenced this issue May 7, 2024
@NikolaMilosavljevic @MichaelSimons
Source-build symbols repackaging (#17454)
447d4a3 
Co-authored-by: Michael Simons <msimons@microsoft.com>
@martincostello
Copy link
Member

I'm running into this quite frequently now since updating to the .NET 9.0.200 SDK but only on Windows (this build is up to 16 attempts now).

@baronfel
Copy link
Member

As of #32798 this target should be properly incremental - however as @martincostello is finding it's generating the tool command xml file per-TFM, while that file is generated to a TFM-oblivious location. We either need to do the safe thing and generate separate, per-TFM XML files or figure out another mechanism to ensure that the Target's only called once (or at least called with enough staggering so that projects building on different nodes can see that the cache has been populated.

Image

@baronfel
Copy link
Member

@martincostello consider instead of multi-targeting your tool, setting RollFoward of Major or LatestMajor so that your tool can run on 9.x runtimes without requiring you to multitarget.

@martincostello
Copy link
Member

For now I've just done this: <BuildInParallel Condition="$([MSBuild]::IsOSPlatform('Windows'))">false</BuildInParallel>

@marcpopMSFT marcpopMSFT modified the milestones: Backlog, 10.0.1xx Mar 11, 2025
@marcpopMSFT
Copy link
Member

Triage: since this seems to be happening more frequently, let's take a look at putting these in files in subfolders during the build as they get packed in TFM specific folders so that should work.

@marcpopMSFT marcpopMSFT marked this as a duplicate of #47269 Mar 11, 2025
@dsplaisted dsplaisted mentioned this issue Mar 21, 2025
Closed
@dsplaisted
Copy link
Member

@dotnet/dotnet-cli Looks like this is starting to fail in our build pipelines somewhat frequently. #47696 was recently filed for this, and I just hit a failure for this too: https://dev.azure.com/dnceng-public/public/_build/results?buildId=986435&view=ms.vss-test-web.build-test-results-tab&runId=26317906&resultId=103428&paneView=debug

So we should probably prioritize fixing this.

And now I realize it's assigned to me :-)

@dsplaisted dsplaisted mentioned this issue Mar 21, 2025
Merged
@andrewlock
Copy link

We're hitting this frequently now too - will the fix be backported to the 9.x.x SDK do you know? Thanks

@andrewlock andrewlock mentioned this issue Apr 29, 2025
Merged
andrewlock added a commit to DataDog/dd-trace-dotnet that referenced this issue Apr 29, 2025
@andrewlock
Try to fix flake in building runner tool (#6905)
92c06c2 
## Summary of changes

Try to fix the flake we're been seeing in the `dotnet pack` stack of the
runner build

## Reason for change

Seeing flake like this, which [seems to be happening in `9.0.2xx` of the
SDK](dotnet/sdk#17454)

```
The "GenerateToolsSettingsFile" task failed unexpectedly.
System.IO.IOException: The process cannot access the file 'D:\a\1\s\tracer\src\Datadog.Trace.Tools.Runner\obj\DotnetToolSettings.xml' because it is being used by another process.
   at Microsoft.Win32.SafeHandles.SafeFileHandle.CreateFile(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, Boolean useAsync)
   at System.Xml.XmlWriterSettings.CreateWriter(String outputFileName)
   at System.Xml.Linq.XDocument.Save(String fileName, SaveOptions options)
   at Microsoft.NET.Build.Tasks.TaskBase.Execute()
   at Microsoft.Build.BackEnd.TaskExecutionHost.Execute()
   at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(TaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask)
```

## Implementation details

Try setting `BuildInParallel=false` [as suggested in this
issue](dotnet/sdk#17454 (comment))

## Test coverage

If this build works, we're ok

## Other details

The issue appears to be fixed in the .NET 10 SDK:
- dotnet/sdk#17454 (comment)

I've asked to see if it will backported, but no answer yet 🙄
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dsplaisted dsplaisted

Labels
None yet
Projects
None yet
Milestone
10.0.1xx
Development

Successfully merging a pull request may close this issue.

Fix path to tool settings file in build dsplaisted/sdk
6 participants
@alexrp @dsplaisted @baronfel @martincostello @marcpopMSFT @andrewlock
0