10000 Enable VMR validation in repo PRs by NikolaMilosavljevic · Pull Request #15840 · dotnet/arcade · GitHub
[go: up one dir, main page]

Skip to content

Enable VMR validation in repo PRs #15840

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
May 16, 2025

Conversation

NikolaMilosavljevic
Copy link
Member

Contributes to: dotnet/arcade-services#4539

This PR brings necessary pipeline templates and associated scripts, for updating and building VMR as part of repo PR verification.

  • vmr-sync scripts use the Darc tool, which required a tools manifest file in .config. This will need to be added to any repo that would want to onboard this experience. I could not find a different way to enable this, and it just follows what we had in sdk before.
  • Content of vmr-sync scripts is the same as in https://github.com/dotnet/sdk/pull/48351/files#diff-05db8cf030b53040062f45ed99f7c8b96efdc35c5e90bc52e03db5fd14c472e8
  • This PR does not enable the update of additional repos, i.e. NuGet for sdk repo. That will be implemented in a follow up PR in sdk repo, so that it can be properly tested first. There is some new code, in vmr-sync for doing the additional syncs, but I believe it's untested or would require further tweaks.
  • There is a draft PR in deployment-tools with this changes, to exercise the functionality: [DRAFT] Vmr verification in repo PRs deployment-tools#460

@NikolaMilosavljevic NikolaMilosavljevic requested review from a team May 15, 2025 20:46
Copy link
Member
@MichaelSimons MichaelSimons left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to address some of my comments in follow-ups.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name of this template doesn't resonate with me. "sync" instead of pull updates makes more sense to me. The steps all use the sync terminology.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Easy to fix - I agree with proposal. I did, however, use the original template (and its name) from sdk repo, with some updated steps.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any preference: vmr-sync.yml or vmr-sync-updates.yml?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

vmr-sync.yml

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, for this change I'd need to update the VMR first.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VMR change needs to be merged first: dotnet/dotnet#594

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with ba6ab36

clean: true

- checkout: self
displayName: Clone dotnet/<repo>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to inject the actual repo name here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might be able to - let me check.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with 4333cd4

displayName: Label PR commit
workingDirectory: $(Agent.BuildDirectory)/repo

- script: |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thoughts on using template conditions to avoid the platform steps from appearing? It would simplify the steps displayed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's better. I have followed the original implementation of several steps (the sync ones).

I'd still want to keep the unique names of those steps, i.e. keep the platform, Unix, Windows.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with 4333cd4

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, this doesn't seem to work right. It always uses Unix steps - Agent.Os variable is supposedly evaluated early and reused in all templates. I need to revert this change.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've reverted this change to use runtime step conditions instead of early template conditions - e96f18c

- script: |
vmr_sha=$(grep -oP '(?<=Sha=")[^"]*' $(Agent.BuildDirectory)/repo/eng/Version.Details.xml)
echo "##vso[task.setvariable variable=vmr_sha]$vmr_sha"
displayName: Obtain the vmr sha from Version.Details.xml (Linux)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This step used "(Linux)" while others use "(Unix)". They should be consistent.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might actually be better to use (Unix) which is the name used in old (existing) steps. I used (Linux) for new steps, but that is not completely correct, as those steps would be used on MacOS as well.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with 4333cd4

endpoint: dotnet

stages:
- template: /eng/pipelines/templates/stages/vmr-build.yml@vmr
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this template should be exposing some options to control what is run. Not all repos should need to run all of the lite configuration. Specifically SBRP and SBE do not need to run anything beside the SB legs. Not everyone may want to run stage 2 SB. I see value in some repos just wanting to run a single quick leg.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would need to be done in a follow up. There were plenty of recent changes in VMR build templates that have removed some of the properties used to condition build types, i.e. removal of isSourceOnlyBuild.

@NikolaMilosavljevic
Copy link
Member Author

Created dotnet/dotnet#1057 to track the remaining work.

@NikolaMilosavljevic
Copy link
Member Author
NikolaMilosavljevic commented May 16, 2025

Hmm, checks are failing due to the following, likely in vmr-sync.sh/ps1 scripts I'm adding:

##[error](NETCORE_ENGINEERING_TELEMETRY=Build) One or more eng/common scripts do not use telemetry categorization.

@mmitche before I make changes to add telemetry logging, is this the right way, or should I add the new scripts to the exclusion list, i.e.

$requireTelemetryExcludeFiles = @(

@NikolaMilosavljevic
Copy link
Member Author
NikolaMilosavljevic commented May 16, 2025

I've excluded the vmr-sync scripts from telemetry verification, with 0894fa4

@NikolaMilosavljevic NikolaMilosavljevic merged commit cd02f96 into dotnet:main May 16, 2025
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
0