8000 MNT Fixes doc building on PR by thomasjpfan · Pull Request #23508 · scikit-learn/scikit-learn · GitHub
[go: up one dir, main page]

Skip to content

MNT Fixes doc building on PR #23508

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Jun 3, 2022

Conversation

thomasjpfan
Copy link
Member
@thomasjpfan thomasjpfan commented Jun 1, 2022

Fixes #23509

This PR is testing out that doc building works as expected on a PR.

Doc building has been moved to GitHub actions: #21137

@thomasjpfan
Copy link
Member Author
thomasjpfan commented Jun 1, 2022

@alfaro96 From the CI run, it looks like the PULL_REQUEST_NUMBER is not set.

@thomasjpfan
Copy link
Member Author
thomasjpfan commented Jun 1, 2022

On the CI on main, the circleci upload returns "Project not found".

Edit: I fixed it. It turns out, circleci v2 API requires a personal token.
XREF: https://discuss.circleci.com/t/triggering-pipeline-via-v2-api-fails-with-404-project-not-found/39342

@thomasjpfan
Copy link
Member Author
thomasjpfan commented Jun 1, 2022

Looking at how gh run view 2418955303 returns the PR number:

test_circleci_doc_building Documentation builder #23508 · 2418955303
Triggered via pull_request about 1 hour ago

It uses a graphql query based on the HeadBranch. For example, this PR has test_circleci_doc_building as the head branch. Because multiple PRs can have the same branch name, gh run view also filters the PRs by it's owner.

Moving forward

We get this to work with PyGitHub:

import os
from github import Github

gh = Github(os.environ["GITHUB_TOKEN"])

sk_repo = gh.get_repo("scikit-learn/scikit-learn")

run = sk_repo.get_workflow_run(2418955303)

head = f"{run.head_repository.owner.login}:{run.head_branch}"

prs = list(sk_repo.get_pulls(state='open', sort='created', head=head))

pr = prs[0]

pr.number
# 23508

@thomasjpfan thomasjpfan changed the title [NOMRG] Test out doc building on PR WIP Test out doc building on PR Jun 1, 2022
@thomasjpfan
Copy link
Member Author

Based on https://github.com/scikit-learn/scikit-learn/runs/6683078988?check_suite_focus=true I think this needs to be merged to test that it works.

@alfaro96
Copy link
Member
alfaro96 commented Jun 1, 2022

@alfaro96 From the CI run, it looks like the PULL_REQUEST_NUMBER is not set.

I am curious why the PULL_REQUEST_NUMBER was set in my pull request but not in this one. Do you have any idea about that?

@alfaro96
Copy link
Member
alfaro96 commented Jun 1, 2022

Based on https://github.com/scikit-learn/scikit-learn/runs/6683078988?check_suite_focus=true I think this needs to be merged to test that it works.

One of the main drawbacks of the workflow_run event is that the corresponding workflow is always taken from the main branch. I suppose that is because it has access to secrets, and we do not want someone to change the behavior in pull requests. Nevertheless, I think that we are not going to change the trigger-hosting.yml file so often, and it is a big deal then.

@alfaro96
Copy link
Member
alfaro96 commented Jun 1, 2022

Let me try the solution proposed in my main branch to see if it is working.

Copy link
Member
@alfaro96 alfaro96 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @thomasjpfan for solving this issue. I have tried it in my own repository and is working so, LGTM.

Minor comments.


gh = Github(args.token)
sk_repo = gh.get_repo(args.repo)
run = sk_repo.get_workflow_run(args.run_id)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because they are asserting that id is of integer data type.

Suggested change
run = sk_repo.get_workflow_run(args.run_id)
run = sk_repo.get_workflow_run(int(args.run_id))

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the type information in 568b971 (#23508)

@@ -0,0 +1,17 @@
import argparse
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this file supposed to be in the maint_tools folder in the build_tools/github one?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved the file to build_tools/github.

@lesteve
Copy link
< 8000 /div>
Member
lesteve commented Jun 1, 2022

Just curious, what is the plan for the doc artifact redirector CI status, to be able to check the rendered doc in one click?

It seems you need a CircleCI job name for .github/workflows/artifact-redirector.yml to work, will the triggered CircleCI job (the one downloading the doc from the GHA artifact) be part of the visible CI statuses of the PR?

@thomasjpfan
Copy link
Member Author

Just curious, what is the plan for the doc artifact redirector CI status, to be able to check the rendered doc in one click?

@lesteve Looking at the PR in @alfaro96's fork: alfaro96#1 the redirector works without any changes. From the redirector point of view everything is the same.

will the triggered CircleCI job (the one downloading the doc from the GHA artifact) be part of the visible CI statuses of the PR?

Looking at the fork, the CircleCI jobs appear after it is triggered by GHA.

@thomasjpfan thomasjpfan changed the title WIP Test out doc building on PR MNT Test out doc building on PR Jun 1, 2022
@alfaro96
Copy link
Member
alfaro96 commented Jun 1, 2022

Just curious, what is the plan for the doc artifact redirector CI status, to be able to check the rendered doc in one click?

@lesteve Looking at the PR in @alfaro96's fork: alfaro96#1 the redirector works without any changes. From the redirector point of view everything is the same.

will the triggered CircleCI job (the one downloading the doc from the GHA artifact) be part of the visible CI statuses of the PR?

Looking at the fork, the CircleCI jobs appear after it is triggered by GHA.

That is. We trigger the same jobs in CircleCI as before (doc and doc-min-dependencies), although now these jobs download the documentation from the corresponding artifact instead of building it.

@alfaro96
Copy link
Member
alfaro96 commented Jun 1, 2022

BTW, the documentation is correctly pushed to the scikit-learn.github.io repository.

@lesteve
Copy link
Member
lesteve commented Jun 1, 2022

OK great!

@thomasjpfan thomasjpfan marked this pull request as draft June 1, 2022 14:35
@thomasjpfan thomasjpfan marked this pull request as ready for review June 1, 2022 14:45
@thomasjpfan thomasjpfan changed the title MNT Test out doc building on PR MNT Fixes doc building on PR Jun 1, 2022
@thomasjpfan
Copy link
Member Author

I updated this PR with a more simple curl command to get the PR number. I tested it out here: https://github.com/thomasjpfan/scikit-learn/runs/6692461745?check_suite_focus=true and it triggers circleci correctly.

@thomasjpfan
Copy link
Member Author

You can also see the CircleCI build running on my fork's PR.

Copy link
Member
@alfaro96 alfaro96 left a comment
9E88

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again @thomasjpfan; I prefer this more straightforward method. I confirm the URL that is supposed to be used for this pull request contains the necessary information.

LGTM after checking the suggested changes.

@@ -0,0 +1,22 @@
"""Gets the Pull Request Number from a workflow's run ID.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is not required anymore, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, this file is not needed anymore. I removed it from the PR.

@thomasjpfan
Copy link
Member Author

@glemaitre @jeremiedbb @lesteve This PR is required so that circleci in PRs works correctly. I tested it out in thomasjpfan#111 and it should work.

@lesteve
Copy link
Member
lesteve commented Jun 3, 2022

Merging this one, thanks!

I don't really see why PULL_REQUEST_NUMBER is not set, as already asked in #23508 (comment) but oh well ...

@lesteve lesteve merged commit fb3ed90 into scikit-learn:main Jun 3, 2022
@lesteve
Copy link
Member
lesteve commented Jun 3, 2022

I don't really see why PULL_REQUEST_NUMBER is not set, as already asked in #23508 (comment) but oh well ...

This is probably this: https://github.community/t/pull-request-attribute-empty-in-workflow-run-event-object-for-pr-from-forked-repo/154682

  • If the PR is from a local branch inside the repo, the pull_request attribute of the github.event context is populated
  • -If the PR is from a forked branch outside the repo, the pull_request attribute of the github.event context is empty

This would explain why it was working when @alfaro96 was testing this on its fork.

The work-around they found seems a bit more complicated than what is implemented in this PR:
https://github.community/t/pull-request-attribute-empty-in-workflow-run-event-object-for-pr-from-forked-repo/154682

@lesteve lesteve mentioned this pull request Jun 3, 2022
ogrisel pushed a commit to ogrisel/scikit-learn that referenced this pull request Jul 11, 2022
glemaitre pushed a commit to glemaitre/scikit-learn that referenced this pull request Aug 4, 2022
glemaitre pushed a commit that referenced this pull request Aug 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Doc building on PRs do not push to circleci yet
3 participants
0