10000 Merge pull request #2014 from github/nickfyson/update-release-process · github/codeql-action@e1d1fad · GitHub
[go: up one dir, main page]
Skip to content

Commit e1d1fad

Browse files
nickfysonnickfyson
authored
Merge pull request #2014 from github/nickfyson/update-release-process
update release process to support multiple version
2 parents 3675be0 + 0e9a210 commit e1d1fad
  • __packaging-inputs-js.yml
  • __remote-config.yml
  • __resolve-environment-action.yml
  • __rubocop-multi-language.yml
  • __ruby.yml
  • __scaling-reserved-ram.yml
  • __split-workflow.yml
  • __submit-sarif-failure.yml
  • __swift-custom-build.yml
  • __test-autobuild-working-dir.yml
  • __test-local-codeql.yml
  • __test-proxy.yml
  • __unset-environment.yml
  • __upload-ref-sha-input.yml
  • __with-checkout-path.yml
  • codeql.yml
  • codescanning-config-cli.yml
  • debug-artifacts-failure.yml
  • debug-artifacts.yml
  • expected-queries-runs.yml
  • post-release-mergeback.yml
  • pr-checks.yml
  • python-deps.yml
  • python312-windows.yml
  • query-filters.yml
  • test-codeql-bundle-all.yml
  • update-release-branch.yml
  • CONTRIBUTING.md
  • pr-checks

    • Some content is hidden

    Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

    56 files changed

    +438
    -111
    lines changed

    ‎.github/actions/release-branches/action.yml

    Lines changed: 25 additions & 0 deletions
    Original file line numberDiff line numberDiff line change
    @@ -0,0 +1,25 @@
    1+
    name: 'Release branches'
    2+
    description: 'Determine branches for release & backport'
    3+
    inputs:
    4+
    major_version:
    5+
    description: 'The version as extracted from the package.json file'
    6+
    required: true
    7+
    latest_tag:
    8+
    description: 'The most recent tag published to the repository'
    9+
    required: true
    10+
    outputs:
    11+
    backport_source_branch:
    12+
    description: "The release branch for the given tag"
    13+
    value: ${{ steps.branches.outputs.backport_source_branch }}
    14+
    backport_target_branches:
    15+
    description: "JSON encoded list of branches to target with backports"
    16+
    value: ${{ steps.branches.outputs.backport_target_branches }}
    17+
    runs:
    18+
    using: "composite"
    19+
    steps:
    20+
    - id: branches
    21+
    run: |
    22+
    python ${{ github.action_path }}/release-branches.py \
    23+
    --major-version ${{ inputs.major_version }} \
    24+
    --latest-tag ${{ inputs.latest_tag }}
    25+
    shell: bash

    ‎.github/actions/release-branches/release-branches.py

    Lines changed: 48 additions & 0 deletions
    Original file line numberDiff line numberDiff line change
    @@ -0,0 +1,48 @@
    1+
    import argparse
    2+
    import json
    3+
    import os
    4+
    import subprocess
    5+
    6+
    # Name of the remote
    7+
    ORIGIN = 'origin'< 10000 /span>
    8+
    9+
    OLDEST_SUPPORTED_MAJOR_VERSION = 2
    10+
    11+
    def main():
    12+
    13+
    parser = argparse.ArgumentParser()
    14+
    parser.add_argument("--major-version", required=True, type=str, help="The major version of the release")
    15+
    parser.add_argument("--latest-tag", required=True, type=str, help="The most recent tag published to the repository")
    16+
    args = parser.parse_args()
    17+
    18+
    major_version = args.major_version
    19+
    latest_tag = args.latest_tag
    20+
    21+
    print("major_version: " + major_version)
    22+
    print("latest_tag: " + latest_tag)
    23+
    24+
    # If this is a primary release, we backport to all supported branches,
    25+
    # so we check whether the major_version taken from the package.json
    26+
    # is greater than or equal to the latest tag pulled from the repo.
    27+
    # For example...
    28+
    # 'v1' >= 'v2' is False # we're operating from an older release branch and should not backport
    29+
    # 'v2' >= 'v2' is True # the normal case where we're updating the current version
    30+
    # 'v3' >= 'v2' is True # in this case we are making the first release of a new major version
    31+
    consider_backports = ( major_version >= latest_tag.split(".")[0] )
    32+
    33+
    with open(os.environ["GITHUB_OUTPUT"], "a") as f:
    34+
    35+
    f.write(f"backport_source_branch=releases/{major_version}\n")
    36+
    37+
    backport_target_branches = []
    38+
    39+
    if consider_backports:
    40+
    for i in range(int(major_version.strip("v"))-1, 0, -1):
    41+
    branch_name = f"releases/v{i}"
    42+
    if i >= OLDEST_SUPPORTED_MAJOR_VERSION:
    43+
    backport_target_branches.append(branch_name)
    44+
    45+
    f.write("backport_target_branches="+json.dumps(backport_target_branches)+"\n")
    46+
    47+
    if __name__ == "__main__":
    48+
    main()

    ‎.github/actions/release-initialise/action.yml

    Lines changed: 33 additions & 0 deletions
    Original file line numberDiff line numberDiff line change
    @@ -0,0 +1,33 @@
    1+
    name: 'Prepare release job'
    2+
    description: 'Prepare for updating a release branch'
    3+
    4+
    runs:
    5+
    using: "composite"
    6+
    steps:
    7+
    8+
    - name: Dump environment
    9+
    run: env
    10+
    shell: bash
    11+
    12+
    - name: Dump GitHub context
    13+
    env:
    14+
    GITHUB_CONTEXT: '${{ toJson(github) }}'
    15+
    run: echo "$GITHUB_CONTEXT"
    16+
    shell: bash
    17+
    18+
    - name: Set up Python
    19+
    uses: actions/setup-python@v4
    20+
    with:
    21+
    python-version: 3.8
    22+
    23+
    - name: Install dependencies
    24+
    run: |
    25+
    python -m pip install --upgrade pip
    26+
    pip install PyGithub==1.55 requests
    27+
    shell: bash
    28+
    29+
    - name: Update git config
    30+
    run: |
    31+
    git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com"
    32+
    git config --global user.name "github-actions[bot]"
    33+
    shell: bash

    ‎.github/update-release-branch.py

    Lines changed: 143 additions & 26 deletions
    Original file line numberDiff line numberDiff line change
    @@ -13,8 +13,9 @@
    1313
    1414
    """
    1515

    16-
    SOURCE_BRANCH = 'main'
    17-
    TARGET_BRANCH = 'releases/v2'
    16+
    # NB: This exact commit message is used to find commits for reverting during backports.
    17+
    # Changing it requires a transition period where both old and new versions are supported.
    18+
    BACKPORT_COMMIT_MESSAGE = 'Update version and changelog for v'
    1819

    1920
    # Name of the remote
    2021
    ORIGIN = 'origin'
    @@ -34,7 +35,9 @@ def branch_exists_on_remote(branch_name):
    3435
    return run_git('ls-remote', '--heads', ORIGIN, branch_name).strip() != ''
    3536

    3637
    # Opens a PR from the given branch to the target branch
    37-
    def open_pr(repo, all_commits, source_branch_short_sha, new_branch_name, conductor):
    38+
    def open_pr(
    39+
    repo, all_commits, source_branch_short_sha, new_branch_name, source_branch, target_branch,
    40+
    conductor, is_primary_release, conflicted_files):
    3841
    # Sort the commits into the pull requests that introduced them,
    3942
    # and any commits that don't have a pull request
    4043
    pull_requests = []
    @@ -56,7 +59,7 @@ def open_pr(repo, all_commits, source_branch_short_sha, new_branch_name, conduct
    5659

    5760
    # Start constructing the body text
    5861
    body = []
    59-
    body.append(f'Merging {source_branch_short_sha} into {TARGET_BRANCH}.')
    62+
    body.append(f'Merging {source_branch_short_sha} into {target_branch}.')
    6063

    6164
    body.append('')
    6265
    body.append(f'Conductor for this PR is @{conductor}.')
    @@ -79,20 +82,38 @@ def open_pr(repo, all_commits, source_branch_short_sha, new_branch_name, conduct
    7982

    8083
    body.append('')
    8184
    body.append('Please do the following:')
    85+
    if len(conflicted_files) > 0:
    86+
    body.append(' - [ ] Ensure `package.json` file contains the correct version.')
    87+
    body.append(' - [ ] Add commits to this branch to resolve the merge conflicts ' +
    88+
    'in the following files:')
    89+
    body.extend([f' - [ ] `{file}`' for file in conflicted_files])
    90+
    body.append(' - [ ] Ensure another maintainer has reviewed the additional commits you added to this ' +
    91+
    'branch to resolve the merge conflicts.')
    8292
    body.append(' - [ ] Ensure the CHANGELOG displays the correct version and date.')
    8393
    body.append(' - [ ] Ensure the CHANGELOG includes all relevant, user-facing changes since the last release.')
    84-
    body.append(f' - [ ] Check that there are not any unexpected commits being merged into the {TARGET_BRANCH} branch.')
    94+
    body.append(f' - [ ] Check that there are not any unexpected commits being merged into the {target_branch} branch.')
    8595
    body.append(' - [ ] Ensure the docs team is aware of any documentation changes that need to be released.')
    96+
    97+
    if not is_primary_release:
    98+
    body.append(' - [ ] Remove and re-add the "Update dependencies" label to the PR to trigger just this workflow.')
    99+
    body.append(' - [ ] Wait for the "Update dependencies" workflow to push a commit updating the dependencies.')
    100+
    101+
    body.append(' - [ ] Mark the PR as ready for review to trigger the full set of PR checks.')
    86102
    body.append(' - [ ] Approve and merge this PR. Make sure `Create a merge commit` is selected rather than `Squash and merge` or `Rebase and merge`.')
    87-
    body.append(' - [ ] Merge the mergeback PR that will automatically be created once this PR is merged.')
    88103

    89-
    title = f'Merge {SOURCE_BRANCH} into {TARGET_BRANCH}'
    104+
    if is_primary_release:
    105+
    body.append(' - [ ] Merge the mergeback PR that will automatically be created once this PR is merged.')
    106+
    body.append(' - [ ] Merge all backport PRs to older release branches, that will automatically be created once this PR is merged.')
    107+
    108+
    title = f'Merge {source_branch} into {target_branch}'
    109+
    labels = ['Update dependencies'] if not is_primary_release else []
    90110

    91111
    # Create the pull request
    92112
    # PR checks won't be triggered on PRs created by Actions. Therefore mark the PR as draft so that
    93113
    # a maintainer can take the PR out of draft, thereby triggering the PR checks.
    94-
    pr = repo.create_pull(title=title, body='\n'.join(body), head=new_branch_name, base=TARGET_BRANCH, draft=True)
    95-
    print(f'Created PR #{pr.number}')
    114+
    pr = repo.create_pull(title=title, body='\n'.join(body), head=new_branch_name, base=target_branch, draft=True)
    115+
    pr.add_to_labels(*labels)
    116+
    print(f'Created PR #{str(pr.number)}')
    96117

    97118
    # Assign the conductor
    98119
    pr.add_to_assignees(conductor)
    @@ -102,10 +123,10 @@ def open_pr(repo, all_commits, source_branch_short_sha, new_branch_name, conduct
    102123
    # since the last release to the target branch.
    103124
    # This will not include any commits that exist on the target branch
    104125
    # that aren't on the source branch.
    105-
    def get_commit_difference(repo):
    126+
    def get_commit_difference(repo, source_branch, target_branch):
    106127
    # Passing split nothing means that the empty string splits to nothing: compare `''.split() == []`
    107128
    # to `''.split('\n') == ['']`.
    108-
    commits = run_git('log', '--pretty=format:%H', f'{ORIGIN}/{TARGET_BRANCH}..{ORIGIN}/{SOURCE_BRANCH}').strip().split()
    129+
    commits = run_git('log', '--pretty=format:%H', f'{ORIGIN}/{target_branch}..{ORIGIN}/{source_branch}').strip().split()
    109130

    110131
    # Convert to full-fledged commit objects
    111132
    commits = [repo.get_commit(c) for c in commits]
    @@ -182,6 +203,24 @@ def main():
    182203
    required=True,
    183204
    help='The nwo of the repository, for example github/codeql-action.'
    184205
    )
    206+
    parser.add_argument(
    207+
    '--source-branch',
    208+
    type=str,
    209+
    required=True,
    210+
    help='Source branch for release branch update.'
    211+
    )
    212+
    parser.add_argument(
    213+
    '--target-branch',
    214+
    type=str,
    215+
    required=True,
    216+
    help='Target branch for release branch update.'
    217+
    )
    218+
    parser.add_argument(
    219+
    '--is-primary-release',
    220+
    action='store_true',
    221+
    default=False,
    222+
    help='Whether this update is the primary release for the current major version.'
    223+
    )
    185224
    parser.add_argument(
    186225
    '--conductor',
    187226
    type=str,
    @@ -191,18 +230,29 @@ def main():
    191230

    192231
    args = parser.parse_args()
    193232

    233+
    source_branch = args.source_branch
    234+
    target_branch = args.target_branch
    235+
    is_primary_release = args.is_primary_release
    236+
    194237
    repo = Github(args.github_token).get_repo(args.repository_nwo)
    195-
    version = get_current_version()
    238+
    239+
    # the target branch will be of the form releases/vN, where N is the major version number
    240+
    target_branch_major_version = target_branch.strip('releases/v')
    241+
    242+
    # split version into major, minor, patch
    243+
    _, v_minor, v_patch = get_current_version().split('.')
    244+
    245+
    version = f"{target_branch_major_version}.{v_minor}.{v_patch}"
    196246

    197247
    # Print what we intend to go
    198-
    print(f'Considering difference between {SOURCE_BRANCH} and {TARGET_BRANCH}...')
    199-
    source_branch_short_sha = run_git('rev-parse', '--short', f'{ORIGIN}/{SOURCE_BRANCH}').strip()
    200-
    print(f'Current head of {SOURCE_BRANCH} is {source_branch_short_sha}.')
    248+
    print(f'Considering difference between {source_branch} and {target_branch}...')
    249+
    source_branch_short_sha = run_git('rev-parse', '--short', f'{ORIGIN}/{source_branch}').strip()
    250+
    print(f'Current head of {source_branch} is {source_branch_short_sha}.')
    201251

    202252
    # See if there are any commits to merge in
    203-
    commits = get_commit_difference(repo=repo)
    253+
    commits = get_commit_difference(repo=repo, source_branch=source_branch, target_branch=target_branch)
    204254
    if len(commits) == 0:
    205-
    print(f'No commits to merge from {SOURCE_BRANCH} to {TARGET_BRANCH}.')
    255+
    print(f'No commits to merge from {source_branch} to {target_branch}.')
    206256
    return
    207257

    208258
    # The branch name is based off of the name of branch being merged into
    @@ -220,17 +270,80 @@ def main():
    220270
    # Create the new branch and push it to the remote
    221271
    print(f'Creating branch {new_branch_name}.')
    222272

    223-
    # If we're performing a standard release, there won't be any new commits on the target branch,
    224-
    # as these will have already been merged back into the source branch. Therefore we can just
    225-
    # start from the source branch.
    226-
    run_git('checkout', '-b', new_branch_name, f'{ORIGIN}/{SOURCE_BRANCH}')
    273+
    # The process of creating the v{Older} release can run into merge conflicts. We commit the unresolved
    274+
    # conflicts so a maintainer can easily resolve them (vs erroring and requiring maintainers to
    275+
    # reconstruct the release manually)
    276+
    conflicted_files = []
    277+
    278+
    if not is_primary_release:
    279+
    280+
    # the source branch will be of the form releases/vN, where N is the major version number
    281+
    source_branch_major_version = source_branch.strip('releases/v')
    282+
    283+
    # If we're performing a backport, start from the target branch
    284+
    print(f'Creating {new_branch_name} from the {ORIGIN}/{target_branch} branch')
    285+
    run_git('checkout', '-b', new_branch_name, f'{ORIGIN}/{target_branch}')
    286+
    287+
    # Revert the commit that we made as part of the last release that updated the version number and
    288+
    # changelog to refer to {older}.x.x variants. This avoids merge conflicts in the changelog and
    289+
    # package.json files when we merge in the v{latest} branch.
    290+
    # This commit will not exist the first time we release the v{N-1} branch from the v{N} branch, so we
    291+
    # use `git log --grep` to conditionally revert the commit.
    292+
    print('Reverting the version number and changelog updates from the last release to avoid conflicts')
    293+
    vOlder_update_commits = run_git('log', '--grep', f'^{BACKPORT_COMMIT_MESSAGE}', '--format=%H').split()
    294+
    295+
    if len(vOlder_update_commits) > 0:
    296+
    print(f' Reverting {vOlder_update_commits[0]}')
    297+
    # Only revert the newest commit as older ones will already have been reverted in previous
    298+
    # releases.
    299+
    run_git('revert', vOlder_update_commits[0], '--no-edit')
    300+
    301+
    # Also revert the "Update checked-in dependencies" commit created by Actions.
    302+
    update_dependencies_commit = run_git('log', '--grep', '^Update checked-in dependencies', '--format=%H').split()[0]
    303+
    print(f' Reverting {update_dependencies_commit}')
    304+
    run_git('revert', update_dependencies_commit, '--no-edit')
    305+
    306+
    else:
    307+
    print(' Nothing to revert.')
    308+
    309+
    print(f'Merging {ORIGIN}/{source_branch} into the release prep branch')
    310+
    # Commit any conflicts (see the comment for `conflicted_files`)
    311+
    run_git('merge', f'{ORIGIN}/{source_branch}', allow_non_zero_exit_code=True)
    312+
    conflicted_files = run_git('diff', '--name-only', '--diff-filter', 'U').splitlines()
    313+
    if len(conflicted_files) > 0:
    314+
    run_git('add', '.')
    315+
    run_git('commit', '--no-edit')
    316+
    317+
    # Migrate the package version number from a vLatest version number to a vOlder version number
    318+
    print(f'Setting version number to {version}')
    319+
    subprocess.check_output(['npm', 'version', version, '--no-git-tag-version'])
    320+
    run_git('add', 'package.json', 'package-lock.json')
    321+
    322+
    # Migrate the changelog notes from vLatest version numbers to vOlder version numbers
    323+
    print(f'Migrating changelog notes from v{source_branch_major_version} to v{target_branch_major_version}')
    324+
    subprocess.check_output(['sed', '-i', f's/^## {source_branch_major_version}\./## {target_branch_major_version}./g', 'CHANGELOG.md'])
    325+
    326+
    # Remove changelog notes from all versions that do not apply to the vOlder branch
    327+
    print(f'Removing changelog notes that do not apply to v{target_branch_major_version}')
    328+
    for v in range(int(source_branch_major_version), int(target_branch_major_version), -1):
    329+
    print(f'Removing changelog notes that are tagged [v{v}+ only\]')
    330+
    subprocess.check_output(['sed', '-i', f'/^- \[v{v}+ only\]/d', 'CHANGELOG.md'])
    331+
    332+
    # Amend the commit generated by `npm version` to update the CHANGELOG
    333+
    run_git('add', 'CHANGELOG.md')
    334+
    run_git('commit', '-m', f'{BACKPORT_COMMIT_MESSAGE}{version}')
    335+
    else:
    336+
    # If we're performing a standard release, there won't be any new commits on the target branch,
    337+
    # as these will have already been merged back into the source branch. Therefore we can just
    338+
    # start from the source branch.
    339+
    run_git('checkout', '-b', new_branch_name, f'{ORIGIN}/{source_branch}')
    227340

    228-
    print('Updating changelog')
    229-
    update_changelog(version)
    341+
    print('Updating changelog')
    342+
    update_changelog(version)
    230343

    231-
    # Create a commit that updates the CHANGELOG
    232-
    run_git('add', 'CHANGELOG.md')
    233-
    run_git('commit', '-m', f'Update changelog for v{version}')
    344+
    # Create a commit that updates the CHANGELOG
    345+
    run_git('add', 'CHANGELOG.md')
    346+
    run_git('commit', '-m', f'Update changelog for v{version}')
    234347

    235348
    run_git('push', ORIGIN, new_branch_name)
    236349

    @@ -240,7 +353,11 @@ def main():
    240353
    commits,
    241354
    source_branch_short_sha,
    242355
    new_branch_name,
    356+
    source_branch=source_branch,
    357+
    target_branch=target_branch,
    243358
    conductor=args.conductor,
    359+
    is_primary_release=is_primary_release,
    360+
    conflicted_files=conflicted_files
    244361
    )
    245362

    246363
    if __name__ == '__main__':

    ‎.github/workflows/__all-platform-bundle.yml

    Lines changed: 1 addition & 1 deletion
    Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

    ‎.github/workflows/__analyze-ref-input.yml

    Lines changed: 1 addition & 1 deletion
    Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

    ‎.github/workflows/__autobuild-action.yml

    Lines changed: 1 addition & 1 deletion
    Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

    ‎.github/workflows/__config-export.yml

    Lines changed: 1 addition & 1 deletion
    Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

    0 commit comments

    Comments
     (0)
    0