8000 [v2.1.2] Release Tracker · Issue #113962 · pytorch/pytorch · GitHub
[go: up one dir, main page]
Skip to content

[v2.1.2] Release Tracker #113962

Ne 8000 w issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
atalman opened this issue Nov 17, 2023 · 26 comments
Closed

[v2.1.2] Release Tracker #113962

atalman opened this issue Nov 17, 2023 · 26 comments
Labels
triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Milestone
2.1.2

Comments

@atalman
Copy link
Contributor
atalman commented Nov 17, 2023 •
edited by malfet
Loading

This issue is for tracking cherry-picks to the release branch. Following is release branch for the 2.1.2 release.

Our plan from this point from this point is roughly:

  • Phase 1 (until 12/1): Cherry-pick post deadline (End of day 5PM PST)
  • Phase 2 (after 12/1): Perform extended integration/stability/performance testing based on Release Candidate builds.

Only issues that have ‘cherry-picks’ in this tracker will be considered for the release.

Cherry-Pick Criteria

Phase 1 (until 12/1):

The Releng team relies on the cherry pick process to manage risk to release quality, i.e. by porting a small set of commit from trunk that are "must-have" into the release branch, we limit the change to the minimal to address pressing issues. Thus, not everything a developer land into the trunk will make it into the release. So, please consider the criteria below and follow the cherry picking process. Only low-risk changes may be cherry-picked from master:

  1. Fixes to regressions against the most recent release (e.g. 2.1.0 for 2.1.2 release; see module: regression issue list)
  2. Low risk critical fixes for: silent correctness, backwards compatibility, crashes, deadlocks, (large) memory leaks
  3. Fixes to new features being introduced in 2.1.0 release
  4. Documentation improvements
  5. Release branch specific changes (e.g. blocking ci fixes, change version identifiers)

Any other change requires special dispensation from the release managers (currently @atalman, @huydhn, @osalpekar, @malfet). If this applies to your change please write "Special Dispensation" in the "Criteria Category:" template below and explain.

Phase 2 (after 12/1):

Note that changes here require us to rebuild a Release Candidate and restart extended testing (likely delaying the release). Therefore, the only accepted changes are Release-blocking critical fixes for: silent correctness, backwards compatibility, crashes, deadlocks, (large) memory leaks

Changes will likely require a discussion with the larger release team over VC or Slack.

Cherry-Pick Process

  1. Ensure your PR has landed in master. This does not apply for release-branch specific changes (see Phase 1 criteria).

  2. Create (but do not land) a PR against the release branch.

    # Find the hash of the commit you want to cherry pick
# (for example, abcdef12345)
git log

git fetch origin release/2.1
git checkout release/2.1
git cherry-pick abcdef12345

# Submit a PR based against 'release/2.1' either:
# via the GitHub UI
git push my-fork

# via the GitHub CLI
gh pr create --base release/2.1

  3. Make a request below with the following format:

Link to landed trunk PR (if applicable):
* 

Link to release branch PR:
* 

Criteria Category:
*
  1. Someone from the release team will reply with approved / denied or ask for more information.
  2. If approved, someone from the release team will merge your PR once the tests pass. Do not land the release branch PR yourself.

NOTE: Our normal tools (ghstack / ghimport, etc.) do not work on the release branch.

See HUD 2.1

Versions

2.1.2
@atalman atalman pinned this issue Nov 17, 2023
@janeyx99 janeyx99 added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Nov 17, 2023
@malfet
Copy link
Contributor
malfet commented Nov 22, 2023 •
edited by atalman
Loading

Link to landed trunk PR :

Link to release branch PR:

Criteria Category:

  • Fixes to new features being introduced in 2.1.0 release (poetry support)

@malfet merged

@atalman
Copy link
Contributor Author
atalman commented Nov 22, 2023 •
edited
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:

@atalman will not be merging it, issue associated with PR: #114489

@atalman atalman added this to the 2.1.2 milestone Nov 22, 2023
@github-project-automation github-project-automation bot moved this to Review Required in Release Milestone Review Nov 22, 2023
@atalman atalman moved this from Review Required to Validation Required in Release Milestone Review Nov 22, 2023
@malfet
Copy link
Contributor
8000
malfet commented Nov 22, 2023 •
edited by atalman
Loading

Link to landed trunk PR :

Link to release branch PR:

Criteria Category:

  • Regressions from the previous release (2.1.0)

@atalman merged

@malfet
Copy link
Contributor
malfet commented Nov 23, 2023 •
edited by atalman
Loading

Link to landed trunk PR :

Link to release branch PR:

Criteria Category:

  • Critical fix to PyTorch behavior as AWS lambda

@atalman merged

@malfet
Copy link
Contributor
malfet commented Nov 23, 2023 •
edited by atalman
Loading

Link to landed trunk PR :

Link to release branch PR:

Criteria Category:

  • Fixes crash during conv backward path when MPS is used

@atalman merged

@malfet
Copy link
Contributor
malfet commented Nov 23, 2023 •
edited by atalman
Loading

Link to landed trunk PR :

Link to release branch PR:

Criteria Category:

  • Critical fix to PyTorch behavior as AWS lambda

@atalman merged

@huydhn
Copy link
Contributor
huydhn commented Nov 24, 2023 •
edited
Loading

Link to landed trunk PR :

Link to release branch PR:

Criteria Category:

  • Critical CI fix to build PyTorch on Windows for 3.10 and 311

@atalman merged

@atalman
Copy link
Contributor Author
atalman commented Nov 24, 2023 •
edited
Loading

Link to landed trunk PR (if applicable):

  • NA

Link to release branch PR:

Criteria Category:

  • Release only, pin tests

@atalman merged

@jcaip
Copy link
Contributor
jcaip commented Nov 27, 2023 •
edited by atalman
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:

  • Critical Fixes

@atalman I can confirm #110664 is regression against 2.1.0 . However PR looks like feature work. There is also a workaround for the issue above: #110664 (comment)

@penguinwu penguinwu unpinned this issue Nov 27, 2023
@atalman atalman pinned this issue Nov 28, 2023
@penguinwu penguinwu unpinned this issue Nov 29, 2023
@atalman atalman pinned this issue Nov 29, 2023
@atalman
Copy link
Contributor Author
atalman commented Nov 29, 2023 •
edited by malfet
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:

  • Regression from a previous release, caused by pybind upgrade

@malfet: merged

@gdippolito
Copy link
Contributor

Would it be possible to cherry-pick this commitin 2.1.2? The change is self-contained and would fix a bug that was introduced in 2.1.0.
I have tested it and the commit can be cherry-picked without conflicts. Thanks 🙏

@atalman atalman moved this from Validation Required to Done in Release Milestone Review Nov 29, 2023
@huydhn
Copy link
Contributor
huydhn commented Nov 29, 2023 •
edited
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:

  • Regression from 2.0.1

@huydhn Confirm locally that the bug in #109333 is a regression from 2.0.1 to 2.1.0. Merged

@drisspg drisspg mentioned this issue Nov 30, 2023
Closed
@drisspg
Copy link
Contributor
drisspg commented Nov 30, 2023 •
edited by atalman
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:

  • Fixes a memory regression introduced in 2.1.1 patch fix

@atalman Looks like cherry pick combines #111721 and #114173 PRs (and also this small tweak #114909)

@atalman merged

@atalman: Because of the new issue found #112577 (comment) will be reverting this cherry-pick

@imzhuhl
Copy link
Contributor
imzhuhl commented Nov 30, 2023 •
edited by atalman
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:

  • Critical Fixes

@atalman I can confirm this is regression against 2.1 release. With 2.0 release this code is passing
@malfet Merged

@atalman
Copy link
Contributor Author
atalman commented Dec 1, 2023 •
edited
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:

  • Critical fix, MPS

@malfet merged

@EikanWang EikanWang unpinned this issue Dec 4, 2023
@atalman atalman pinned this issue Dec 4, 2023
@bhack
Copy link
Contributor
bhack commented Dec 4, 2023 •
edited by atalman
Loading

Are we sure that we could still deliver Cuda 11.x wheels?
Please check:
#106144
#115075

@atalman wrote: @bhack this is release 2.1.2 tracker, both of the issues above refer to nightly builds.

@bhack
Copy link
Contributor
bhack commented Dec 4, 2023 •
edited by atalman
Loading

@atalman but they are related as the first one is nightly but since july. Are we not in that branch cut since july?

@atalman wrote:

@bhack branch for 2.1.0 was created on Aug 27, not July.

#115075 is about current nightly, however there is missing reproduction step. Only install command and error are included. Please add proper repro code.

#106144 I tried to repro this issue with 2.1.2 and reproduction code from this comment: #106144 (comment) could not repro it, please see this comment: #106144 (comment)

For release 2.1.2, please install 2.1.2 binary using following command:

pip install --upgrade torch torchvision torchaudio --index-url https://download.pytorch.org/whl/test/cu118
or
pip install --upgrade torch torchvision torchaudio --index-url https://download.pytorch.org/whl/test/cu121

Please provide more details on both issues, including full reproduction your environment and cuda driver version you are running.

@huydhn
Copy link
Contributor
huydhn commented Dec 6, 2023 •
edited
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:

@huydhn merged

@bhack
Copy link
Contributor
bhack commented Dec 7, 2023

@atalman I don't have any notification if you post-edit my own messages inline. So please, if you can, reply with a new comment as all other people are doing in almost every Github repo.

For the others interested, we are discussing details in the mentioned tickets own threads.

@drisspg
Copy link
Contributor
drisspg commented Dec 12, 2023

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:

  • Fixes a memory regression introduced in 2.1.1 patch fix

@atalman Looks like cherry pick combines #111721 and #114173 PRs (and also this small tweak #114909)

@atalman merged

@atalman: Because of the new issue found #112577 (comment) will be reverting this cherry-pick

Follow up here, users in HuggingFace are saying that this inclusion has caused an IMA in one of their models, therefore we are going to revert for the cherry pick but still include in Main while I work to debug this issue.

@atalman
Copy link
Contributor Author
atalman commented Dec 12, 2023 •
edited
Loading

Link to landed trunk PR (if applicable):

  • NA

Link to release branch PR:

Criteria Category:

@atalman merged

@atalman
Copy link
Contributor Author
atalman commented Dec 12, 2023

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:

  • Critical CI fix

@atalman merged

@huydhn
Copy link
Contributor
huydhn commented Dec 13, 2023 •
edited by atalman
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:

  • Critical CI fix to allow building domain RC for both 2.1.2 and 2.2.0 in parallel

@atalman merged

@atalman
Copy link
Contributor Author
atalman commented Dec 13, 2023 •
edited
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:

  • Critical CI fix to allow building domain RC for both 2.1.2 and 2.2.0 in parallel

@atalman merged

@droberts195 droberts195 mentioned this issue Dec 14, 2023
Merged
@malfet
Copy link
Contributor
malfet commented Dec 14, 2023

Closing this issue as wheels and conda binaries has been published https://pypi.org/project/torch/2.1.2/ and https://anaconda.org/pytorch/pytorch/files?version=2.1.2

@malfet malfet closed this as completed Dec 14, 2023
@malfet malfet unpinned this issue Dec 14, 2023
@atalman
Copy link
Contributor Author
atalman commented Dec 18, 2023 •
edited
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:

  • CI build docker cuda 11.8 for cuda 11.8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Milestone
2.1.2
Development

No branches or pull requests
10 participants
@huydhn @bhack @malfet @atalman @jcaip @imzhuhl @gdippolito @janeyx99 @drisspg and others
0