8000 [v2.4.1] Release Tracker · Issue #132400 · pytorch/pytorch · GitHub
[go: up one dir, main page]
Skip to content

[v2.4.1] Release Tracker #132400

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
atalman opened this issue Aug 1, 2024 · 45 comments
Closed

[v2.4.1] Release Tracker #132400

atalman opened this issue Aug 1, 2024 · 45 comments
Labels
oncall: releng In support of CI and Release Engineering release tracker Add this label to release tracker issues triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Milestone
2.4.1

Comments

@atalman
Copy link
Contributor
atalman commented Aug 1, 2024 •
edited
Loading

🐛 Describe the bug

This issue is for tracking cherry-picks to the release branch. Following is release branch for the 2.4.1 release.

Our plan from this point is roughly the following:

  • Phase 1 (until 8/26): Cherry-pick post deadline (End of day 5PM PST)
  • Phase 2 (after 8/26): Perform extended integration/stability/performance testing based on Release Candidate builds.

Only issues that have ‘cherry-picks’ in this tracker will be considered for the release.

Cherry-Pick Criteria

Phase 1 (until 8/26):

Please note: No feature work allowed for cherry picks. The Releng team relies on the cherry pick process to manage risk to release quality, i.e. by porting a small set of commit from trunk that are "must-have" into the release branch, we limit the change to the minimal to address pressing issues. Thus, not everything a developer land into the trunk will make it into the release. So, please consider the criteria below and follow the cherry picking process. Only low-risk changes may be cherry-picked from master:

  1. Fixes to regressions against the most recent release (e.g. 2.4.0 for 2.4.1 release; see module: regression issue list)
  2. Low risk critical fixes for: silent correctness, backwards compatibility, crashes, deadlocks, (large) memory leaks
  3. Critical Fixes to new features being introduced in 2.4.0 release
  4. Documentation improvements
  5. Release branch specific changes (e.g. blocking ci fixes, change version identifiers)

Any other change requires special dispensation from the release managers (currently @atalman, @kit1980, @huydhn, @PaliC, @malfet). If this applies to your change please write "Special Dispensation" in the "Criteria Category:" template below and explain.

Phase 2 (after 8/26):

Note that changes here require us to rebuild a Release Candidate and restart extended testing (likely delaying the release). Therefore, the only accepted changes are Release-blocking critical fixes for: silent correctness, backwards compatibility, crashes, deadlocks, (large) memory leaks

Changes will likely require a discussion with the larger release team over VC or Slack.

Cherry-Pick Process

  1. Ensure your PR has landed in master. This does not apply for release-branch specific changes (see Phase 1 criteria).

  2. Create (but do not land) a PR against the release branch.

    # Find the hash of the commit you want to cherry pick
# (for example, abcdef12345)
git log

git fetch origin release/2.4
git checkout release/2.4
git cherry-pick abcdef12345

# Submit a PR based against 'release/2.4' either:
# via the GitHub UI
git push my-fork

# via the GitHub CLI
gh pr create --base release/2.4

  3. Make a request below with the following format:

Link to landed trunk PR (if applicable):
* 

Link to release branch PR:
* 

Criteria Category:
*
  1. Someone from the release team will reply with approved / denied or ask for more information.
  2. If approved, someone from the release team will merge your PR once the tests pass. Do not land the release branch PR yourself.

NOTE: Our normal tools (ghstack / ghimport, etc.) do not work on the release branch.

See HUD 2.4

Versions

2.4.1
@atalman atalman pinned this issue Aug 1, 2024
@atalman atalman added this to the 2.4.1 milestone Aug 1, 2024
@atalman
Copy link
Contributor Author
atalman commented Aug 1, 2024 •
edited
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:

@atalman merged

@atalman
Copy link
Contributor Author
atalman commented Aug 1, 2024 •
edited
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:

  • Critical CI fixes, builder repo

@atalman merged

@malfet malfet added the oncall: releng In support of CI and Release Engineering label Aug 1, 2024
@atalman atalman added the release tracker Add this label to release tracker issues label Aug 1, 2024
@yiming0416 yiming0416 unpinned this issue Aug 1, 2024
@atalman atalman pinned this issue Aug 2, 2024
@soulitzer soulitzer added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Aug 5, 2024
@malfet malfet mentioned this issue Aug 6, 2024
Closed
@angelayi
Copy link
Contributor
angelayi commented Aug 7, 2024 •
edited by malfet
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:

  • Fixes regression on tracing HF llama models ([export] Failed to trace HF Llama2 model #128394). Due to a change in the HF transformers repo and an unrelated change in PyTorch, torch 2.4 is unable to trace any HF llama models, which is a regression from torch 2.3.1. This PR fixes the issue by introducing a new higher order operator and pass. I understand this PR is very big though, which doesn't meet the criteria of "low-risk changes". I'll make a chat offline.

cc @guangy10

@malfet: This clearly looks like a feature work, but may be an exception could be made, depending on maturity of torch.export feature and how many customers are broken by that regression. Let's continue the discussion...

@mberr mberr mentioned this issue Aug 8, 2024
Merged
4 tasks
mberr added a commit to pykeen/pykeen that referenced this issue Aug 8, 2024
@mberr
🧹⚙️ Update GHA (#1415)
42e005c 
- [x] check documentation in default job
- [x] only run `mypy` & `flake8` on different Python versions
- [x] temporarily add a `torch<2.4` version constraint to fix issues on
Windows, cf. pytorch/pytorch#132400
@pytorchbot
Copy link
Collaborator
pytorchbot commented Aug 12, 2024 •
edited by malfet
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:
Critical - Fix silent correctness that people are reporting: #133100

@malfet: merged

@pytorchbot pytorchbot mentioned this issue Aug 12, 2024
Closed
mberr added a commit to mberr/torch-max-mem that referenced this issue Aug 12, 2024
@mberr
add max version constraint for torch on Windows
819931f 
cf. pytorch/pytorch#132400
@malfet malfet mentioned this issue Aug 13, 2024
Closed
@pytorchbot
Copy link
Collaborator
pytorchbot commented Aug 13, 2024 •
edited by atalman
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:
Docs

@atalman merged

@pytorchbot pytorchbot mentioned this issue Aug 13, 2024
Closed
@pytorchbot
Copy link
Collaborator
pytorchbot commented Aug 13, 2024 •
edited by atalman
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:
Critical fix for #130806

@atalman merged

@pytorchbot pytorchbot mentioned this issue Aug 13, 2024
Closed
@pytorchbot
Copy link
Collaborator
pytorchbot commented Aug 13, 2024 •
edited by atalman
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:
Critical - addresses: #119081 (comment)

@atalman merged

@pytorchbot pytorchbot mentioned this issue Aug 13, 2024
Closed
@pytorchbot
Copy link
Collaborator
pytorchbot commented Aug 14, 2024 •
edited by malfet
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category: Fixes to regressions against the most recent release

@atalman wrote: Looks like it fixes #128130 is this critical issue for release ? This issue is not part of milestone 2.4.1 for the release

@malfet: merged

@atalman
Copy link
Contributor Author
atalman commented Aug 14, 2024 •
edited
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:

@atalman merged

@pytorchbot
Copy link
Collaborator
pytorchbot commented Aug 15, 2024 •
edited by kit1980
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:
Critical - Https://github.com/pytorch/pytorch/issues/131864

@kit1980 merged

@pytorchbot pytorchbot mentioned this issue Aug 15, 2024
Closed
@pytorchbot
Copy link
Collaborator
pytorchbot commented Aug 15, 2024 •
edited by kit1980
Loading
8000

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:
Critical -

merged by @malfet

@atalman
Copy link
Contributor Author
atalman commented Aug 22, 2024 •
edited
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:

  • Critical fixes for CI infrastructure

@atalman merged

@atalman
Copy link
Contributor Author
atalman commented Aug 23, 2024 •
edited
Loading

Link to landed trunk PR (if applicable):

  • NA

Link to release branch PR:

Criteria Category:

  • Release only changes, make CI green. Critical fixes for CI infrastructure

@atalman merged

@atalman
Copy link
Contributor Author
atalman commented Aug 26, 2024 •
edited
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:

  • Use ephemeral runners for release building

@malfet merged

@pytorchbot
Copy link
Collaborator
pytorchbot commented Aug 26, 2024 •
edited by atalman
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:
Critical - Docker builds related

@malfet merged

@pytorchbot pytorchbot mentioned this issue Aug 26, 2024
Closed
@albanD
Copy link
Collaborator
albanD commented Aug 26, 2024 •
edited by atalman
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:

  • Fixes to regressions against the most recent release

@atalman merged

@pytorchbot
Copy link
Collaborator
pytorchbot commented Aug 27, 2024 •
edited by atalman
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:
Critical

@atalman merged

@pytorchbot pytorchbot mentioned this issue Aug 27, 2024
Closed
@atalman
Copy link
Contributor Author
atalman commented Aug 27, 2024 •
edited by malfet
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:

  • Regression against the most recent release

@atalman: merged

@bhack
Copy link
Contributor
bhack commented Aug 27, 2024

Remote cache is crashing I don't know if we could cherry-pick #134032

@malfet
Copy link
Contributor
malfet commented Aug 28, 2024

Remote cache is crashing I don't know if we could cherry-pick #134032

Is there an issue describing the crash somewhere? (As PR does not mention that it fixes any issues)

@bhack
Copy link
Contributor
bhack commented Aug 28, 2024 •
edited
Loading

Remote cache is crashing I don't know if we could cherry-pick #134032

Is there an issue describing the crash somewhere? (As PR does not mention that it fixes any issues)

pytorch/torch/_inductor/runtime/triton_heuristics.py

Line 1122 in d990dad

remote_cache.put(remote_cache_key, data)

pytorch/torch/_inductor/remote_cache.py

Line 44 in d990dad

return self._redis.set(self._get_key(key), data)

it is going then to cause redis encoders failure:
redis.exceptions.DataError: Invalid input of type: 'dict'. Convert to a bytes, string, int or float first.

@s-deeper s-deeper mentioned this issue Aug 28, 2024
Closed
@bhack
Copy link
Contributor
bhack commented Aug 28, 2024

From the author #134032 (comment)

@pytorchbot
Copy link
Collaborator
pytorchbot commented Aug 28, 2024 •
edited by malfet
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category:
Docs -

@malfet: merged

@pytorchbot pytorchbot mentioned this issue Aug 28, 2024
Closed
@pytorchbot
Copy link
Collaborator
pytorchbot commented Aug 28, 2024 •
edited by malfet
Loading

Link to landed trunk PR (if applicable):

Link to release branch PR:

Criteria Category: Doc rendering regression fix

@malfet: merged

@pytorchbot pytorchbot mentioned this issue Aug 28, 2024
Closed
@bhack
Copy link
Contributor
bhack commented Aug 28, 2024

@malfet Another confirmation #134032 (comment)

@bhack bhack mentioned this issue Aug 28, 2024
Closed
@bhack
Copy link
Contributor
bhack commented Sep 1, 2024 •
edited
Loading

#134032 was merged on main. There were enough moving parts in these classes since 2.4.0 so a straight cherry-picking will not work. Do we have a margin for a backport as we are so near to 2.4.1 release? If not we need to advise in the doc and release notes that https://pytorch.org/tutorials/recipes/torch_compile_caching_tutorial.html#torchinductor-autotune-remote-cache is broken on OSS/Redis.

@anzr299 anzr299 mentioned this issue Sep 4, 2024
Merged
@Geremia
Copy link
Geremia commented Sep 4, 2024

What time today is 2.4.1 expected to be released?

@atalman
Copy link
Contributor Author
atalman commented Sep 4, 2024

Release 2.4.1 has been released. Release notes are here: https://github.com/pytorch/pytorch/releases/tag/v2.4.1
Closing this issue.

@atalman atalman closed this as completed Sep 4, 2024
@bhack
Copy link
Contributor
bhack commented Sep 4, 2024

Why we have ignored that TORCHINDUCTOR_AUTOTUNE_REMOTE_CACHE is broken in the release note? I suppose that we could add an entry.

@atalman atalman unpinned this issue Sep 4, 2024
@atalman
Copy link
Contributor Author
atalman commented Sep 4, 2024 •
edited
Loading

Hi @bhack sorry we where not able to cherry-pick the fix for TORCHINDUCTOR_AUTOTUNE_REMOTE_CACHE PR: #134032 in the release 2.4.1. We will be targeting it to release 2.5.0

@bhack
Copy link
Contributor
bhack commented Sep 4, 2024

Do we want to add to the documentantion that currenty is borken?

@atalman
Copy link
Contributor Author
atalman commented Sep 5, 2024

hi @bhack You are correct, added tracked regressions section to Release notes: https://github.com/pytorch/pytorch/releases/tag/v2.4.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
oncall: releng In support of CI and Release Engineering release tracker Add this label to release tracker issues triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Milestone
2.4.1
Development

No branches or pull requests
9 participants
@bhack @malfet @Geremia @zou3519 @albanD @atalman @angelayi @soulitzer @pytorchbot
FD
0