10000 multiple values for argument `softmax_scale` · Issue #101603 · pytorch/pytorch · GitHub
[go: up one dir, main page]
Skip to content

multiple values for argument softmax_scale #101603

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
msarou 8000 fim opened this issue May 16, 2023 · 5 comments
Closed

multiple values for argument softmax_scale #101603

msaroufim opened this issue May 16, 2023 · 5 comments
Labels
module: fsdp oncall: distributed Add this issue/PR to distributed oncall triage queue oncall: pt2 pt2d-triage-nov2024 triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@msaroufim
Copy link
Member
msaroufim commented May 16, 2023 •
edited by pytorch-bot bot
Loading

🐛 Describe the bug

Posting this on behalf of Mosaic

pip install --no-cache-dir --find-links https://download.pytorch.org/whl/torch_stable.html torch==2.0.0+cu117 torchvision torchtext
pip install einops

python attention.py

https://raw.githubusercontent.com/sashaDoubov/llm-foundry/sasha/repro-issue/llmfoundry/models/layers/attention.py

Error logs

Not an error but instead seeing this scary warning


[2023-05-15 23:04:35,681] torch._dynamo.symbolic_convert: [WARNING] /llm-foundry/llmfoundry/models/layers/attention.py <function xformers_attn_fn at 0x7f2a6834ed40> [UnspecializedNNModuleVariable(MultiheadAttention), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(int)] {'softmax_scale': ConstantVariable(float), 'attn_bias': TensorVariable(), 'key_padding_mask': ConstantVariable(NoneType), 'is_causal': ConstantVariable(bool), 'dropout_p': ConstantVariable(float), 'training': ConstantVariable(bool), 'needs_weights': ConstantVariable(bool)} multiple values for argument 'softmax_scale'

[2023-05-15 23:04:35,707] torch._dynamo.symbolic_convert: [WARNING] /llm-foundry/llmfoundry/models/layers/attention.py <function xformers_attn_fn at 0x7f2a6834ed40> [UnspecializedNNModuleVariable(MultiheadAttention), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(int)] {'softmax_scale': ConstantVariable(float), 'attn_bias': TensorVariable(), 'key_padding_mask': ConstantVariable(NoneType), 'is_causal': ConstantVariable(bool), 'dropout_p': ConstantVariable(float), 'training': ConstantVariable(bool), 'needs_weights': ConstantVariable(bool)} multiple values for argument 'softmax_scale'

How to remove the error

As a workaround you can comment out softmax_scale and the code then works https://gist.github.com/msaroufim/5fe1a5cf745e31baabeb62b8dce10c82 but that's not a real solution

Versions

n

cc @H-Huang @awgu @kwen2501 @wanchaol @fegin @fduwjj @wz337 @wconstab @d4l3k @c-p-i-o @zhaojuanmao @mrshenli @rohan-varma @chauhang @penguinwu @pritamdamania87 @satgera @gqchen @aazzolini @osalpekar @jiayisuse @XilunWu @tianyu-l @yf225 @ezyang @bdhirsh @anijain2305 @zou3519 @kiukchung @LucasLLC @ngimel
@msaroufim
Copy link
Member Author
msaroufim commented May 16, 2023 •
edited
Loading

cc @wconstab @sashaDoubov @voznesenskym

@anijain2305
Copy link
Contributor

Cc @mlazos if you have bandwidth

@sashaDoubov
Copy link

I have also tried this with DDP and don't see those warnings, making this seem specific to FSDP.

@Chillee Chillee added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label May 22, 2023
@Chillee
Copy link
Collaborator
Chillee commented May 22, 2023

Assigning this to @voznesenskym arbitrarily.

@albanD albanD added oncall: distributed Add this issue/PR to distributed oncall triage queue and removed module: distributed labels Dec 8, 2023
@yf225 yf225 assigned yf225 and unassigned voznesenskym Feb 27, 2024
@yf225 yf225 removed their assignment Nov 27, 2024
@rakirs3333
Copy link

Do we still see this happening? I tried running the code but do not see anything.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
module: fsdp oncall: distributed Add this issue/PR to distributed oncall triage queue oncall: pt2 pt2d-triage-nov2024 triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
9 participants
@penguinwu @msaroufim @yf225 @sashaDoubov @voznesenskym @Chillee @albanD @anijain2305 @rakirs3333
0