8000 [Intel GPU] Enable mkdnn._linear_pointwise at XPU backend by ZhiweiYan-96 · Pull Request #140365 · pytorch/pytorch · GitHub
[go: up one dir, main page]
Skip to content

[Intel GPU] Enable mkdnn._linear_pointwise at XPU backend #140365

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 28 commits into from
Closed

[Intel GPU] Enable mkdnn._linear_pointwise at XPU backend #140365

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
249d701
Update
ZhiweiYan-96 Nov 12, 2024
f1dcb1a
Update
ZhiweiYan-96 Nov 12, 2024
691449b
Update
ZhiweiYan-96 Nov 12, 2024
5a0e425
Update
ZhiweiYan-96 Nov 13, 2024
2caa517
Update
ZhiweiYan-96 Nov 13, 2024
ef27ccb
Update
ZhiweiYan-96 Nov 13, 2024
be0755f
Update
ZhiweiYan-96 Nov 13, 2024
f87eb96
Update
ZhiweiYan-96 Nov 14, 2024
8b87482
Update
ZhiweiYan-96 Nov 25, 2024
5dd4806
Update
ZhiweiYan-96 Jan 7, 2025
6a0e085
Update
ZhiweiYan-96 Mar 31, 2025
c9f9ccd
Update
ZhiweiYan-96 May 13, 2025
2149ea7
Update
ZhiweiYan-96 May 13, 2025
fe81e43
Update
ZhiweiYan-96 May 13, 2025
82ce0a4
Update
ZhiweiYan-96 May 13, 2025
6399e84
Update
ZhiweiYan-96 May 14, 2025
9060933
Update
ZhiweiYan-96 May 15, 2025
3e66b0c
Update
ZhiweiYan-96 May 21, 2025
f112b57
Update
ZhiweiYan-96 May 21, 2025
60d2cb7
Update
ZhiweiYan-96 May 21, 2025
ba9b916
Update
ZhiweiYan-96 May 23, 2025
342fbb6
Update
ZhiweiYan-96 May 23, 2025
7400f84
Update
ZhiweiYan-96 May 23, 2025
c9a0e43
Update
ZhiweiYan-96 May 26, 2025
94c476f
Update
ZhiweiYan-96 May 26, 2025
d74e58b
Update
ZhiweiYan-96 May 26, 2025
4efb54f
Update
ZhiweiYan-96 May 26, 2025
c3718b7
Update
ZhiweiYan-96 May 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update 
[ghstack-poisoned]
  • Loading branch information
@ZhiweiYan-96
ZhiweiYan-96 committed May 21, 2025
commit f112b57cc8c49e0dd2d812e70228fd4cf07aa0c7
6 changes: 3 additions & 3 deletions aten/src/ATen/native/mkldnn/xpu/FusionUtils.cpp
View file
Open in desktop
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ZhiweiYan-96 , in terms of these literals like hardswish, slilu, etc, they should be defined as a constant variable.

Copy link
Collaborator Author
@ZhiweiYan-96 ZhiweiYan-96 May 26, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for reminding. I am investigating the elegant way to remove verbose if-else branches. Final solution is not ready yet, though.
I tried, using lambda function+ hash map ( e.g silu -> lambda func(silu)). The argument list is big, the code become more verbose.
Using template to dispatch function (e.g. silu -> handle<silu_func>()), but meet some syntax errors due to ambiguous function call roots from the complex argument list with std::optional.
Please allow me more time to carefully refactor this.

Copy link
Collaborator Author
@ZhiweiYan-96 ZhiweiYan-96 May 26, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@EikanWang @guangyey @etaf
Could you please help review the design? The overall logic is

The process would be

  1. Choose handle function according to categories. Categories consist of 3 type( no argument/need scalar/need algorithm)
  2. Mapping attr string to attribute insertion function in handle functions

Pros:

  1. no verbose if-else branch anymore, cleaner codes.
  2. User of this code could add append attribute insertion func in unordered_map following current code and does not need to handle if-else logic. extensibility is better.
  3. Classify the attribute to 3 categories can reduce the complexity of argument passing in std::function at unordered_map signature.

Cons:

  1. Categories still need a if-else.

Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ onednn::Attr unary_attr_with_arg(
std::string_view unary,
torch::List<std::optional<at::Scalar>> scalars =
torch::List<std::optional<at::Scalar>>(),
std::optional<std::string_view> algorithm = c10::nullopt,
std::optional<std::string_view> algorithm = std::nullopt,
onednn::Attr attr = Attr()) {
if (unary == "hardswish") {
return attr.append_post_eltwise(
Expand Down Expand Up @@ -53,11 +53,11 @@ onednn::Attr string_to_unary_attr(std::string_view unary, onednn::Attr attr) {
return unary_attr_with_arg(
"hardswish",
torch::List<std::optional<at::Scalar>>(),
c10::nullopt,
std::nullopt,
attr);
} else if (unary == "swish") {
return unary_attr_with_arg(
"silu", torch::List<std::optional<at::Scalar>>(), c10::nullopt, attr);
"silu", torch::List<std::optional<at::Scalar>>(), std::nullopt, attr);
}
return attr;
}
Expand Down
12 changes: 6 additions & 6 deletions aten/src/ATen/native/mkldnn/xpu/FusionUtils.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,24 +10,24 @@

namespace at::native::xpu {
at::native::onednn::Attr unary_attr_with_arg(
c10::string_view unary,
std::string_view unary,
torch::List<std::optional<at::Scalar>> scalars,
std::optional<c10::string_view> algorithm,
std::optional<std::string_view> algorithm,
onednn::Attr attr);

at::native::onednn::Attr string_to_unary_attr(
c10::string_view unary,
std::string_view unary,
onednn::Attr attr);

at::native::onednn::Attr construct_unary_attr(
c10::string_view unary,
std::string_view unary,
torch::List<std::optional<at::Scalar>> scalars,
std::optional<c10::string_view> algorithm,
std::optional<std::string_view> algorithm,
onednn::Attr attr);

template <bool is_matmul = false>
onednn::Attr construct_binary_attr(
c10::string_view binary,
std::string_view binary,
std::optional<at::Scalar> alpha,
const Tensor& other,
onednn::Attr attr) {
Expand Down
6 changes: 5 additions & 1 deletion aten/src/ATen/native/mkldnn/xpu/Linear.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
#include <FusionUtils.h>
#include <ATen/DeviceGuard.h>
#include <torch/library.h>

#include <FusionUtils.h>

namespace at::native::xpu {

Tensor linear_pointwise(
Expand All @@ -11,6 +13,7 @@ Tensor linear_pointwise(
torch::List<std::optional<at::Scalar>> scalars,
std::optional<std::string_view> algorithm) {
onednn::Attr att;
const OptionalDeviceGuard device_guard(device_of(input_t));
att = construct_unary_attr(attr, scalars, algorithm, att);
auto input = input_t.contiguous();

Expand Down Expand Up @@ -51,6 +54,7 @@ Tensor linear_pointwise_binary(
const Tensor& weight_t,
const std::optional<Tensor>& bias_opt,
std::string_view binary_attr) {
const OptionalDeviceGuard device_guard(device_of(input_t));
onednn::Attr attr;
attr = construct_binary_attr<true>(binary_attr, /*alpha*/ 1.f, other_t, attr);
auto input = input_t.contiguous();
Expand Down
Loading
0