8000 SYCL: Implement few same quantized type copy kernels by qnixsynapse · Pull Request #13739 · ggml-org/llama.cpp · GitHub
[go: up one dir, main page]
Skip to content

SYCL: Implement few same quantized type copy kernels #13739

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jun 7, 2025
Merged

SYCL: Implement few same quantized type copy kernels #13739

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
c26934d
SYCL: Implement few same quantized type copy kernels
qnixsynapse May 24, 2025
608e881
Use memcpy for copying contiguous tensors
qnixsynapse May 28, 2025
faeb7f3
feat(sycl): add contiguous tensor copy support and device checks
qnixsynapse May 31, 2025
b36c550
refactor: replace specific block copy functions with template
qnixsynapse Jun 1, 2025
b6db005
Exclude BF16 support for COPY tensors for now
qnixsynapse Jun 3, 2025
4aa261a
perf: adjust SYCL copy kernel block sizes for efficiency
qnixsynapse Jun 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
feat(sycl): add contiguous tensor copy support and device checks 
Adds a memcpy path for contiguous tensors of the same type to optimize data transfer. Updates device support checks to recognize contiguous tensor operations, improving compatibility and performance.
  • Loading branch information
@qnixsynapse
qnixsynapse committed Jun 3, 2025
commit faeb7f3493923b48d4be68d5aa2cca363f867165
4 changes: 2 additions & 2 deletions ggml/src/ggml-sycl/cpy.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
#include "cpy.hpp"

#include <float.h>
#include <string>

#include "dequantize.hpp"
#include "ggml-sycl/common.hpp"
Expand Down Expand Up @@ -759,9 +760,8 @@ void ggml_sycl_cpy(ggml_backend_sycl_context & ctx, const ggml_tensor * src0, co
< B2B2 br>
char * src0_ddc = (char *) src0->data;
char * src1_ddc = (char *) src1->data;
GGML_SYCL_DEBUG("[SYCL] %s: Tensor supplied: %s to %s\n", __func__, ggml_type_name(src0->type),
ggml_type_name(src1->type));
if ((src0->type == src1->type) && (ggml_is_contiguous(src0) && ggml_is_contiguous(src1))) {
GGML_SYCL_DEBUG("%s: memcpy path\n", __func__);
main_stream->memcpy(src1_ddc, src0_ddc, ggml_nbytes(src0));
} else if (src0->type == GGML_TYPE_F32 && src1->type == GGML_TYPE_F32) {
ggml_cpy_f32_f32_sycl(src0_ddc, src1_ddc, ne, ne00, ne01, ne02, nb00, nb01, nb02, nb03, ne10, ne11, ne12, nb10,
Expand Down
3 changes: 3 additions & 0 deletions ggml/src/ggml-sycl/ggml-sycl.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4226,6 +4226,9 @@ static bool ggml_backend_sycl_device_supports_op(ggml_backend_dev_t dev, const g
{
ggml_type src0_type = op->src[0]->type;
ggml_type src1_type = op->src[1]->type;
if (src0_type == src1_type && (ggml_is_contiguous(op->src[0]) && ggml_is_contiguous(op->src[1]))) {
return true;
}
if (src0_type == GGML_TYPE_F32 && src1_type == GGML_TYPE_F32) {
return true;
}
Expand Down
0