8000 SYCL: Implement few same quantized type copy kernels by qnixsynapse · Pull Request #13739 · ggml-org/llama.cpp · GitHub
[go: up one dir, main page]
Skip to content

SYCL: Implement few same quantized type copy kernels #13739

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jun 7, 2025
Merged

SYCL: Implement few same quantized type copy kernels #13739

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
c26934d
SYCL: Implement few same quantized type copy kernels
qnixsynapse May 24, 2025
608e881
Use memcpy for copying contiguous tensors
qnixsynapse May 28, 2025
faeb7f3
feat(sycl): add contiguous tensor copy support and device checks
qnixsynapse May 31, 2025
b36c550
refactor: replace specific block copy functions with template
qnixsynapse Jun 1, 2025
b6db005
Exclude BF16 support for COPY tensors for now
qnixsynapse Jun 3, 2025
4aa261a
perf: adjust SYCL copy kernel block sizes for efficiency
qnixsynapse Jun 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Use memcpy for copying contiguous tensors 
ggml-ci
  • Loading branch information
@qnixsynapse
qnixsynapse committed Jun 3, 2025
commit 608e88115244f1aa3c587336b3134b4f8c070007
7 changes: 5 additions & 2 deletions ggml/src/ggml-sycl/cpy.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -759,8 +759,11 @@ void ggml_sycl_cpy(ggml_backend_sycl_context & ctx, const ggml_tensor * src0, co

char * src0_ddc = (char *) src0->data;
char * src1_ddc = (char *) src1->data;

if (src0->type == GGML_TYPE_F32 && src1->type == GGML_TYPE_F32) {
GGML_SYCL_DEBUG("[SYCL] %s: Tensor supplied: %s to %s\n", __func__, ggml_type_name(src0->type),
ggml_type_name(src1->type));
if ((src0->type == src1->type) && (ggml_is_contiguous(src0) && ggml_is_contiguous(src1))) {
main_stream->memcpy(src1_ddc, src0_ddc, ggml_nbytes(src0));
} else if (src0->type == GGML_TYPE_F32 && src1->type == GGML_TYPE_F32) {
ggml_cpy_f32_f32_sycl(src0_ddc, src1_ddc, ne, ne00, ne01, ne02, nb00, nb01, nb02, nb03, ne10, ne11, ne12, nb10,
nb11, nb12, nb13, main_stream);
} else if (src0->type == GGML_TYPE_F32 && src1->type == GGML_TYPE_F16) {
Expand Down
0