8000 add GGML_USE_NUMA_MIGRATE feature to optimize cross NUMA op computation by wenlujon · Pull Request #13649 · ggml-org/llama.cpp · GitHub
[go: up one dir, main page]
Skip to content

add GGML_USE_NUMA_MIGRATE feature to optimize cross NUMA op computation #13649

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

ad 8000 d GGML_USE_NUMA_MIGRATE feature to optimize cross NUMA op computation #13649

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
a7a6ce9
add GGML_USE_NUMA_MIGRATE feature to optimize cross NUMA op computation
boltliu85 May 20, 2025
23fe7d3
fix the threads number larger than numa node numbers
boltliu85 May 20, 2025
e5cb47d
fix the buffer allocate size for NUMA page migration
boltliu85 May 21, 2025
ab80f55
fix the cgraph null issue when running with llama-bench
boltliu85 May 21, 2025
3e3a878
use ggml_barrier instead of ggml_barrier_numa_aware for src1 data bar…
boltliu85 May 21, 2025
ed4d987
remove debug code for ggml barrier
boltliu85 May 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix the threads number larger than numa node numbers
  • Loading branch information
@boltliu85
boltliu85 committed May 20, 2025
commit 23fe7d38d2c591ea2f7f554bfbb41e23f09cfbb5
2 changes: 1 addition & 1 deletion ggml/src/ggml-cpu/ggml-cpu.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -593,7 +593,7 @@ void ggml_barrier_numa_aware(struct ggml_threadpool * tp, int ith, int node_n) {
int cores_per_numa = g_state.numa.nodes[0].n_cpus;
int numa_nodes = n_threads / cores_per_numa;
int remaining_cores = n_threads % cores_per_numa;
if (numa_nodes <= 1 || remaining_cores) {
if ((numa_nodes != GGML_NUMA_MIGRATE_NODES) || remaining_cores) {
ggml_barrier(tp);
return;
}
Expand Down
0