10000 Basic Vulkan Multi-GPU implementation by 0cc4m · Pull Request #5321 · ggml-org/llama.cpp · GitHub

Basic Vulkan Multi-GPU implementation#5321

Merged

0cc4m merged 8 commits intomasterfrom

0cc4m/vulkan-multigpu

Feb 7, 2024

Commits on Feb 5, 2024

generalize LLAMA_SPLIT_LAYER for all backends, do not expose device count and memory in llama.h
slaren
committed
Only do device info print in the beginning and initialize one backend for cpu assist
0cc4m
committed

Commits on Feb 6, 2024

Rework backend memory management to make sure devices and buffers get properly allocated and freed
0cc4m
committed
Rename cpu assist free function
0cc4m
committed

0