8000 Mark starcoder as fully-offloadable · brittlewis12/llmfarm_core.swift@15ab3c1 · GitHub
[go: up one dir, main page]
Skip to content
This repository was archived by the owner on Feb 6, 2024. It is now read-only.

Commit 15ab3c1

Browse files
brittlewis12brittlewis12
committed
Mark starcoder as fully-offloadable
* 4cd5f2d * ggml-org/llama.cpp#3945
1 parent 32dbf8d commit 15ab3c1

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

‎Sources/llmfarm_core_cpp/llama/llama.cpp

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6295,6 +6295,7 @@ static int llama_decode_internal(
62956295
model.arch == LLM_ARCH_FALCON ||
62966296
model.arch == LLM_ARCH_REFACT ||
62976297
model.arch == LLM_ARCH_MPT ||
6298+
model.arch == LLM_ARCH_STARCODER ||
62986299
model.arch == LLM_ARCH_STABLELM;
62996300
const bool fully_offloaded = model.n_gpu_layers >= (int) hparams.n_layer + 3;
63006301
if (ggml_cpu_has_cublas() && full_offload_supported && fully_offloaded) {

0 commit comments

Comments
 (0)
0