8000 perf: assign to candidates data structure instead · KoolSameer/llama-cpp-python@a14d8a9 · GitHub
[go: up one dir, main page]
Skip to content

Commit a14d8a9

Browse files
abetlenabetlen
committed
perf: assign to candidates data structure instead
1 parent ca11673 commit a14d8a9

File tree

< 8000 button data-component="IconButton" type="button" data-testid="collapse-file-tree-button" aria-expanded="true" aria-controls="diff_file_tree" class="prc-Button-ButtonBase-c50BI d-none d-md-flex position-relative fgColor-muted prc-Button-IconButton-szpyj" data-loading="false" data-no-visuals="true" data-size="medium" data-variant="invisible" aria-describedby=":Rsatdab:-loading-announcement" aria-labelledby=":R4atdab:">

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

‎llama_cpp/llama.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -487,9 +487,9 @@ def _sample(
487487
nl_logit = logits[self._token_nl]
488488
candidates = self._candidates
489489
candidates_data = self._candidates_data
490-
candidates_data["id"] = np.arange(n_vocab, dtype=np.intc) # type: ignore
491-
candidates_data["logit"] = logits
492-
candidates_data["p"] = np.zeros(n_vocab, dtype=np.single)
490+
candidates_data["id"][:] = np.arange(n_vocab, dtype=np.intc) # type: ignore
491+
candidates_data["logit"][:] = logits
492+
candidates_data["p"][:] = np.zeros(n_vocab, dtype=np.single)
493493
candidates.data = candidates_data.ctypes.data_as(llama_cpp.llama_token_data_p)
494494
candidates.sorted = llama_cpp.c_bool(False)
495495
candidates.size = llama_cpp.c_size_t(n_vocab)

0 commit comments

Comments
 (0)
0