Closed
Description
The function llama_kv_self_used_cells is marked deprecated:
Function 'llama_kv_self_used_cells' is deprecated, reason: 'Use llama_kv_self_seq_pos_max() instead'However, if we examine the code:
int32_t llama_kv_self_used_cells(const llama_context * ctx) {
const auto * kv = ctx->get_kv_self();
if (!kv) {
return 0;
}
int32_t res = 0;
for (uint32_t s = 0; s < ctx->get_cparams().n_seq_max; s++) {
const llama_pos p0 = kv->seq_pos_min(s);
const llama_pos p1 = kv->seq_pos_max(s);
if (p0 >= 0) {
res += (p1 - p0) + 1;
}
}
return res;
}vs the proposed replacement:
llama_pos llama_kv_self_seq_pos_max(llama_context * ctx, llama_seq_id seq_id) {
const auto * kv = ctx->get_kv_self();
if (!kv) {
return -1;
}
return kv->seq_pos_max(seq_id);
}I'm perhaps missing something, but it appears these are not equivalent functions, as self_seq_pos_max returns the max given a sequence id, whereas used cells gives the global used cell count.
Here's an example where I don't think it's appropriate to use the seq_id version:
void defrag_kv_if_thresh_greater(const float thresh) const {
const auto used_cells = llama_kv_self_used_cells(ctx);
const auto total_cells = llama_n_ctx(ctx);
if (static_cast<float>(used_cells) > thresh * static_cast<float>(total_cells) && slots.size() > 1) {
llama_kv_self_defrag(ctx);
}
}Thanks
Metadata
Metadata
Assignees
Labels
No labels