10000 batch : auto-gen positions + verify multi-sequence input by ggerganov · Pull Request #14177 · ggml-org/llama.cpp · GitHub
[go: up one dir, main page]

Skip to content

batch : auto-gen positions + verify multi-sequence input #14177

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jun 15, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
cont : engrish [no ci]
  • Loading branch information
ggerganov committed Jun 14, 2025
commit a6b0e85b8b1caf8950a3bf5a4980c616a28695b2
2 changes: 1 addition & 1 deletion src/llama-batch.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -402,7 +402,7 @@ bool llama_batch_allocr::init(
const llama_seq_id s0 = batch.seq_id[i][0];
const llama_seq_id s1 = batch.seq_id[i][s];

// mark that sequences s1 is couled to s0
// mark that sequence s1 is coupled to s0
seq_cpl[s1][s0] = true;

// note: the other way around is not necessary for now
Expand Down
2 changes: 1 addition & 1 deletion src/llama-batch.h
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ struct llama_sbatch {
llama_sbatch(const llama_batch & batch, size_t n_embd, bool simple_split = false);
};

// a helper for sanitizing and fullfilling a batch
// a helper for sanitizing and fulfilling a batch
class llama_batch_allocr {
public:
llama_batch_allocr();
Expand Down
0