8000 Add Jinja template support by ochafik · Pull Request #11016 · ggml-org/llama.cpp · GitHub
[go: up one dir, main page]
Skip to content

Add Jinja template support #11016

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 47 commits into from
Jan 21, 2025
Merged

Add Jinja template support #11016

Changes from 1 commit
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
abd274a
Copy minja from https://github.com/google/minja/commit/58f0ca6dd74bcb…
Dec 30, 2024
e5113e8
Add --jinja and --chat-template-file flags
Dec 30, 2024
80138d9
Add missing <optional> include
Dec 30, 2024
06b5159
Avoid print in get_hf_chat_template.py
Dec 30, 2024
ce48584
No designated initializers yet
Dec 30, 2024
389d79b
Try and work around msvc++ non-macro max resolution quirk
Dec 30, 2024
238b968
Update test_chat_completion.py
Dec 30, 2024
cb72cf1
Merge remote-tracking branch 'origin/master' into jinja
Jan 13, 2025
78861a3
Wire LLM_KV_TOKENIZER_CHAT_TEMPLATE_N in llama_model_chat_template
Jan 13, 2025
1aac99a
Refactor test-chat-template
Jan 13, 2025
7c84ebc
Test templates w/ minja
Jan 13, 2025
18f257b
Fix deprecation
Jan 13, 2025
8dd4f33
Add --jinja to llama-run
Jan 13, 2025
c04c50e
Merge remote-tracking branch 'origin/master' into jinja
Jan 13, 2025
a6afb27
Update common_chat_format_example to use minja template wrapper
Jan 13, 2025
b4083e4
Test chat_template in e2e test
Jan 13, 2025
b7e2171
Update utils.py
Jan 13, 2025
a57bb94
Update test_chat_completion.py
Jan 13, 2025
4daae0b
Update run.cpp
Jan 13, 2025
1b3bb7e
Update arg.cpp
ochafik Jan 14, 2025
3ed670b
Merge remote-tracking branch 'origin/master' into jinja
Jan 14, 2025
b75d062
Refactor common_chat_* functions to accept minja template + use_jinja…
Jan 18, 2025
40db789
Merge remote-tracking branch 'origin/master' into jinja
Jan 18, 2025
81c0d43
Attempt to fix linkage of LLAMA_CHATML_TEMPLATE
Jan 18, 2025
d5fa351
Revert LLAMA_CHATML_TEMPLATE refactor
Jan 18, 2025
ee1e10e
Normalize newlines in test-chat-templates for windows tests
Jan 18, 2025
e63520f
Forward decl minja::chat_template to avoid eager json dep
Jan 18, 2025
33322e8
Flush stdout in chat template before potential crash
Jan 18, 2025
5074e6f
Fix copy elision warning
Jan 18, 2025
fc60802
Rm unused optional include
Jan 18, 2025
0e74c9d
Add missing optional include to server.cpp
Jan 18, 2025
e3c475c
Disable jinja test that has a cryptic windows failure
Jan 18, 2025
cc50356
minja: fix vigogne (https://github.com/google/minja/pull/22)
Jan 18, 2025
153e852
Apply suggestions from code review
ochafik Jan 20, 2025
db9dd0c
Finish suggested renamings
Jan 20, 2025
c9e8fdd
Move chat_templates inside server_context + remove mutex
Jan 20, 2025
8c84aef
Update --chat-template-file w/ recent change to --chat-template
Jan 20, 2025
154bfaa
Refactor chat template validation
Jan 20, 2025
099f983
Merge remote-tracking branch 'origin/master' into jinja
Jan 20, 2025
54a669e
Guard against missing eos/bos tokens (null token otherwise throws in …
Jan 20, 2025
8348c60
Warn against missing eos / bos tokens when jinja template references …
Jan 20, 2025
ee475d2
rename: common_chat_template[s]
Jan 20, 2025
8a7c89e
reinstate assert on chat_templates.template_default
Jan 20, 2025
8347da9
Update minja to https://github.com/google/minja/commit/b8437df626ac6c…
Jan 20, 2025
ff2cce5
Update minja to https://github.com/google/minja/pull/25
Jan 21, 2025
9d8ebd6
Update minja from https://github.com/google/minja/pull/27
Jan 21, 2025
cbb9b81
rm unused optional header
Jan 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix deprecation
  • Loading branch information
ochafik committed Jan 13, 2025
commit 18f257bf1a1aabea100935151a9e7eb09ff80f93
4 changes: 2 additions & 2 deletions common/common.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1825,8 +1825,8 @@ std::string common_chat_format_example(const struct llama_model * model,
llama_chat_templates llama_chat_templates_from_model(const struct llama_model * model, const std::string & chat_template_override)
{
auto vocab = llama_model_get_vocab(model);
auto bos_token = common_token_to_piece(vocab, llama_token_bos(vocab), true);
auto eos_token = common_token_to_piece(vocab, llama_token_eos(vocab), true);
auto bos_token = common_token_to_piece(vocab, llama_vocab_bos(vocab), true);
auto eos_token = common_token_to_piece(vocab, llama_vocab_eos(vocab), true);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this handle missing BOS/EOS tokens?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah we should, thanks! Defaulting to empty string + emitting warning if they're (likely) referenced from the jinja template.

std::string default_template_src = chat_template_override;
std::string tool_use_template_src = chat_template_override;
if (chat_template_override.empty()) {
Expand Down
Loading
0