10000 JSON schema conversion: ⚡️ faster repetitions, min/maxLength for strings, cap number length by ochafik · Pull Request #6555 · ggml-org/llama.cpp · GitHub
[go: up one dir, main page]
Skip to content

JSON schema conversion: ⚡️ faster repetitions, min/maxLength for strings, cap number length #6555

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Apr 12, 2024
+2,326 −1,907
Merged

JSON schema conversion: ⚡️ faster repetitions, min/maxLength for strings, cap number length #6555

Changes from 1 commit
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
2148f24
json: rename python schema converter to make import easier
Apr 8, 2024
f771a8f
server: skip null json_schema / grammar fields
Apr 8, 2024
159b883
json: deps management for primitive rules (+ allow null values)
Apr 8, 2024
a59e943
json: optimize repetitions for minItems/maxItems and regexps: `a{,3}`…
Apr 8, 2024
07163fb
grammars: add troubleshooting section to readme
Apr 8, 2024
dcf5d32
json: cap length of numbers to 15 digits before/after decimal point
Apr 8, 2024
181f984
json: unify all repetition code (w/ or w/o sep)
Apr 8, 2024
de4e60e
json: support string minLength/maxLength
Apr 8, 2024
6c885dc
server+json: update server/README w/ result_format
Apr 8, 2024
3c81e94
nits
Apr 8, 2024
67a5184
json: fix type error w/ python 3.8
Apr 8, 2024
9c33ee9
json: fix server/README (json_schema in /completion vs. result_format…
Apr 9, 2024
ed13d47
json: simplify DOT `{"type": "string", "pattern": "^.$"}`
Apr 12, 2024
958bdda
Merge remote-tracking branch 'origin/master' into json-faster-repetit…
Apr 12, 2024
64e3059
json: remove recursion in opt_repetitions (avoids Python stack overflow)
Apr 12, 2024
ba90d5b
json: rm dead code
Apr 12, 2024
dfd4eb3
json: rm useless assert & ggml.h import
Apr 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
json: fix server/README (json_schema in /completion vs. result_format… 
8000
 in /v1/chat/completions)
  • Loading branch information
Olivier Chafik committed Apr 9, 2024
commit 9c33ee99302caac14c79f12c43e7a61462dc0730
4 changes: 3 additions & 1 deletion examples/server/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,7 @@ node index.js

`grammar`: Set grammar for grammar-based sampling. Default: no grammar

`response_format`: Set the response format. Only supports JSON (e.g. `{"type": "json_object"}`), optionally with a schema (e.g. `{"type": "json_object", "schema": {"type": "string", "minLength": 10, "maxLength": 100}}`). See [test-json-schema-to-grammar.cpp](../../tests/test-json-schema-to-grammar.cpp). Default: no response format.
`json_schema`: Set a JSON schema for grammar-based sampling (e.g. `{"items": {"type": "string"}, "minItems": 10, "maxItems": 100}` of a list of strings, or `{}` for any JSON). See [tests](../../tests/test-json-schema-to-grammar.cpp) for supported features. Default: no JSON schema.

`seed`: Set the random number generator (RNG) seed. Default: `-1`, which is a random seed.

Expand Down Expand Up @@ -368,6 +368,8 @@ Notice that each `probs` is an array of length `n_probs`.

See [OpenAI Chat Completions API documentation](https://platform.openai.com/docs/api-reference/chat). While some OpenAI-specific features such as function calling aren't supported, llama.cpp `/completion`-specific features such as `mirostat` are supported.

The `response_format` parameter supports both plain JSON output (e.g. `{"type": "json_object"}`) and schema-constrained JSON (e.g. `{"type": "json_object", "schema": {"type": "string", "minLength": 10, "maxLength": 100}}`), similar to other OpenAI-inspired API providers.

*Examples:*

You can use either Python `openai` library with appropriate checkpoints:
Expand Down
0