8000 server : fix first message identification by doringeman · Pull Request #13634 · ggml-org/llama.cpp · GitHub
[go: up one dir, main page]
Skip to content

server : fix first message identification #13634

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 21, 2025
Merged

server : fix first message identification #13634

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
98fa621
server : fix first message identification
doringeman May 19, 2025
1e81756
server : Fix checks for first role message for stream=True
doringeman May 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
server : fix first message identification 
When using the OpenAI SDK (https://github.com/openai/openai-node/blob/master/src/lib/ChatCompletionStream.ts#L623-L626) we noticed that the expected assistant role is missing in the first streaming message. Fix this by correctly checking for the first message.

Co-authored-by: Piotr Stankiewicz <piotr.stankiewicz@docker.com>
Signed-off-by: Dorin Geman <dorin.geman@docker.com>
  • Loading branch information
@doringeman @p1-0tr
doringeman and p1-0tr committed May 19, 2025
commit 98fa621ef9f179057580147e85e6e9ff09bbff16
2 changes: 1 addition & 1 deletion tools/server/server.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -951,7 +951,7 @@ struct server_task_result_cmpl_partial : server_task_result {
}

json to_json_oaicompat_chat() {
bool first = n_decoded == 0;
bool first = n_decoded == 1;
std::time_t t = std::time(0);
json choices;

Expand Down
Loading
0