server : fix first message identification

doringeman · p1-0tr · doringeman · commit 98fa621ef9f1 · 2025-05-19T14:43:54.000+03:00
When using the OpenAI SDK (https://github.com/openai/openai-node/blob/master/src/lib/ChatCompletionStream.ts#L623-L626) we noticed that the expected assistant role is missing in the first streaming message. Fix this by correctly checking for the first message. Co-authored-by: Piotr Stankiewicz <piotr.stankiewicz@docker.com> Signed-off-by: Dorin Geman <dorin.geman@docker.com>
diff --git a/tools/server/server.cpp b/tools/server/server.cpp
@@ -951,7 +951,7 @@ struct server_task_result_cmpl_partial : server_task_result {
     }
 
     json to_json_oaicompat_chat() {
-        bool first = n_decoded == 0;
+        bool first = n_decoded == 1;
         std::time_t t = std::time(0);
         json choices;
 

Original file line number	Diff line number	Diff line change
`@@ -951,7 +951,7 @@ struct server_task_result_cmpl_partial : server_task_result {`
`951`	`951`	`}`
`952`	`952`
`953`	`953`	`json to_json_oaicompat_chat() {`
`954`		`- bool first = n_decoded == 0;`
	`954`	`+ bool first = n_decoded == 1;`
`955`	`955`	`std::time_t t = std::time(0);`
`956`	`956`	`json choices;`
`957`	`957`