Closed
Description
Name and Version
$ llama-server --version
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
Device 0: NVIDIA RTX A6000, compute capability 8.6, VMM: yes
version: 5129 (526739b)
built with cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 for x86_64-linux-gnu
Operating systems
Linux
GGML backends
CUDA
Hardware
NVIDIA RTX A6000, compute capability 8.6, VMM: yes
Models
Llama-4-Scout-17B-16E-Instruct
Problem description & steps to reproduce
When I invoke OpenAI api with image, it got 500 error.
First Bad Commit
500: Failed to parse messages: Unsupported content part type: "image_url"; messages = [
{
"role": "user",
"content": [
{
"type": "text",
"text": "PLS desc this pic?"
},
{
"type": "image_url",
"image_url": {
"url": "data:image/png;base64,iVBORw0KGgoxxxxxxxxTkSuQmCC"
}
}
]
}
]
Relevant log output
got exception: {"code":500,"message":"Failed to parse messages: Unsupported content part type: \"image_url\"; messages = [\n {\n \"role\": \"user\",\n \"content\": [\n {\n \"type\": \"text\",\n \"text\": \"PLS desc this pic?\"\n },\n {\n \"type\": \"image_url\",\n \"image_url\": {\n \"url\": \"data:image/xxxxx\"\n }\n }\n ]\n }\n]","type":"server_error"}
srv log_server_r: request: POST /v1/chat/completions 10.13.23.105 500