Description
** Please make sure you read the contribution guide and file the issues in the right place. **
Contribution guide.
Describe the bug
Using StreamingMode.SSE and with LiteLLM/OpenAI model - if the model responds with multiple tool calls, the code in LiteLlm.generate_content_async and _model_response_to_chunk doesn't consider the index (choices[0].delta.tool_calls[0].index) and appends function names and arguments.
Toy example with two tools - today() and name()
For example, the response.completion_stream.response.content:
{"id":"chatcmpl-BSI52NJOz2cVKe17YQN9zjn5wudKd","object":"chat.completion.chunk","created":1746082384,"model":"gpt-4o-2024-11-20","service_tier":"default","system_fingerprint":"fp_3bdddbcbe8","usage":null,"choices":[{"index":0,"delta":{"role":"assistant","content":null},"logprobs":null,"finish_reason":null}]}
{"id":"chatcmpl-BSI52NJOz2cVKe17YQN9zjn5wudKd","object":"chat.completion.chunk","created":1746082384,"model":"gpt-4o-2024-11-20","service_tier":"default","system_fingerprint":"fp_3bdddbcbe8","usage":null,"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"call_SjCn2tyqM31QMmZZmd0uoqhI","type":"function","function":{"name":"name","arguments":""}}]},"logprobs":null,"finish_reason":null}]}
{"id":"chatcmpl-BSI52NJOz2cVKe17YQN9zjn5wudKd","object":"chat.completion.chunk","created":1746082384,"model":"gpt-4o-2024-11-20","service_tier":"default","system_fingerprint":"fp_3bdddbcbe8","usage":null,"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{}"}}]},"logprobs":null,"finish_reason":null}]}
{"id":"chatcmpl-BSI52NJOz2cVKe17YQN9zjn5wudKd","object":"chat.completion.chunk","created":1746082384,"model":"gpt-4o-2024-11-20","service_tier":"default","system_fingerprint":"fp_3bdddbcbe8","usage":null,"choices":[{"index":0,"delta":{"tool_calls":[{"index":1,"id":"call_vN1NqEUSYwlU0QYwdEwiIa0X","type":"function","function":{"name":"today","arguments":""}}]},"logprobs":null,"finish_reason":null}]}
{"id":"chatcmpl-BSI52NJOz2cVKe17YQN9zjn5wudKd","object":"chat.completion.chunk","created":1746082384,"model":"gpt-4o-2024-11-20","service_tier":"default","system_fingerprint":"fp_3bdddbcbe8","usage":null,"choices":[{"index":0,"delta":{"tool_calls":[{"index":1,"function":{"arguments":"{}"}}]},"logprobs":null,"finish_reason":null}]}
{"id":"chatcmpl-BSI52NJOz2cVKe17YQN9zjn5wudKd","object":"chat.completion.chunk","created":1746082384,"model":"gpt-4o-2024-11-20","service_tier":"default","system_fingerprint":"fp_3bdddbcbe8","choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"tool_calls"}],"usage":null}
{"id":"chatcmpl-BSI52NJOz2cVKe17YQN9zjn5wudKd","object":"chat.completion.chunk","created":1746082384,"model":"gpt-4o-2024-11-20","service_tier":"default","system_fingerprint":"fp_3bdddbcbe8","choices":[],"usage":{"prompt_tokens":441,"completion_tokens":38,"total_tokens":479,"prompt_tokens_details":{"cached_tokens":0,"audio_tokens":0},"completion_tokens_details":{"reasoning_tokens":0,"audio_tokens":0,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}}}The final function call is calculated to be:
nametoday
{}{}
Which then leads to JSONDecodeError in _message_to_generate_content_response
To Reproduce
Steps to reproduce the behavior:
- Install ADK, LiteLLM
- Run an agent with multiple tools and a prompt such that LLM responds with multiple tool calls in the same response. Use StreamingMode.SSE
- JSON Decoding error from LiteLLM since it is expecting a valid JSON while reading tool call arguments.
Expected behavior
Parsing of tool call deltas in SSE should handle multiple tool calls and function names, arguments should be calculated taking index into consideration
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
- OS: macOS
- Python version(python -V): 3.13
- ADK version(pip show google-adk): 0.3.0
- LiteLLM version 1.67.2
Additional context
Usage is via LlmAgent and function tools, with OpenAI model using LiteLlm