10000 [FEATURE] Implement native async iterator support · Issue #83 · strands-agents/sdk-python · GitHub
[go: up one dir, main page]
Skip to content

[FEATURE] Implement native async iterator support #83

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
awsarron opened this issue May 23, 2025 · 1 comment
Open

[FEATURE] Implement native async iterator support #83

awsarron opened this issue May 23, 2025 · 1 comment
Assignees
@pgrayy
Labels
enhancement New feature or request

Comments

@awsarron
Copy link
Member

Problem Statement

The Agent class provides the stream_async function that returns an AsyncIterator:

sdk-python/src/strands/agent/agent.py

Line 356 in 77f5fa7

async def stream_async(self, prompt: str, **kwargs: Any) -> AsyncIterator[Any]:

The current stream_async implementation calls _run_loop and uses threads with a queue to yield streamed data events from the Strands Agents event loop:

sdk-python/src/strands/agent/agent.py

Line 423 in 77f5fa7

def _run_loop(

sdk-python/src/strands/agent/agent.py

Lines 409 to 421 in 77f5fa7

thread = Thread(target=target_callback, daemon=True)
thread.start()
try:
while True:
item = await queue.get()
if item == _stop_event:
break
if isinstance(item, BaseException):
raise item
yield item
finally:
thread.join()

The problem is that this threading implementation doesn't work with Python features like thread local contextvars. Certain Python libraries (e.g. https://github.com/UKGovernmentBEIS/inspect_ai) use thread local context vars and require them to function correctly. The only reasonable solution for correctly supporting concurrency within a single thread is async.

This issue is to propose, discuss, and refine a true async implementation of the Agent.stream_async function and Strands Agent event loop.

Proposed Solution

No response

Use Case

  • Thread local contextvars
  • A more maintainable implementation of async agent requests
  • Allows for future enhancements that enable streamed tool results

Alternatives Solutions

No response

Additional Context

No response
@awsarron awsarron added the enhancement New feature or request label May 23, 2025
@zastrowm
Copy link
Member
zastrowm commented May 23, 2025 •
edited
Loading

Another use case is providing caller control over the lifecycle of an agent request. Right now all events are enqueued eagerly - meaning that if even when callers stop iterating, the agent continues until it hits a natural stopping point. In theory, an async iterator could/would be pull-based, allowing the agent to suspend if the caller stops iterating which should enable more advanced use cases or control.

@awsarron awsarron mentioned this issue May 26, 2025
Open
@zastrowm zastrowm changed the title [FEATURE] Async without threads [Epic] Implement Native Async Iterator support without threads May 27, 2025
@zastrowm zastrowm changed the title [Epic] Implement Native Async Iterator support without threads [FEATURE] Async without threads May 27, 2025
@zastrowm zastrowm changed the title [FEATURE] Async without threads [EPIC] Implement native async iterator support May 27, 2025
@zastrowm zastrowm changed the title [EPIC] Implement native async iterator support Implement native async iterator support May 27, 2025
@zastrowm zastrowm changed the title Implement native async iterator support [FEATURE] Implement native async iterator support May 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pgrayy pgrayy

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@zastrowm @awsarron @pgrayy
0