Enhance per step profiling with more request-level metrics by Weili-0234 · Pull Request #25 · ThunderAgent-org/ThunderAgent

Weili-0234 · 2026-03-04T00:38:29Z

Summary

This PR enhances ThunderAgent profiling with token-level step metrics.

Per-step CSV now records:
- prompt_tokens
- completion_tokens
- cached_tokens
Profile API outputs now include:
- avg_prompt_tokens
- avg_completion_tokens
- avg_cached_tokens
Usage parsing/callback flow is extended to propagate these token fields end-to-end.
Existing finalize behavior is preserved (token stats are finalized when usage.total_tokens is available).

Verified locally with Qwen/Qwen3-0.6B on both vLLM and SGLang backends

In both cases, the new token fields were correctly written to step_profiles.csv and reflected in profile API averages.

…es.csv lines

Weili-0234 added 2 commits March 3, 2026 16:16

add prompt_tokens, cached_tokens and completion_tokens in step_profil…

7d940a3

…es.csv lines

book keep enhanced profiling

ffc1182