8000 GenAI Utils | Inference Type and Span Creation by keith-decker · Pull Request #3768 · open-telemetry/opentelemetry-python-contrib · GitHub
[go: up one dir, main page]
Skip to content

GenAI Utils | Inference Type and Span Creation #3768

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 43 commits into from
Oct 10, 2025
Merged

GenAI Utils | Inference Type and Span Creation #3768

merged 43 commits into from
Oct 10, 2025

Conversation

keith-decker
Copy link
Contributor

Description

The GenAI Utils package will include boilerplate and helpers to standardize instrumentation for Generative AI.
This PR provides APIs to minimize the work needed to instrument genai libraries. It provides helpers for starting and stopping LLM invocations, it generates spans aligned with semconvs.

Metrics and Events to come on future PRs

Type of change

  • New feature (non-breaking change which adds functionality)

How Has This Been Tested?

  • Basic unit test for starting and stopping LLM - Spans
  • Unit test for parent/child invocations

Does This PR Require a Core Repo Change?

  • Yes. - Link to PR:
  • No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

  • Followed the style guidelines of this project
  • Changelogs have been updated
  • Unit tests have been added
  • Documentation has been updated

@keith-decker keith-decker requested a review from a team as a code owner September 18, 2025 14:54
@keith-decker
Copy link
Contributor Author

A cleaned up version of: #3732

  • Flattened LLMInvocation to not use dictionary keys
  • Removed the parent run_id, use OTEL context to manage parent/child relationships on spans
  • Return the run_id on creation if one is not provided, in order to specify which span to end on finish

@keith-decker keith-decker mentioned this pull request Sep 18, 2025
Closed
9 tasks
@aabmass
Copy link
Member
aabmass commented Sep 18, 2025

@keith-decker thing I was mentioning in the SIG:
Looks like you already figured out the context manager thing but figured I'd share what I wrote:

opentelemetry-python-contrib/instrumentation-genai/opentelemetry-instrumentation-vertexai/src/opentelemetry/instrumentation/vertexai/patch.py

Lines 101 to 107 in ee947ce

def _with_instrumentation(
self,
instance: client.PredictionServiceClient
| client_v1beta1.PredictionServiceClient,
args: Any,
kwargs: Any,
):

It's used for both sync and async wrapper, and would work for streaming as well.

DylanRussell
DylanRussell reviewed Sep 18, 2025
View reviewed changes
DylanRussell
DylanRussell reviewed Sep 25, 2025
View reviewed changes
DylanRussell
DylanRussell reviewed Sep 30, 2025
View reviewed changes
@DylanRussell
Copy link
Contributor

This looks good to me now. Just added a few comments

@keith-decker keith-decker requested a review from aabmass October 3, 2025 16:54
@keith-decker
Copy link
Contributor Author

@aabmass genai-file upload on python 3.9 seems to be flakey. I've seen it fail occasionally, and I don't have the ability to restart the test.

@aabmass
Copy link
Member
aabmass commented Oct 9, 2025

Sorry about that tracking here #3812 (no idea why)

aabmass
aabmass approved these changes Oct 9, 2025
View reviewed changes
Copy link
Member
@aabmass aabmass left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly nits, thanks for your patience!

@aabmass aabmass merged commit e3d3817 into open-telemetry:main Oct 10, 2025
638 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pmcollins pmcollins pmcollins left review comments

@aabmass aabmass aabmass approved these changes

@DylanRussell DylanRussell DylanRussell approved these changes

+2 more reviewers

@zhirafovod zhirafovod zhirafovod approved these changes

@nagkumar91 nagkumar91 nagkumar91 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@DylanRussell DylanRussell

@keith-decker keith-decker

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@keith-decker @aabmass @DylanRussell @pmcollins @zhirafovod @nagkumar91
0