8000 [Hackweek] Add explain plan to db spans. by antonpirker · Pull Request #2315 · getsentry/sentry-python · GitHub
[go: up one dir, main page]
Skip to content

[Hackweek] Add explain plan to db spans. #2315

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 16 commits into from
Oct 2, 2023
Merged

[Hackweek] Add explain plan to db spans. #2315

merged 16 commits into from
Oct 2, 2023

Conversation

antonpirker
Copy link
Member
@antonpirker antonpirker commented Aug 21, 2023 •
edited
Loading

This is a proof of concept of adding the explain plan to db spans. The explain plan will be added to the span in the db.explain_plan data item.

There is a cache to make sure that the explain plan for each db query is only executed ever X seconds and there is also a max number of elements that are cached. To make sure we do not put to much strain on CPU or memory.

Usage:

sentry_sdk.init(
    dsn="...",
    _experiments={
        "attach_explain_plans": {
            "explain_cache_size": 1000,  # Run explain plan for the 1000 most run queries
            "explain_cache_timeout_seconds": 60 * 60 * 24,  # Run the explain plan for each statement only every 24 hours
            "use_explain_analyze": True,  # Run "explain analyze" instead of only "explain"
        }
    }

If you then look at the db span in Sentry.io it looks like this:

Screenshot 2023-08-21 at 15 14 32

@antonpirker antonpirker marked this pull request as ready for review September 29, 2023 12:23
@antonpirker antonpirker self-assigned this Sep 29, 2023
sentrivana
sentrivana reviewed Sep 29, 2023
View reviewed changes
Copy link
Contributor
@sentrivana sentrivana left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One nit: if we add the new experiment to

sentry-python/sentry_sdk/consts.py

Lines 35 to 49 in 692c0e9

Experiments = TypedDict(
"Experiments",
{
"max_spans": Optional[int],
"record_sql_params": Optional[bool],
# TODO: Remove these 2 profiling related experiments
"profiles_sample_rate": Optional[float],
"profiler_mode": Optional[ProfilerMode],
"otel_powered_performance": Optional[bool],
"transport_zlib_compression_level": Optional[int],
"enable_metrics": Optional[bool],
"before_emit_metric": Optional[Callable[[str, MetricTags], bool]],
},
total=False,
)
we should get nicer code completion I assume.

And one thing regarding the cache. If I read it right, once a key is in the cache, it'll never get deleted if it's never accessed again. Is this what we want? Let's say my app does 50 one-time SELECTs at startup which I'm not that interested in, but they'll then occupy the cache, so this will always apply for any future SELECTS

    if len(EXPLAIN_CACHE.keys()) >= explain_cache_size:
        return False

and we won't run any additional explains.

@antonpirker
Copy link
Member Author

You are right about the cache! Good catch. (how did I pass our coding interviews? ;-) )
And about the experiments thing: right! will add it to the TypedDict.

sentrivana
sentrivana approved these changes Oct 2, 2023
View reviewed changes
Copy link
Contributor
@sentrivana sentrivana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, left a couple of comments.

antonpirker and others added 4 commits October 2, 2023 10:48
@antonpirker antonpirker merged commit 2faf03d into master Oct 2, 2023
@antonpirker antonpirker deleted the antopirker/hackweek branch October 2, 2023 09:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sentrivana sentrivana sentrivana approved these changes

Assignees

@antonpirker antonpirker

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@antonpirker @sentrivana
0