[go: up one dir, main page]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: prqlc compile --debug-log #4646

Merged
merged 4 commits into from
Jun 24, 2024
Merged

feat: prqlc compile --debug-log #4646

merged 4 commits into from
Jun 24, 2024

Conversation

aljazerzen
Copy link
Member
@aljazerzen aljazerzen commented Jun 21, 2024

Adds a framework for capturing debug information during the compilation process.

First stab at #4636

The result can be serialized into JSON or rendered into HTML (image below). Current display is not much better than plain terminal output, but this has potential for better visualization of what is going on in the compiler.

image

prqlc/prqlc/src/debug.rs Outdated Show resolved Hide resolved
Copy link
Member
@max-sixty max-sixty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks! This direction be really helpful I think.


Not a strong view at all but two other approaches to the logging infra:

  • just write structured logs in JSON to stderr / possibly an optional log life
  • then have a function to convert to a different representation; e.g. HTML
    ...that's a bit more modular (though also requires some serialization from serde)
  • A specific command like prqlc debug dump foo.prql which dumps everything — my guess is that much of the time we'd want to drill down on a specific query, rather than always having some exhaust

...but as above, not a strong view

@aljazerzen
Copy link
Member Author
aljazerzen commented Jun 21, 2024

Actually, this PR is combination of the two things you've suggested.

  1. --debug-log=my_log.json will create a JSON file with all of the AST reprs emitted. This file could probably be read back and rendered to the HTML output,
  2. the debug log should contain basically all intermediate information that compiler has. The idea is to replace various commands we have with these "debug log entries", so a normal compile invocation produces all that you need. ATM, we are 80% there.

@max-sixty
Copy link
Member

Actually, this PR is combination of the two things you've suggested.

Yes! My musing was whether splitting it up made sense (but really just a musing and maybe not that useful..)

@aljazerzen
Copy link
Member Author

More refinement:
image

@aljazerzen aljazerzen merged commit 8c6a851 into main Jun 24, 2024
38 checks passed
@aljazerzen aljazerzen deleted the debug-log branch June 24, 2024 12:57
@max-sixty
Copy link
Member

I tried this out — really really cool! Very impressive you could pull together a UI so quickly!


FWIW when I try I don't get the highlights across the different stages (like in your query) unless I select the whole query, but possibly that's because the spans aren't serialized yet.


Other small thoughts (obv you are the owner and so only consider these if they're useful, am strongly not trying to say these are necessarily good)

  • integrate with playground? Maybe that's more work than it saves though. Mostly similar goals (would want a check-box to show / hide, similar to what we have with outputting PL & RQ now)
  • minor but it would be v nice to have the stages in each column, so we can see across them — i.e. "what RQ comes from this expression"

@aljazerzen
Copy link
Member Author

integrate with playground? Maybe that's more work than it saves though. Mostly similar goals (would want a check-box to show / hide, similar to what we have with outputting PL & RQ now)

Yes, this is possible to do. We just need to expose the --debug-log argument to JS bindings somehow. We'd probably change the type signature, so the debug log is returned instead of written into a file.


minor but it would be v nice to have the stages in each column, so we can see across them — i.e. "what RQ comes from this expression"

This is easy to do! Newer versions are somewhat easier to parse visually, so it might not even be needed, but yes, can be done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants