8000 Add a fuzzer for `Py_CompileStringExFlags` by bradlarsen · Pull Request #111721 · python/cpython · GitHub
[go: up one dir, main page]
Skip to content

Add a fuzzer for Py_CompileStringExFlags #111721

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
Dec 10, 2023
Merged

Add a fuzzer for Py_CompileStringExFlags #111721

merged 6 commits into from
Dec 10, 2023

Conversation

bradlarsen
Copy link
Contributor

This pull request adds a new fuzz target for the Py_CompileStringExFlags function in the "Very High Level Layer" C API. This function is essentially a C entry point to the Python compile builtin.

This pull request also includes a handful of short, mostly nonsensical Python files written to demonstrate a number of syntactic forms that appear in Python.

Also included is a dictionary of most Python syntactic elements, often included with trailing spaces, to aid the fuzzers in producing inputs that have a chance of being syntactically valid.

Future work could include writing a structure-aware input mutator, which would take syntactically valid Python programs as input, and produce syntactically-valid mutated Python programs as outputs. This would likely get much deeper coverage from the fuzzers.

CC @alex @gpshead

@bradlarsen
Copy link
Contributor Author

Note that this new fuzz target already uncovers assertion failures in the CPython parser after running for a few minutes under honggfuzz. I probably should open an issue for that?

@sobolevn
Copy link
Member
sobolevn commented Nov 4, 2023

Yes, please :)

@bradlarsen
Copy link
Contributor Author

In particular, I'm seeing two assertion failures:

fuzz_pycompile: Parser/pegen_errors.c:286: PyObject *get_error_line_from_tokenizer_buffers(Parser *, Py_ssize_t): Assertion `new_line != NULL && new_line + 1 < buf_end' failed.

fuzz_pycompile: Objects/call.c:342: PyObject *_PyObject_Call(PyThreadState *, PyObject *, PyObject *, PyObject *): Assertion `!_PyErr_Occurred(tstate)' failed.

I will collect the relevant inputs and stack traces and open an issue for each one.

@alex
Copy link
Member
alex commented Nov 17, 2023

Were those issues filed?

@bradlarsen
Copy link
Contributor Author

@alex no, I haven't filed the issues yet. Let me give that another look this weekend!

@alex
Copy link
Member
alex commented Nov 17, 2023 via email

This was referenced Nov 25, 2023
Closed
Closed
@bradlarsen
Copy link
Contributor Author

@alex @sobolevn @gpshead I opened two issues for the crashes, which are reproducible from pure Python code:

#112387
#112388

alex
alex reviewed Nov 26, 2023
View reviewed changes
alex
alex reviewed Dec 10, 2023
View reviewed changes
Copy link
Member
@alex alex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two small style suggestions, otherwise LGTM.

@bradlarsen @alex
Apply suggestions from code review
3d8147e 
Co-authored-by: Alex Gaynor <alex.gaynor@gmail.com>
@bradlarsen
Copy link
Contributor Author

Thank you @alex!

@alex alex merged commit eb27c9a into python:main Dec 10, 2023
@alex
Copy link
Member
alex commented Dec 10, 2023

FYI, you may be able to apply for a Patch Reward from Google for the fuzzers you've contributed: https://bughunters.google.com/open-source-security/patch-rewards

aisk pushed a commit to aisk/cpython that referenced this pull request Feb 11, 2024
@bradlarsen @aisk
Add a fuzzer for Py_CompileStringExFlags (python#111721)
7ecaea0
Glyphack pushed a commit to Glyphack/cpython that referenced this pull request Sep 2, 2024
@bradlarsen @Glyphack
Add a fuzzer for Py_CompileStringExFlags (python#111721)
01241cf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alex alex alex approved these changes

Assignees
No one assigned
Labels
skip issue skip news
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@bradlarsen @sobolevn @alex
0