gh-97556: Fix crash on null characters in string literal parser #97577
Conversation
|
Most changes to Python require a NEWS entry. Please add it using the blurb_it web app or the blurb command-line tool. |
|
All commit authors signed the Contributor License Agreement. |
|
Most changes to Python require a NEWS entry. Please add it using the blurb_it web app or the blurb command-line tool. |
Parser/string_parser.c
Outdated
| if (len > INT_MAX) { | ||
| PyErr_SetString(PyExc_OverflowError, "string to parse is too long"); | ||
| return -1; |
There was a problem hiding this comment.
Maybe move this check to the top? I really don't care about the case where the string including quotes is just over INT_MAX but after stripping quotes it fits.
There was a problem hiding this comment.
Can do. Also, I just realized I copied the wrong issue number in the title (I meant #97556): is it enough to change the title, or should I create a whole new pull request?
|
I'm having trouble including a unit test due to a similar issue, #96670. Null characters raise errors when they are included in source code that is parsed using ast.parse, or which is imported from another file. So I have a test case for this which can be run directly, but it cannot be imported as part of the standard test suite. Any suggestions on how to work around that? |
You can write the test program to a file with a .py extension, manipulate sys.path so you can import that file, and attempt to import it. I think there are a few other tests that do that. E.g. look in test_bdb.py for import_helper. |
An easier way is to write the test code as a string and use the cpython/Lib/test/test_cmd_line_script.py Lines 220 to 226 in 68c46ae |
The _PyPegen_parsestr function was crashing on null characters inside of string literals due to a call to strlen, which treated it as a null terminator.
This change removes that call, which was actually redundant: the length was already calculated by the caller function and can be retrieved using PyBytes_AsStringAndSize. This has the added benefit of reducing the number of times the length gets calculated.