8000 [3.6] bpo-45001: Make email date parsing more robust against malformed input (GH-27946) by miss-islington · Pull Request #27976 · python/cpython · GitHub
[go: up one dir, main page]
Skip to content

[3.6] bpo-45001: Make email date parsing more robust against malformed input (GH-27946) #27976

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Aug 30, 2021
Merged

[3.6] bpo-45001: Make email date parsing more robust against malformed input (GH-27946) #27976

merged 1 commit into from
Aug 30, 2021

Conversation

miss-islington
Copy link
Contributor
@miss-islington miss-islington commented Aug 26, 2021 •
edited by bedevere-bot
Loading

Various date parsing utilities in the email module, such as
email.utils.parsedate(), are supposed to gracefully handle invalid
input, typically by raising an appropriate exception or by returning
None.

The internal email._parseaddr._parsedate_tz() helper used by some of
these date parsing routines tries to be robust against malformed input,
but unfortunately it can still crash ungracefully when a non-empty but
whitespace-only input is passed. This manifests as an unexpected
IndexError.

In practice, this can happen when parsing an email with only a newline
inside a ‘Date:’ header, which unfortunately happens occasionally in the
real world.

Here's a minimal example:

$ python
Python 3.9.6 (default, Jun 30 2021, 10:22:16)
[GCC 11.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import email.utils
>>> email.utils.parsedate('foo')
>>> email.utils.parsedate(' ')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.9/email/_parseaddr.py", line 176, in parsedate
    t = parsedate_tz(data)
  File "/usr/lib/python3.9/email/_parseaddr.py", line 50, in parsedate_tz
    res = _parsedate_tz(data)
  File "/usr/lib/python3.9/email/_parseaddr.py", line 72, in _parsedate_tz
    if data[0].endswith(',') or data[0].lower() in _daynames:
IndexError: list index out of range

The fix is rather straight-forward: guard against empty lists, after
splitting on whitespace, but before accessing the first element.
(cherry picked from commit 989f6a3)

Co-authored-by: wouter bolsterlee wouter@bolsterl.ee

https://bugs.python.org/issue45001

@wbolster @miss-islington
bpo-45001: Make email date parsing more robust against malformed input (
3e66263 
pythonGH-27946)

Various date parsing utilities in the email module, such as
email.utils.parsedate(), are supposed to gracefully handle invalid
input, typically by raising an appropriate exception or by returning
None.

The internal email._parseaddr._parsedate_tz() helper used by some of
these date parsing routines tries to be robust against malformed input,
but unfortunately it can still crash ungracefully when a non-empty but
whitespace-only input is passed. This manifests as an unexpected
IndexError.

In practice, this can happen when parsing an email with only a newline
inside a ‘Date:’ header, which unfortunately happens occasionally in the
real world.

Here's a minimal example:

    $ python
    Python 3.9.6 (default, Jun 30 2021, 10:22:16)
    [GCC 11.1.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import email.utils
    >>> email.utils.parsedate('foo')
    >>> email.utils.parsedate(' ')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib/python3.9/email/_parseaddr.py", line 176, in parsedate
        t = parsedate_tz(data)
      File "/usr/lib/python3.9/email/_parseaddr.py", line 50, in parsedate_tz
        res = _parsedate_tz(data)
      File "/usr/lib/python3.9/email/_parseaddr.py", line 72, in _parsedate_tz
        if data[0].endswith(',') or data[0].lower() in _daynames:
    IndexError: list index out of range

The fix is rather straight-forward: guard against empty lists, after
splitting on whitespace, but before accessing the first element.
(cherry picked from commit 989f6a3)

Co-authored-by: wouter bolsterlee <wouter@bolsterl.ee>
@bedevere-bot bedevere-bot mentioned this pull request Aug 26, 2021
Merged
@miss-islington
Copy link
Contributor Author

@wbolster and @ambv: Status check is done, and it's a success ✅ .

@miss-islington
Copy link
Contributor Author

@wbolster and @ambv: Status check is done, and it's a failure ❌ .

@wbolster
Copy link
Contributor

the CI failure is unrelated, and seems caused by flaky behaviour. maybe someone can hit a retry button?

https://dev.azure.com/Python/8e426817-76c0-4b99-ba9e-a48a1e4bd5db/_apis/build/builds/86573/logs/50

2021-08-26T14:56:33.5108651Z ======================================================================
2021-08-26T14:56:33.5109279Z FAIL: test_fontlist_key (idlelib.idle_test.test_configdialog.FontPageTest)
2021-08-26T14:56:33.5110407Z ----------------------------------------------------------------------
2021-08-26T14:56:33.5111094Z Traceback (most recent call last):
2021-08-26T14:56:33.5111742Z   File "/home/vsts/work/1/s/Lib/idlelib/idle_test/test_configdialog.py", line 104, in test_fontlist_key
2021-08-26T14:56:33.5112370Z     self.assertNotEqual(down_font, font)
2021-08-26T14:56:33.5113206Z AssertionError: 'DejaVu Sans' == 'DejaVu Sans'
2021-08-26T14:56:33.5113627Z 
2021-08-26T14:56:33.5114438Z ----------------------------------------------------------------------
2021-08-26T14:56:33.5114868Z 
2021-08-26T14:56:33.5115313Z Ran 465 tests in 5.782s
2021-08-26T14:56:33.5115610Z 
2021-08-26T14:56:33.5116059Z FAILED (failures=1, skipped=1)

@ambv ambv requested a review from ned-deily August 26, 2021 15:50
@ambv
Copy link
Contributor
ambv commented Aug 26, 2021

@ned-deily, this is a bug fix with security-related connotations. I merged it to 3.8 so we might as well get it to 3.7 and 3.6.

@miss-islington
Copy link
Contributor Author

@wbolster and @ambv: Status check is done, and it's a failure ❌ .

@ambv
Copy link
Contributor 8000
ambv commented Aug 26, 2021

Problem persists after a re-run but is pretty clearly unrelated.

@ned-deily ned-deily merged commit da9d6c5 into python:3.6 Aug 30, 2021
@miss-islington miss-islington deleted the backport-989f6a3-3.6 branch August 30, 2021 18:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ned-deily ned-deily ned-deily approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@miss-islington @wbolster @ambv @ned-deily @the-knights-who-say-ni @bedevere-bot
0