8000 Fix test URLs containing whitespaces by skazi0 · Pull Request #2025 · urllib3/urllib3 · GitHub
[go: up one dir, main page]

Skip to content

Fix test URLs containing whitespaces #2025

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

skazi0
Copy link
@skazi0 skazi0 commented Oct 13, 2020

They triggered CVE-2019-9740 checks added in python here [0].
The problematic test should fail because of invalid source address but it
failed earlier because of invalid request URL. Request URLs contained string
representation of the tested source address which can contain whitespaces.
E.g. "/source_address?('192.0.2.255', 0)"
The source addresses seem to be there only for information and were added as
part of [1]. Removing them from the request URL makes the tests pass again.

[0] python/cpython#12755
[1] #703

They triggered CVE-2019-9740 checks added in python here [0].
The problematic test should fail because of invalid source address but it
failed earlier because of invalid request URL. Request URLs contained string
representation of the tested source address which can contain whitespaces.
E.g. "/source_address?(\'192.0.2.255\', 0)"
The source addresses seem to be there only for information and were added as
part of [1]. Removing them from the request URL makes the tests pass again.

[0] python/cpython#12755
[1] urllib3#703
@codecov
Copy link
codecov bot commented Oct 13, 2020

Codecov Report

Merging #2025 into master will increase coverage by 0.04%.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff             @@
##           master     #2025      +/-   ##
===========================================
+ Coverage   99.95%   100.00%   +0.04%     
===========================================
  Files          25        25              
  Lines        2294      2294              
===========================================
+ Hits         2293      2294       +1     
+ Misses          1         0       -1     
Impacted Files Coverage Δ
src/urllib3/contrib/_appengine_environ.py 100.00% <0.00%> (+10.00%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3308d65...864c839. Read the comment docs.

@sethmlarson
Copy link
Member

Without this PR our test suite appears to be passing, are you also seeing this or am I misunderstanding something?

@skazi0
Copy link
Author
skazi0 commented Oct 16, 2020

It depends on the python version you have on your CIs. If it doesn't have the CVE-2019-9740 fix included, it will pass without problems.
Note that having unescaped whitespace in the URL is wrong even if it doesn't fail the tests :o)

@sethmlarson
Copy link
Member

We percent-encode whitespace before passing to HTTPConnection the same way as all characters that aren't valid in a request target.

We have the latest 3.9-dev (and 3.8 and 3.7.9) in our CI, those should contain the patch correct?

@skazi0
Copy link
Author
skazi0 commented Oct 16, 2020

OK. Maybe this is not a proper fix for master then.
We've encountered this problem in older urllib3 versions.
Probably they didn't include the encoding change yet.
Can you post some reference to the change which introduced it?

@sethmlarson
Copy link
Member
sethmlarson commented Oct 16, 2020

Looks like v1.24.3 (and 1.25.x):

Apply fix for CVE-2019-9740. (Pull #1591)

@skazi0
Copy link
Author
skazi0 commented Oct 16, 2020

@sethmlarson the linked PR is just throwing exception if invalid URL is passed. These changes are included in the packages I'm working with. Do you remember when the percent-encoding of whitespace was added in urllib3?

@skazi0
Copy link
Author
skazi0 commented Oct 16, 2020

OK, I think I've found it: #1673
This is 1.25.4 and I hit the problem in some 1.22.
BTW, https://github.com/urllib3/urllib3/blob/master/src/urllib3/connectionpool.py#L66 is clearly not valid anymore since #1673 is merged.

@sethmlarson
Copy link
Member

@skazi0 Very true, we can remove that note.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
0