8000 bpo-29654 : Support If-Modified-Since HTTP header (browser cache) by PierreQuentel · Pull Request #298 · python/cpython · GitHub
[go: up one dir, main page]

Skip to content

bpo-29654 : Support If-Modified-Since HTTP header (browser cache) #298

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 32 commits into from
Apr 2, 2017
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
20cc898
Support If-Modified-Since HTTP headers, return 304 response if file w…
Feb 25, 2017
f1d8a48
Fix bug in datetime comparisons. Ignore If-Modified-Since if If-None-…
Feb 26, 2017
c4403ec
Update Misc/NEWS and What's New
Feb 26, 2017
3225080
Fix typo
Feb 26, 2017
415e4af
Use parsedate_to_datetime to extract datetime from If-Modified-Since,…
Feb 26, 2017
8595735
Change computing of dates used to test browser cache
Feb 26, 2017
2be2f8c
Remove microseconds from time of last modification with .replace(micr…
Feb 26, 2017
c3337f4
Put os.fstat() call inside the try block ; handle obsolete HTTP date …
Feb 26, 2017
143e1e9
Restore alphabetical order in section Improved Modules
Feb 26, 2017
42edfe3
Store last modification date in setUp ; split browser cache tests in …
Feb 26, 2017
1e68be9
Specify the exceptions to catch in parsedate_to_datetime
PierreQuentel Feb 27, 2017
fc20596
Use except/else for parsedate_to_datetime exception handling
PierreQuentel Feb 27, 2017
9436e45
Restore blank line
Feb 28, 2017
708ff03
Put imports in alphabetical order. Change version to 0.7.
Mar 1, 2017
6a58894
Presentation changes.
Mar 1, 2017 8000
db46671
Change order of imports.
Mar 1, 2017
41f74ef
Update Misc/NEWS (conflict)
Mar 2, 2017
204c503
Merge branch 'master' into master
PierreQuentel Mar 2, 2017
e2fee49
Merge branch 'master' into master
PierreQuentel Mar 4, 2017
6dadf6e
Restore version number 0.6
Mar 7, 2017
c389bf6
Replace "datetime" by "time"
Mar 7, 2017
3cc4f6d
Merge branch 'master' of https://github.com/PierreQuentel/cpython
Mar 7, 2017
294e164
Proposal for an update of the http.server module documentation
Mar 7, 2017
6c5ba39
Merge branch 'master' into master
PierreQuentel Mar 8, 2017
1e0f3e3
Merge branch 'master' into master
PierreQuentel Mar 15, 2017
2503add
Merge branch 'master' into master
serhiy-storchaka Mar 20, 2017
36a8e2a
Merge branch 'master' into master
PierreQuentel Mar 23, 2017
c8b108f
Add a "Changed in version 3.7" comment
Mar 24, 2017
bd91346
Merge branch 'master' of https://github.com/PierreQuentel/cpython
Mar 24, 2017
d3238d2
Merge branch 'master' into master
PierreQuentel Apr 1, 2017
16f9e11
Remove trailing whitespaces
Apr 1, 2017
c669909
Merge branch 'master' of https://github.com/PierreQuentel/cpython
Apr 1, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions Doc/whatsnew/3.7.rst
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,12 @@ urllib.parse
adding `~` to the set of characters that is never quoted by default.
(Contributed by Christian Theune and Ratnadeep Debnath in :issue:`16285`.)

http.server
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move before unittest.mock. Modules should be lexicographically ordered.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in commit 143e1e9

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated change.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I don't understand what I should do here

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You removed an empty line. Insert it back.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, done in commit 9436e45

-----------
:class:`SimpleHTTPRequestHandler` supports the HTTP If-Modified-Since header.
The server returns the 304 response status if the target file was not
modified after the datetime specified in the header.
(Contributed by Pierre Quentel in :issue:`29654`.)

Optimizations
=============
Expand Down
24 changes: 23 additions & 1 deletion Lib/http/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,7 @@
import socketserver
import sys
import time
import datetime
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try to support alphabetical order of imports. I.e. add import datetime before import email.utils.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in commit 708ff03. The order was not supported for argparse and copy...
I also changed to version to 0.7.

import urllib.parse
import copy
import argparse
Expand Down Expand Up @@ -686,10 +687,30 @@ def send_head(self):
except OSError:
self.send_error(HTTPStatus.NOT_FOUND, "File not found")
return None

fs = os.fstat(f.fileno())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

f is leaked if os.fstat() raises an exception.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


try:
# Use browser cache if possible
if ("If-Modified-Since" in self.headers
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HTTP headers are case-insensitive. Does this work with "if-modified-since"?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self.headers should be (similar to) an email.message.Message object, so I think it should work fine

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I confirm that it's case-insensitive

and "If-None-Match" not in self.headers):
# compare If-Modified-Since and time of last file modification
ims = email.utils.parsedate_to_datetime(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

email.utils.parsedate_to_datetime() can raise an exception. It might be better just to ignore the cache rather than return an error.

Needed a cache for ill-formed "If-Modified-Since" value.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done in c3337f4

self.headers["If-Modified-Since"])
if (ims is not None and
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be written in one line.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done in c3337f4

ims.tzinfo is datetime.timezone.utc):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe you should also allow tzinfo to be None, to support the obsolete “asctime” format from https://tools.ietf.org/html/rfc7231#section-7.1.1.1 without a timezone (e.g. Sun Nov 6 08:49:37 1994)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should not happen often, but it is supported in commit c3337f4

# compare to UTC datetime of last modification,
# rounded to the second
mtime = int(fs.st_mtime)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

“Int” rounds towards 1970. The actual rounding that “date_time_string” uses seems to be a combination of rounding half to even microseconds (thanks to datetime), and then rounding down (always backwards in time) to seconds.

It might be more sensible to call datetime.replace(microseconds=0).

last_modif = datetime.datetime.fromtimestamp(mtime,
ims.tzinfo)
if last_modif <= ims:
self.send_response(HTTPStatus.NOT_MODIFIED)
self.end_headers()
f.close()
return
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return None. If there are return statements with value, all None values should be returned explicitly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

self.send_response(HTTPStatus.OK)
self.send_header("Content-type", ctype)
fs = os.fstat(f.fileno())
self.send_header("Content-Length", str(fs[6]))
self.send_header("Last-Modified", self.date_time_string(fs.st_mtime))
self.end_headers()
Expand Down Expand Up @@ -1212,3 +1233,4 @@ def test(HandlerClass=BaseHTTPRequestHandler,
else:
handler_class = SimpleHTTPRequestHandler
test(HandlerClass=handler_class, port=args.port, bind=args.bind)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated changes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't add this line.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in commit fc20596

39 changes: 39 additions & 0 deletions Lib/test/test_httpservers.py
Original file line number Diff line number Diff line change
Expand Up @@ -444,6 +444,45 @@ def test_head(self):
self.assertEqual(response.getheader('content-type'),
'application/octet-stream')

def test_browser_cache(self):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be better to split this test on few tests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in commit 42edfe3.
The datetime of last modification is stored in method setUp and used in the browser cache tests. I also added a test for the Last-Modified header.

#constructs the path relative to the root directory of the HTTPServer
response = self.request(self.base_url + '/test')
self.check_status_and_reason(response, HTTPStatus.OK, data=self.data)
last_modif = response.headers['Last-modified']

# send new request to the same url with request header
# If-Modified-Since set to Last-Modified : must return 304
from email.message import Message
headers = Message()
headers['If-Modified-Since'] = last_modif
response = self.request(self.base_url + '/test', headers=headers)
self.check_status_and_reason(response, HTTPStatus.NOT_MODIFIED)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please repeat the test with an earlier date, verifying that a response body is returned.


# if If-None-Match header is present, ignore If-Modified-Since
headers['If-None-Match'] = "*"
response = self.request(self.base_url + '/test', headers=headers)
self.check_status_and_reason(response, HTTPStatus.OK)

# with If-Modified-Since earlier than Last-Modified, must return 200
import datetime
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Import at module level. Importing inside functions is used in exceptional cases.

Copy link
Contributor Author

C F438 hoose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done in commit 42edfe3

import email.utils
dt = email.utils.parsedate_to_datetime(last_modif)
# build datetime object : 365 days before last modification
old_dt = dt - datetime.timedelta(days=365)
headers = Message()
headers['If-Modified-Since'] = email.utils.format_datetime(old_dt,
usegmt=True)
response = self.request(self.base_url + '/test', headers=headers)
self.check_status_and_reason(response, HTTPStatus.OK)

# one hour after last modification : must return 304
new_dt = dt + datetime.timedelta(hours=1)
headers = Message()
headers['If-Modified-Since'] = email.utils.format_datetime(new_dt,
usegmt=True)
response = self.request(self.base_url + '/test', headers=headers)
self.check_status_and_reason(response, HTTPStatus.NOT_MODIFIED)

def test_invalid_requests(self):
response = self.request('/', method='FOO')
self.check_status_and_reason(response, HTTPStatus.NOT_IMPLEMENTED)
Expand Down
3 changes: 3 additions & 0 deletions Misc/NEWS
Original file line number Diff line number Diff line change
Expand Up @@ -666,6 +666,9 @@ Library
- Issue #24142: Reading a corrupt config file left configparser in an
invalid state. Original patch by Florian Höch.

- bpo-29654 : Support If-Modified-Since HTTP header (browser cache). Patch
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the space before colon. Use two spaces after sentence ending period. Move the entry to the start of the "Library" section (new entries are added at the start).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done in 6a58894

by Pierre Quentel.

Windows
-------

Expand Down
0