Speed up JSON parsing with readinto #3728
Conversation
Get a chunk of data from readinto instead of a single byte. This speeds up the parsing by reducing the number of function calls. Fixes micropython#3703
|
@jepler build is green. Please review. |
There was a problem hiding this comment.
This looks fine. There is a small chance of changed behavior, namely that if the calling code depends on being able to do something else with the data "after" the json object, up to READ_CHUNK_SIZE-1 bytes of it will have been already consumed and it won't work.
Specifically, with the current implementation and using a file, not a socket, you can have a file with the content {}\n{} and call json.load() twice on it to get an empty object {} each time.
If this proves to be a problem in practice, this optimization may have to be revisited. I wonder if there's some way to make sockets conform to the stream protocol.
|
Ah, true! I think the alternative to that would be to provide the full json string yourself. That's what most uses already do. |
Speed up JSON parsing with readinto (#3728 for 6.0.x)
Get a chunk of data from readinto instead of a single byte. This
speeds up the parsing by reducing the number of function calls.
Fixes #3703