Additional support for zip files #68
Conversation
* Added Finances API Feature * .setup.py: bump version to 0.7.5-dev0 * Split out request_description building from make_request() So that it can be tested more easily, and refactored * Split out building the initial params dict from make_request() So that it can be tested more easily * Add fake MWS credentials pytest fixture * test_service_status() should use the pytest fake credentials fixture * Add more pytest fixtures (access_key, secret_key, account_id, timestamp) * Add test for calc_request_description() * Split out calc_request_description() into more statements So that it is easier to debug * Fix calc_request_description - don't include leading ampersand * Don't do automated deployments via Travis (for the moment) * Update README.md badges * InboundShipments, next_token_action decorator, and some style cleanups. (python-amazon-mws#33) * Testing out git commits from VS Code * Reverting the test commit * Adding VS Code settings to gitignore. * Style fixes * MWS.enumerate_param deprecated: now using utils.enumerate_param and utils.enumerate_params * InboundShipments fleshed out; added `utils.next_token_action` decorator; deprecated separate methods for `...by_next_token()` * Bugfix, rename `_enumerate_param` to `enumerate_param` (no need for private) * Fix for next_token issues. * TravisCI flake8 complaining, fixes. * Minor flake8 complaint. * Hack to get flake8 to stop complaining. * Strip pylint disables to clear line length issue. * Correction to keyed params, now tests every item in values sequence to ensure all are dicts. * Add tests for param methods in utils. * Add test for next token decorator. * Adding 'InboundShipments' to `__all__` * Assigning response a default in __init__ for DictWrapper and DataWrapper * Unneeded line breaks removed + docstring formatting * Comment corrected. They're tuples, not sets. * Finances methods updated to use next_token_action decorator * Create .travis.yaml * Update .gitignore * Removing deploy code from local travis * Delete .travis.yaml * Pushed to 0.8.0-dev0 Recently added functionality to InboundShipments, as well as Finances API. These constitute feature additions with backwards compatibility, which calls for a minor version update. * Adding Python 3.6 category We are testing in 3.6 in Travis anyway. May as well include the note. * Specified master and develop branches for badges Ensured the badges for Travis and Codecov are pointing to the appropriate branches (used to be pointing to default, which was master in both cases). * Updated comments throughout module No substantial code changes, comment changes only. Also ensured all docstrings follow same format. * Fixed docstring formatting Also made slight update to docstring for ObjectDict to more clearly note what it does, vs what the original code did. * Fix for flake8 (trailing whitespace) * Fix for flake8 (trailing whitespace) * Bump to 0.8.0 (drop dev tag) for release * Bug: Incorrect use of `super` for back-compat Using the old-style `super` syntax to comply with Python 2.7 compatibility. Not revealed in tests, because current tests don't touch the APIs. Whoops! * Added back old object names in case needed Old names `object_dict` and `xml2dict` added back in case the old objects are being used directly by some users. To be removed in 1.0.0 release down the road.
Remove old-style string formatting for sake of clarity.
Added in with a hotfix.
* version bump * No from_address assignment needed in Inbound init Similar fix as commit on `develop`, which will come in with 1.0 release. Also, version bump for new bugfix release.
mws/mws.py
Outdated
| raise MWSError("Wrong Contentlength, maybe amazon error...") | ||
|
|
||
| if headers['content-type'] == 'application/zip': | ||
| target_file_name = input("The response is a zip file. Enter name of zip file to store in current folder: ") |
There was a problem hiding this comment.
We cannot assume that all users will be using this package on the command line, thus we cannot reliably ask for input here.
Instead, we would need to store a file-like object on the response so that end users can save it, read it, etc., as needed.
There was a problem hiding this comment.
If we put in a UI interface for storing a file-object (windows/mac), is there a way to continue programmatic unzipping and further data manipulations? I'm currently using this wrapper to automatically check if some reports are zip files (and not XML), and if yes, unzip and continue manipulations on the unzipped file (take the output data and feed it into another system). My guess is this functionality would break on a UI.
Any thoughts?
There was a problem hiding this comment.
Maybe another function definition (eg., get_zipped_file()?)
There was a problem hiding this comment.
Sorry for the delay in responding :) .
Users typically make use of python-amazon-mws as a connector between their other projects and Amazon MWS. For instance, a Django website can get shipment data from their API and store it in data models. Continuing the example, it would not be within the scope of this package to write a Django model or an admin interface: those would be written separately by the developer(s) who need(s) them.
We can offer some assistance to make things easier for certain general uses. For example, in responses that contain nothing but zipped files, we could transform the response data into a file-like object in memory, which the user might then save to their file system through their own code.
Other responses, like the package label responses in FulfillmentInboundShipments, contain PDFs that are ZIPped and sent as base64-encoded strings (see here). I don't think we want to assume too much about how a user may want to print these documents, but we could offer a helper method that would parse the base64 string, unzip the resulting ZIP file, extract the PDF contents into a file in-memory, and return that file-like object.
Codecov Report
@@ Coverage Diff @@
## master #68 +/- ##
==========================================
- Coverage 46.67% 46.42% -0.25%
==========================================
Files 4 4
Lines 632 644 +12
Branches 64 65 +1
==========================================
+ Hits 295 299 +4
- Misses 324 330 +6
- Partials 13 15 +2
Continue to review full report at Codecov.
|
|
Thank you Galen! I've added an unzipped() helper method in the DataWrapper object as I encountered some zip files in the reports section (which were not base 64 encoded). If this commit makes sense, I'll add support for the Package Labels as well (decoding base64 and then unzipping to return a file object). |
|
Sorry for the inconvenience. This error is to do the implicit chaos how we handle responses. You can just access parsed_response.response.text for fixing this. Requests has a build in zip support. Can you confirm this? |
…ignore flag for xml reports xml reports all have different root nodes
|
I'm sorry I had this issue again today. Amazon has changed the type of file and I'm unable to check the filetype from outside the package. This is what the response looks like What edits can I do to understand the kind of content? I'd be glad to make a pull request and modify this. |
When certain responses are not in XML format, it has been found that the content-type is a zip file, which contains the appropriate data when unzipped. This happens primarily with the Reports objects. Therefore, instead of getting a Datawrapper object in bytes, the content can be stored locally in zip folder by the user. On entering the name of the target file where the zip file is to be stored, the code pushes the response into this target file and returns a success/error response.