Description
Description of the problem, including code/CLI snippet
The artifact downloads support a streaming mode which wraps the iterator provided by the requests library and allows the user to provide an action callable that is called with the chunks provided by the iterator as arguments.
I would like to be able to access the response.iter_content() iterator directly.
Expected Behavior
My use case would be something like this:
iter_response = project.artifacts.download(
ref_name="main", job="build", streamed=True, action='iterator'
)
do_things_with_iterator(iter_response)( In my real project do_things_with_iterator is actually a fastapi StreamingResponse which takes an iterator or generator. This would allow downloads of artifacts without first having to download the entire file server side and then forwarding it to the client)
Actual Behavior
As far as I can tell this is currently not possible.
I thought this would be relatively simple to implement, so I adapted the response_content function in utils.py to
def response_content(
response: requests.Response,
streamed: bool,
action: Optional[Union[Callable, Literal["iterator"]]],
chunk_size: int,
) -> Optional[Union[bytes, Iterator[Any]]]:
if streamed is False:
return response.content
if action is None:
action = _StdoutStream()
if action == "iterator":
return response.iter_content(chunk_size=chunk_size)
for chunk in response.iter_content(chunk_size=chunk_size):
if chunk:
action(chunk)
return NoneBut I ran into issues adapting the type annotations in other places, as some functions are not designed to handle iterators and will only handle bytes. I did not have a lot of time to spend on figuring this out though so if it seems worth the effort I can look into this further and create a pull request later.
Specifications
- python-gitlab version: 3.2.0
- API version you are using (v3/v4): v4
- Gitlab server version (or gitlab.com): gitlab.com