fix ASF handler chain streaming #7522
Conversation
There was a problem hiding this comment.
Kudos for this clever solution! It took a while to wrap my head around it, but it's a nice case of a decorator pattern working it's magic 😁
If we move forward with the approach, I would at least suggest to make it a bit more explicit and maybe encapsulate the decorator into a class or at least a new generator method.
I wonder though, since the log output will now be streamed like the response data, it will be jumbled through out the log output, as well, right? So I would imagine it to be fairly hard to trace in a real log file which chunks belong together. Moreover, we may be returning bytes so, in some cases, printing it may not make sense in the first place.
I was originally thinking about a really simple solution actually. For example, when we print a service response, e.g., a response to s3 GetObject, we would have the same issue, namely that we consume the S3 body stream. So instead we just print StreamedBytes(...) to the log.
localstack/localstack/aws/client.py
Lines 54 to 59 in c882f29
In this case it seems the tradeoff leans in favor of the simple solution rather than getting the output of the stream into the log. But I'd be OK with the log streaming approach as well.
2898154 to
3e3ddfe
Compare
3e3ddfe to
029be07
Compare
029be07 to
7415771
Compare
|
Unfortunately, some tests are currently failing in CI (actually blocking).
|
With the access to
response.data, the response logger consumed the whole iterator / response data generator when trying to log the response data.This PR fixes the ASF response streaming by wrapping the response data iterable with a logging function in case of a streaming response.
TODO:
Resolves #7518.