Closed
Description
Is there an existing issue for this?
- I have searched the existing issues
Current Behavior
It seems chunked transfer-encoding is broken when using the ASF handler chain.
With the handler chain
When running the following scratch file, and then curl -X POST localhost:5000/_pods, I receive the payload foobar\nbaz\nbazed at once after 3 seconds.
Here's the snippet:
import asyncio
import time
from hypercorn import Config
from localstack.aws.app import LocalstackAwsGateway
from localstack.aws.serving.asgi import AsgiGateway
from localstack.http import Request, Response, route
from localstack.http.hypercorn import HypercornServer
from localstack.services.edge import ROUTER
class CloudPodsPublicApi:
@route("/_pods", methods=["POST"])
def pods(self, _request: Request) -> Response:
def _gen():
time.sleep(1)
yield "foo"
time.sleep(1)
yield "bar\n"
time.sleep(1)
yield "baz\n"
time.sleep(1)
yield "bazed\n"
return Response(_gen(), 200)
def main():
ROUTER.add_route_endpoints(CloudPodsPublicApi())
# application
gateway = LocalstackAwsGateway()
# server control loop
config = Config()
config.bind = f"localhost:5000"
loop = asyncio.new_event_loop()
srv = HypercornServer(AsgiGateway(gateway, event_loop=loop), config, loop=loop)
srv.start()
try:
srv.join()
except KeyboardInterrupt:
pass
finally:
srv.shutdown()
if __name__ == '__main__':
main()output: (note the missing Transfer-Encoding: chunked header in the response)
% curl -v -X POST localhost:5000/_pods
* Trying 127.0.0.1:5000...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 5000 (#0)
> POST /_pods HTTP/1.1
> Host: localhost:5000
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200
< Content-Type: text/plain; charset=utf-8
< Connection: close
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Methods: HEAD,GET,PUT,POST,DELETE,OPTIONS,PATCH
< Access-Control-Allow-Headers: authorization,cache-control,content-length,content-md5,content-type,etag,location,x-amz-acl,x-amz-content-sha256,x-amz-date,x-amz-request-id,x-amz-security-token,x-amz-tagging,x-amz-target,x-amz-user-agent,x-amz-version-id,x-amzn-requestid,x-localstack-target,amz-sdk-invocation-id,amz-sdk-request
< Access-Control-Expose-Headers: etag,x-amz-version-id
< Content-Length: 17
< date: Wed, 18 Jan 2023 23:35:06 GMT
< server: hypercorn-h11
<
foobar
baz
bazed
* Closing connection 0
Without the ASF handler chain, just serving the router
Whereas this here works fine:
import asyncio
import time
from hypercorn import Config
from localstack.aws import handlers
from localstack.aws.gateway import Gateway
from localstack.aws.serving.asgi import AsgiGateway
from localstack.http import Request, Response, route
from localstack.http.hypercorn import HypercornServer
from localstack.services.edge import ROUTER
class CloudPodsPublicApi:
@route("/_pods", methods=["POST"])
d
6EE2
ef pods(self, _request: Request) -> Response:
def _gen():
time.sleep(1)
yield "foo"
time.sleep(1)
yield "bar\n"
time.sleep(1)
yield "baz\n"
time.sleep(1)
yield "bazed\n"
return Response(_gen(), 200)
def main():
# application
ROUTER.add_route_endpoints(CloudPodsPublicApi())
gateway = Gateway()
gateway.request_handlers.append(handlers.serve_edge_router_rules)
# server control loop
config = Config()
config.bind = f"localhost:5000"
loop = asyncio.new_event_loop()
srv = HypercornServer(AsgiGateway(gateway, event_loop=loop), config, loop=loop)
srv.start()
try:
srv.join()
except KeyboardInterrupt:
pass
finally:
srv.shutdown()
if __name__ == '__main__':
main()Output:
% curl -v -X POST localhost:5000/_pods
* Trying 127.0.0.1:5000...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 5000 (#0)
> POST /_pods HTTP/1.1
> Host: localhost:5000
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200
< Content-Type: text/plain; charset=utf-8
< date: Wed, 18 Jan 2023 23:33:50 GMT
< server: hypercorn-h11
< Transfer-Encoding: chunked
<
foobar
baz
bazed
* Connection #0 to host localhost left intact
Expected Behavior
Both snippets should work in the same way
How are you starting LocalStack?
Custom (please describe below)
Steps To Reproduce
- run one of the scratch files
- run
curl -v -X POST localhost:5000/_pods
Environment
- OS:
- LocalStack: latestAnything else?
Clearly something in the handler chain is messing with the request or the headers, or perhaps reading the response and re-calculating the content-length. Perhaps a response logger?
I haven't had time to write a test for it yet, but should be easily reproducible. It's odd that this doesn't affect our kinesis tests, but I'm not sure whether we are even testing the partial receiving of data 🤷
/cc @alexrashed @bentsku