Closed
Description
Is there an existing issue for this?
- I have searched the existing issues
Current Behavior
Hello, I'm the maintainer of the LocalStack.NET client library. LocalStack.NET is a thin wrapper around the official aws-sdk-net just like localstack-python-client.
I've discovered issues related to any S3-related operation when testing against LocalStack version 2.2. Notably, these operations are successful with previous versions like 1.3.1 and 2.0.
Related Issue: localstack-dotnet/localstack-dotnet-client#24
Description of the Problem: The operation fails when attempting any S3-related operation using the LocalStack.NET library. I think the logs provided below might offer more insight.
Test Scenario
Scenario Link: S3Service_Should_Create_A_Bucket
[Fact]
public async Task S3Service_Should_Create_A_Bucket()
{
var bucketName = Guid.NewGuid().ToString();
PutBucketResponse putBucketResponse = await CreateTestBucket(bucketName);
Assert.Equal(HttpStatusCode.OK, putBucketResponse.HttpStatusCode);
}
protected Task<PutBucketResponse> CreateTestBucket(string bucketName = null)
{
var putBucketRequest = new PutBucketRequest { BucketName = bucketName ?? BucketName, UseClientRegion = true };
return AmazonS3.PutBucketAsync(putBucketRequest);
}The test scenario above generates a GUID and uses it as the bucket name when creating a bucket. The above is just an example. All other S3-related operations also fail.
LocalStack Testcontainer setup
public static LocalStackBuilder LocalStackBuilder(string version)
{
return new LocalStackBuilder().WithImage($"localstack/localstack:{version}")
.WithName($"localStack-{version}-{Guid.NewGuid().ToString().ToLower()}")
.WithEnvironment("DOCKER_HOST", "unix:///var/run/docker.sock")
.WithEnvironment("DEBUG", "1")
.WithEnvironment("LS_LOG", "trace-internal")
.WithPortBinding(4566, true)
.WithCleanUp(true);
}Mitmproxy raw request
PUT http://s3.eu-central-1.amazonaws.com/d9137298-3452-477d-9c0c-30f7ec52f120/ HTTP/1.1
User-Agent: aws-sdk-dotnet-coreclr/3.7.102.0 aws-sdk-dotnet-core/3.7.200.17 .NET_Core/7.0.9 OS/Microsoft_Windows_10.0.22621 ClientAsync
amz-sdk-invocation-id: 148e1966-07dd-403e-bcb1-b3ea27097db6
amz-sdk-request: attempt=1; max=5
x-amz-security-token: my-AwsSessionToken
Host: s3.eu-central-1.amazonaws.com
X-Amz-Date: 20230817T074856Z
X-Amz-Content-SHA256: 270d7010e28541d025cba79779722247d464e2edd8b448b42fdf618a477e0432
Authorization: AWS4-HMAC-SHA256 Credential=my-AwsAccessKeyId/20230817/eu-central-1/s3/aws4_request, SignedHeaders=content-md5;content-type;host;user-agent;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=280f9675d5de93f8e991d81b1f0796d70ad031d4d777027398662d48ac7db162
Content-Length: 156
Content-Type: application/xml
Content-MD5: 3TSjRZmsSt7Rb6d4WVpfOQ==
<CreateBucketConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><LocationConstraint>eu-central-1</LocationConstraint></CreateBucketConfiguration>Localstack container logs
2023-08-17T07:48:54.135648905Z 2023-08-17T07:48:54.135 DEBUG --- [ asgi_gw_0] l.aws.serving.wsgi : PUT s3.eu-central-1.amazonaws.comhttp://s3.eu-central-1.amazonaws.com/6d34cfe9-0400-494e-9551-dfaf9798030e/
2023-08-17T07:48:54.140
7C6B
016928Z 2023-08-17T07:48:54.136 ERROR --- [ asgi_gw_0] l.aws.handlers.logging : exception during call chain
2023-08-17T07:48:54.140025268Z Traceback (most recent call last):
2023-08-17T07:48:54.140027608Z File "/opt/code/localstack/localstack/aws/chain.py", line 90, in handle
2023-08-17T07:48:54.140029348Z handler(self, self.context, response)
2023-08-17T07:48:54.140030938Z File "/opt/code/localstack/localstack/aws/handlers/routes.py", line 27, in __call__
2023-08-17T07:48:54.140032538Z router_response = self.router.dispatch(context.request)
2023-08-17T07:48:54.140033908Z File "/opt/code/localstack/localstack/http/router.py", line 443, in dispatch
2023-08-17T07:48:54.140035318Z handler, args = matcher.match(get_raw_path(request), method=request.method)
2023-08-17T07:48:54.140036718Z File "/opt/code/localstack/.venv/lib/python3.10/site-packages/werkzeug/routing/map.py", line 635, in match
2023-08-17T07:48:54.140038268Z raise RequestRedirect(
2023-08-17T07:48:54.140039648Z werkzeug.routing.exceptions.RequestRedirect: 308 Permanent Redirect: http://s3.eu-central-1.amazonaws.com/http:/s3.eu-central-1.amazonaws.com/6d34cfe9-0400-494e-9551-dfaf9798030e/
2023-08-17T07:48:54.140233396Z 2023-08-17T07:48:54.140 DEBUG --- [ asgi_gw_0] l.aws.protocol.serializer : No accept header given. Using request's Content-Type (application/xml) as preferred response Content-Type.
2023-08-17T07:48:54.142987739Z 2023-08-17T07:48:54.142 INFO --- [ asgi_gw_0] localstack.request.http : PUT /6d34cfe9-0400-494e-9551-dfaf9798030e/ => 500; Request(b'<CreateBucketConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><LocationConstraint>eu-central-1</LocationConstraint></CreateBucketConfiguration>', headers={'User-Agent': 'aws-sdk-dotnet-coreclr/3.7.102.0 aws-sdk-dotnet-core/3.7.200.17 .NET_Core/7.0.9 OS/Microsoft_Windows_10.0.22621 ClientAsync', 'amz-sdk-invocation-id': '148e1966-07dd-403e-bcb1-b3ea27097db6', 'amz-sdk-request': 'attempt=1; max=5', 'x-amz-security-token': 'my-AwsSessionToken', 'Host': 's3.eu-central-1.amazonaws.com', 'X-Amz-Date': '20230817T074851Z', 'X-Amz-Content-SHA256': '270d7010e28541d025cba79779722247d464e2edd8b448b42fdf618a477e0432', 'Authorization': 'AWS4-HMAC-SHA256 Credential=my-AwsAccessKeyId/20230817/eu-central-1/s3/aws4_request, SignedHeaders=content-md5;content-type;host;user-agent;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=64a84fae5e3cc966e56f4561053b2d74c1893dcc27347eeda8c02c274b8d5bbd', 'Content-Length': '156', 'Content-Type': 'application/xml', 'Content-MD5': '3TSjRZmsSt7Rb6d4WVpfOQ==', 'x-localstack-tgt-api': 's3'}); Response(b'<?xml version=\'1.0\' encoding=\'utf-8\'?>\n<Error><Code>InternalError</Code><Message>exception while calling s3 with unknown operation: Traceback (most recent call last):\n File "/opt/code/localstack/localstack/aws/chain.py", line 90, in handle\n handler(self, self.context, response)\n File "/opt/code/localstack/localstack/aws/handlers/routes.py", line 27, in __call__\n router_response = self.router.dispatch(context.request)\n File "/opt/code/localstack/localstack/http/router.py", line 443, in dispatch\n handler, args = matcher.match(get_raw_path(request), method=request.method)\n File "/opt/code/localstack/.venv/lib/python3.10/site-packages/werkzeug/routing/map.py", line 635, in match\n raise RequestRedirect(\nwerkzeug.routing.exceptions.RequestRedirect: 308 Permanent Redirect: http://s3.eu-central-1.amazonaws.com/http:/s3.eu-central-1.amazonaws.com/6d34cfe9-0400-494e-9551-dfaf9798030e/\n</Message><RequestId>d5ced6d2-d015-4c6c-8c83-0fef33501c7a</RequestId></Error>', headers={'Content-Type': 'application/xml', 'Content-Length': '981', 'x-amz-request-id': 'd5ced6d2-d015-4c6c-8c83-0fef33501c7a', 'x-amz-id-2': 's9lzHYrFp76ZVxRcpX9+5cjAnEH2ROuNkd2BHfIa6UkFVdtjf5mKR3/eTPFvsiP/XV/VLi31234=', 'Connection': 'close'})
2023-08-17T07:48:54.584494830Z 2023-08-17T07:48:54.584 DEBUG --- [ asgi_gw_0] l.aws.serving.wsgi : PUT s3.eu-central-1.amazonaws.comhttp://s3.eu-central-1.amazonaws.com/6d34cfe9-0400-494e-9551-dfaf9798030e/
2023-08-17T07:48:54.584981886Z 2023-08-17T07:48:54.584 ERROR --- [ asgi_gw_0] l.aws.handlers.logging : exception during call chain
2023-08-17T07:48:54.584999376Z Traceback (most recent call last):
2023-08-17T07:48:54.585002226Z File "/opt/code/localstack/localstack/aws/chain.py", line 90, in handle
2023-08-17T07:48:54.585004066Z handler(self, self.context, response)
2023-08-17T07:48:54.585005506Z File "/opt/code/localstack/localstack/aws/handlers/routes.py", line 27, in __call__
2023-08-17T07:48:54.585006916Z router_response = self.router.dispatch(context.request)
2023-08-17T07:48:54.585008256Z File "/opt/code/localstack/localstack/http/router.py", line 443, in dispatch
2023-08-17T07:48:54.585009746Z handler, args = matcher.match(get_raw_path(request), method=request.method)
2023-08-17T07:48:54.585011126Z File "/opt/code/localstack/.venv/lib/python3.10/site-packages/werkzeug/routing/map.py", line 635, in match
2023-08-17T07:48:54.585012576Z raise RequestRedirect(
2023-08-17T07:48:54.585013906Z werkzeug.routing.exceptions.RequestRedirect: 308 Permanent Redirect: http://s3.eu-central-1.amazonaws.com/http:/s3.eu-central-1.amazonaws.com/6d34cfe9-0400-494e-9551-dfaf9798030e/
2023-08-17T07:48:54.585077746Z 2023-08-17T07:48:54.584 DEBUG --- [ asgi_gw_0] l.aws.protocol.serializer : No accept header given. Using request's Content-Type (application/xml) as preferred response Content-Type.
2023-08-17T07:48:54.585715972Z 2023-08-17T07:48:54.585 INFO --- [ asgi_gw_0] localstack.request.http : PUT /6d34cfe9-0400-494e-9551-dfaf9798030e/ => 500; Request(b'<CreateBucketConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><LocationConstraint>eu-central-1</LocationConstraint></CreateBucketConfiguration>', headers={'User-Agent': 'aws-sdk-dotnet-coreclr/3.7.102.0 aws-sdk-dotnet-core/3.7.200.17 .NET_Core/7.0.9 OS/Microsoft_Windows_10.0.22621 ClientAsync', 'amz-sdk-invocation-id': '148e1966-07dd-403e-bcb1-b3ea27097db6', 'amz-sdk-request': 'attempt=2; max=5', 'x-amz-security-token': 'my-AwsSessionToken', 'Host': 's3.eu-central-1.amazonaws.com', 'X-Amz-Date': '20230817T074854Z', 'X-Amz-Content-SHA256': '270d7010e28541d025cba79779722247d464e2edd8b448b42fdf618a477e0432', 'Authorization': 'AWS4-HMAC-SHA256 Credential=my-AwsAccessKeyId/20230817/eu-central-1/s3/aws4_request, SignedHeaders=content-length;content-md5;content-type;host;user-agent;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=8bf2a988e09b2cf55f7eace8c036f9858ec742f7cf1b9ce6f3e3be16a4d769a6', 'Content-Length': '156', 'Content-Type': 'application/xml', 'Content-MD5': '3TSjRZmsSt7Rb6d4WVpfOQ==', 'x-localstack-tgt-api': 's3'}); Response(b'<?xml version=\'1.0\' encoding=\'utf-8\'?>\n<Error><Code>InternalError</Code><Message>exception while calling s3 with unknown operation: Traceback (most recent call last):\n File "/opt/code/localstack/localstack/aws/chain.py", line 90, in handle\n handler(self, self.context, response)\n File "/opt/code/localstack/localstack/aws/handlers/routes.py", line 27, in __call__\n router_response = self.router.dispatch(context.request)\n File "/opt/code/localstack/localstack/http/router.py", line 443, in dispatch\n handler, args = matcher.match(get_raw_path(request), method=request.method)\n File "/opt/code/localstack/.venv/lib/python3.10/site-packages/werkzeug/routing/map.py", line 635, in match\n raise RequestRedirect(\nwerkzeug.routing.exceptions.RequestRedirect: 308 Permanent Redirect: http://s3.eu-central-1.amazonaws.com/http:/s3.eu-central-1.amazonaws.com/6d34cfe9-0400-494e-9551-dfaf9798030e/\n</Message><RequestId>11e660d6-ff95-490b-9e59-4af093e60738</RequestId></Error>', headers={'Content-Type': 'application/xml', 'Content-Length': '981', 'x-amz-request-id': '11e660d6-ff95-490b-9e59-4af093e60738', 'x-amz-id-2': 's9lzHYrFp76ZVxRcpX9+5cjAnEH2ROuNkd2BHfIa6UkFVdtjf5mKR3/eTPFvsiP/XV/VLi31234=', 'Connection': 'close'})Expected Behavior
Using the LocalStack.NET library, all S3-related operations should be successful, consistent with the behavior observed on LocalStack versions 1.3.1 and 2.0.
How are you starting LocalStack?
Custom (please describe below)
Steps To Reproduce
How are you starting localstack (e.g., bin/localstack command, arguments, or docker-compose.yml)
Using TestContainers with the specific configurations mentioned above.
Client commands (e.g., AWS SDK code snippet, or sequence of "awslocal" commands)
Using the LocalStack.NET client library mentioned above. I can provide and example project for this specific case as well.
Environment
- OS: Windows 11 x64
- LocalStack: 2.2
- Docker: Docker Desktop running on WSL 2Anything else?
No response