Description
Is there an existing issue for this?
- I have searched the existing issues
Current Behavior
User's of Localstack's Lambda service are unable to reproduce EMFILE issues observed in real AWS deployments. This is due to the differences in ulimit configuration that does not match real AWS quotas defined here
ulimit -a on local machine -> -n: file descriptors 10240
ulimit -a from Localstack container -> nofiles 1048576
ulimit -a from lambci/lambda:nodejs12.x container after invoking Lambda function -> open files (-n) 1048576
When overriding Localstack's ulimits via docker-compose file the ulimit is not applied to Lambda service.
services:
localstack:
ulimits:
nofile:
soft: 1024
hard: 1024
ulimit -a from Localstack container -> nofiles 1024
ulimit -a from lambci/lambda:nodejs12.x container after invoking Lambda function -> open files (-n) 1048576
Expected Behavior
By default, Localstack's Lambda service should mirror and enforce AWS limits for more accurate simulation.
At a minimum, Localstack users should be able to apply overrides to services to better align with real AWS quotas to reproduce issues observed from real AWS deployments such as EMFILE.
How are you starting LocalStack?
With a docker-compose file
Steps To Reproduce
How are you starting localstack (e.g., bin/localstack command, arguments, or docker-compose.yml)
With `docker-compose.yml`...
```
version: "3"
services:
localstack:
image: "localstack/localstack:0.14.2"
ports:
- "4566:4566"
environment:
AWS_ACCESS_KEY_ID: "local-access-key-id"
AWS_SECRET_ACCESS_KEY: "local-secret-access-key"
HOSTNAME_EXTERNAL: "localstack"
LAMBDA_DOCKER_NETWORK: "project-name-default-438332cd_default"
LAMBDA_EXECUTOR: "docker"
SERVICES: "lambda"
volumes:
- "/path/to/localstack:/docker-entrypoint-initaws.d"
- "/var/run/docker.sock:/var/run/docker.sock"
- "/path/to/project/lambdaArtifact.zip"
# Uncomment when comparing Localstack's ulimits overrides to spun up lambci/lambda containers
#ulimits:
# nofile:
# soft: 1024
# hard: 1024
```
Client commands (e.g., AWS SDK code snippet, or sequence of "awslocal" commands)
- Start Localstack with docker compose
- docker exec -it sh
- ulimit -a (or ulimit -n)
- Invoke lambda function (i.e
awslocal lambda invoke --cli-read-timeout 0 --cli-connect-timeout 0 --endpoint "http://localhost:4566" --region "insert-regions" --function-name "arn:aws:lambda:the-region:123456789876:function:the-lambda-function-name" --payload '{}' --invocation-type RequestResponse out && cat out) watch docker psto find container ID for spun uplambci/lambdacontainerdocker exec -it <lambci/lambdaContainerID> shulimit -aorulimit -nnotice value does not match real AWS limits and will differ from Localstack's ulimits configuration if overridden through docker compose
To reproduce EMFILE in real AWS but unable to observe in Localstack
- Make a simple Lambda function that issues more than 1024 HTTP requests (EMFILE relates to file and/or socket descriptors. I am using typescript 4.8.2 with axios 0.27.2 to make HTTP requests)
- Deploy to real AWS, invoke Lambda function, observe EMFILE error logs
- Then
- Deploy same Lambda function with Localstack
- Invoke Lambda function and notice EMFILE error logs are not found
Environment
- OS: macOS Monterey Version 12.6 (Apple M1 Pro Chip)
- LocalStack: 0.14.2
- Lambda runtime: nodejs12.xAnything else?
No response