10000 Async Lambda Event Sources by efritz · Pull Request #1352 · localstack/localstack · GitHub
[go: up one dir, main page]

Skip to content

Async Lambda Event Sources #1352

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed

Async Lambda Event Sources #1352

wants to merge 3 commits into from

Conversation

efritz
Copy link
Contributor
@efritz efritz commented Jun 4, 2019

To speed up local actions and to be a bit closer to production behavior, lambda events triggered by an event source should be done so asynchronously.

This change makes async processing the default behavior, but allows each service (API Gateway, SNS, Kinesis, DyanmoDB, and SQS) to turn on synchronous processing with environment variables.

This may be a short-sighted change, but I can't see a major reason for synchronous processing to be blocking by default - please let me know if I'm missing a use-case.

This change is necessary as I'm using a custom go1.x runtime image that only allows on event to be processed at a time (simulating a reserved concurrency of one), and synchronous events deadlock if a dynamodb handler publishes a record back to dynamodb. Setting them as async does exactly what we're looking for.

@whummer
Copy link
Member
whummer commented Jun 12, 2019

Thanks @efritz - this sounds like a reasonable proposal to me. However, it seems that there are a number of tests currently failing which probably make the assumption that Lambdas are executed synchronously.

I guess we'd have to dig into each of these cases and figure out if the assumptions in the tests are valid or not, and then fix the behavior accordingly. I'd be happy to provide guidance, but would you be able to take over the lead on this task? Thanks!

@efritz
Copy link
Contributor Author
efritz commented Jun 14, 2019

When running locally, make test-docker seems to pass for me. Are the tests only failing when running outside of a container?

@whummer
Copy link
Member
whummer commented Jan 24, 2020

@efritz Would love to revisit this PR. It seems that the functionality in this PR is (partly) covered by #1973 , at least as far as SNS triggered Lambdas are concerned.

Can we try to rebase to master and get the remaining changes in?

I think it should now (hopefully) also be easier to get the tests working. If you are not planning to make these changes, I can try to take over this PR and take a stab at it. Thanks

@efritz
Copy link
Contributor Author
efritz commented Jan 25, 2020

Sure, I’ll take another look this weekend.

@efritz
Copy link
Contributor Author
efritz commented Jan 27, 2020

Looks like it didn't help the automated tests all that much :/

@whummer
Copy link
Member
whummer commented Jun 6, 2020

Rebased this PR onto latest master and create a new PR here: #2521 . Hope to get the tests working to be able to merge it soon. Thanks again @efritz !

@whummer whummer closed this Jun 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
0