8000 Replicator user defined id generator by cloutierMat · Pull Request #11626 · localstack/localstack · GitHub
[go: up one dir, main page]

Skip to content

Replicator user defined id generator #11626

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Oct 17, 2024

Conversation

cloutierMat
Copy link
Contributor

Motivation

As we are starting to build the replicator, we need a mechanism to create resources with custom ids. This pr introduces a set of resource agnostic utils that can be used to handle custom id registration and id generation.

By providing a simple decorator that will return a custom id if one was registered or the result of the decorated function if none are found.

Changes

  • Added utils around id generator
  • Added a sample for Cfn to see what implementation can look like for a service with an LS backend
  • Added a sample for secrets manager to see what implementation can look like for a service with a Moto Backend

@cloutierMat cloutierMat added semver: minor Non-breaking 8000 changes which can be included in minor releases, but not in patch releases area: replicator labels Oct 2, 2024
@cloutierMat cloutierMat added this to the Playground milestone Oct 2, 2024
@cloutierMat cloutierMat self-assigned this Oct 2, 2024
Copy link
github-actions bot commented Oct 2, 2024

LocalStack Community integration with Pro

    2 files  ±0      2 suites  ±0   1h 43m 44s ⏱️ +57s
3 508 tests +3  3 095 ✅ +3  413 💤 ±0  0 ❌ ±0 
3 510 runs  +3  3 095 ✅ +3  415 💤 ±0  0 ❌ ±0 

Results for commit 85db009. ± Comparison against base commit 1bc0db5.

♻️ This comment has been updated with latest results.

@cloutierMat cloutierMat force-pushed the replicator-user-defined-id-generator branch from a38d58b to 8be6bd6 Compare October 15, 2024 00:52
@cloutierMat cloutierMat modified the milestones: Playground, 4.0 Oct 15, 2024
@cloutierMat cloutierMat force-pushed the replicator-user-defined-id-generator branch from 8be6bd6 to 2a01941 Compare October 15, 2024 17:54
@@ -145,6 +145,7 @@ def apigateway_models_stage_to_json(fn, self):

return result

# TODO remove this patch when the behavior is implemented in moto
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick question as this has been an issue in the past, we should try to prevent - in custom ids, because it messes up with the regex matching the id (mixing it with the -vpc suffix or something), and also it should probably be lower case. Where could we implement this?

Copy link
Contributor Author
@cloutierMat cloutierMat Oct 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the feedback @bentsku. This is not something that is currently possible. I think it could be done by adding a validator method to ResourceIdentifier. By default, in would always return True, but then it can be overriden by specific Indentifier. All it would need then is to call it in find_id_from_sources.

It might be easier to implement in moto to prevent patching. Or we can also patch the specific Identifier we need a validator for and use getattr(identifier, "validator", None) in LocalstackIdManager.find_id_from_sources.

Copy link
Contributor Author
@cloutierMat cloutierMat Oct 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, that could be nice! The validator could also log a message, nice that it falls back to the default. We can discuss this, it's currently not the case but sometimes leads to confused users as to why trying to hit their API doesn't match when there's an hyphen.. so thank you for looking into this already!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yah, I think it could be a nice to have. The logging could be improved I agree. I will keep thinking on a nice solution for this as well, and see if I can come up with a more general solution as I see more use cases.

@cloutierMat cloutierMat marked this pull request as ready for review October 15, 2024 19:55
Copy link
Contributor
@bentsku bentsku left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks awesome! 🚀 LGTM for the APIGW part, just missing the API key cleanup, but I'll let others more close to the project approve the PR! Great work! 🥳

@@ -145,6 +145,7 @@ def apigateway_models_stage_to_json(fn, self):

return result

# TODO remove this patch when the behavior is implemented in moto
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick question as this has been an issue in the past, we should try to prevent - in custom ids, because it messes up with the regex matching the id (mixing it with the -vpc suffix or something), and also it should probably be lower case. Where could we implement this?

Copy link
Member
@pinzon pinzon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great 💪

@cloutierMat cloutierMat force-pushed the replicator-user-defined-id-generator branch from 933a29e to 85db009 Compare October 17, 2024 01:35
@cloutierMat cloutierMat merged commit bf8fafe into master Oct 17, 2024
34 checks passed
@cloutierMat cloutierMat deleted the replicator-user-defined-id-generator branch October 17, 2024 03:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: replicator semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
0