8000 CloudFormation: Update Graph Preprocessor by MEPalma · Pull Request #12447 · localstack/localstack · GitHub
[go: up one dir, main page]
Skip to content

CloudFormation: Update Graph Preprocessor #12447

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 17 commits into from
Apr 9, 2025
Merged

CloudFormation: Update Graph Preprocessor #12447

merged 17 commits into from
Apr 9, 2025

Conversation

MEPalma
Copy link
Contributor
@MEPalma MEPalma commented Mar 27, 2025 •
edited
Loading

Motivation

Downstream components such as the Change Set Describer and the Change Set Executor require consistent access to resolved template values—including intrinsic functions, parameters, conditions, and mappings. Additionally, the current UpdateGraph implementation exposes too much of its internal complexity to downstream consumers. This results in overly tight coupling and the dealing with tedious logic in tasks that should remain focused on higher-level concerns.
This PR addresses these concerns by introducing a Preprocessor base class within the UpdateGraph visitor pattern. The Preprocessor consolidates resolution logic and exposes clean, structured data for both the “before” and “after” versions of a template. These changes also address a considerable number of fixes and improvements to the logic of the update graph modeler and the preprocessing.

Changes

  • Introduced a Preprocessor base class for resolving intrinsic functions, parameters, conditions, and mappings.
  • Export resolved values for both “before” and “after” templates using the new visitor approach.
  • Refactored the Describer as a preprocessor application and added support masking logic (e.g., KNOWN_AFTER_APPLY)
  • Updated UpdateGraph traversal results to produce distinct result objects per resource/entity, containing both “before” and “after” states.
  • Added temporary changes to improve v2 provider compatibility with the new UpdateGraph data model and resolution flow, and being able to run deployment process tests against the v2 engine
  • Adjusted deployment process capture tests to use transformers for replicability and validate behavior against real AWS responses However, some tests remain skipped due to the ongoing refactor of the Executor, which is still a work-in-progress.
  • Refactoring of the Executor to rely on the Preprocessor instead of the Describer
  • Adjustments in the provider v2 to use the update graph, and added notes for future work in this integration
  • Several improvements to update graph modeler and preprocessing logic

@MEPalma MEPalma added the semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases label Mar 27, 2025
@MEPalma MEPalma added this to the Playground milestone Mar 27, 2025
@MEPalma MEPalma self-assigned this Mar 27, 2025
@github-actions GitHub Actions
Copy link
github-actions bot commented Mar 27, 2025 •
edited
Loading

LocalStack Community integration with Pro

    2 files  ± 0      2 suites  ±0   1h 51m 46s ⏱️ - 2m 1s
4 342 tests +29  3 981 ✅  - 3  361 💤 +32  0 ❌ ±0 
4 344 runs  +29  3 981 ✅  - 3  363 💤 +32  0 ❌ ±0 

Results for commit 177db4c. ± Comparison against base commit 245ebe0.

This pull request skips 16 tests. 
tests.aws.services.cloudformation.api.test_changesets.TestUpdates ‑ test_simple_update_two_resources
tests.aws.services.transcribe.test_transcribe.TestTranscribe ‑ test_transcribe_happy_path
tests.aws.services.transcribe.test_transcribe.TestTranscribe ‑ test_transcribe_start_job[None-None]
tests.aws.services.transcribe.test_transcribe.TestTranscribe ‑ test_transcribe_start_job[test-output-bucket-2-None]
tests.aws.services.transcribe.test_transcribe.TestTranscribe ‑ test_transcribe_start_job[test-output-bucket-3-test-output]
tests.aws.services.transcribe.test_transcribe.TestTranscribe ‑ test_transcribe_start_job[test-output-bucket-4-test-output.json]
tests.aws.services.transcribe.test_transcribe.TestTranscribe ‑ test_transcribe_start_job[test-output-bucket-5-test-files/test-output.json]
tests.aws.services.transcribe.test_transcribe.TestTranscribe ‑ test_transcribe_start_job[test-output-bucket-6-test-files/test-output]
tests.aws.services.transcribe.test_transcribe.TestTranscribe ‑ test_transcribe_supported_media_formats[../../files/en-gb.amr-hello my name is]
tests.aws.services.transcribe.test_transcribe.TestTranscribe ‑ test_transcribe_supported_media_formats[../../files/en-gb.flac-hello my name is]
…

♻️ This comment has been updated with latest results.

@MEPalma MEPalma marked this pull request as ready for review April 3, 2025 08:39
simonrw and others added 3 commits April 3, 2025 15:26
@simonrw
Set resolved parameters on change set
bd64aa2 
This was required to prevent `order_changes` -> `order_resources` to
correctly determine if a reference is a resource or not.
@MEPalma
resole before parameters arg for update graph
0122390
@MEPalma
more update process tests on getattr and ref, and improvements
b4d4fb5
@MEPalma MEPalma requested review from bentsku and baermat as code owners April 8, 2025 16:15
@MEPalma MEPalma changed the title CloudFormation: [POC] Update Graph Preprocessor CloudFormation: Update Graph Preprocessor Apr 8, 2025
@MEPalma MEPalma removed request for bentsku and baermat April 8, 2025 16:22
@MEPalma MEPalma requested a review from alexrashed as a code owner April 8, 2025 16:31
@MEPalma MEPalma requested a review from dfangl as a code owner April 8, 2025 16:31
@MEPalma MEPalma removed request for alexrashed and dfangl April 8, 2025 16:31
@MEPalma MEPalma merged commit 377b1a7 into master Apr 9, 2025
37 checks passed
@MEPalma MEPalma deleted the MEP-CFN-update_graph_processor branch April 9, 2025 12:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dominikschubert dominikschubert Awaiting requested review from dominikschubert dominikschubert is a code owner

@pinzon pinzon Awaiting requested review from pinzon pinzon is a code owner

@simonrw simonrw Awaiting requested review from simonrw simonrw is a code owner

Assignees

@MEPalma MEPalma

Labels
semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases
Projects
None yet
Milestone
Playground
Development

Successfully merging this pull request may close these issues.
2 participants
@MEPalma @simonrw
0