8000 feat: isolate flow execution from business process by Benedikt-Brunner · Pull Request #5744 · shopware/shopware · GitHub
[go: up one dir, main page]

Skip to content

feat: isolate flow execution from business process #5744

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

Benedikt-Brunner
Copy link
Contributor
@Benedikt-Brunner Benedikt-Brunner commented Dec 3, 2024

1. Why is this change necessary?

Currently when a flow fails for any reason, the business process (request, queue message, etc.) which triggered it will also fail. This is due to the database transaction being marked for rollback only. In the worst case, this can lead to the store checkout failing for customers, which should always be avoided.

2. What does this change do, exactly?

This changes the flow executor to collect all flow triggers which are dispatched during the business process and then run them only after the business process has concluded.

3. Describe each step to reproduce the issue or behaviour.

  • Create a flow that in any way crashes (illegal state transition, infinite loop, etc.)
  • Execute a business process which triggers the broken flow

4. Please link to the relevant issues (if any).

5. Checklist

  • I have rebased my changes to remove merge conflicts
  • I have written tests and verified that they fail without my change
  • I have created a changelog file with all necessary information about my changes
  • I have written or adjusted the documentation according to my changes
  • This change has comments for package types, values, functions, and non-obvious lines of code
  • I have read the contribution requirements and fulfill them.

Feature Flag

These changes are all contained within the V6.7.0.0-Major-Feature-Flag!

@Benedikt-Brunner Benedikt-Brunner changed the title Isolate flow execution from business process [Hacktoberfest] Isolate flow execution from business process Dec 3, 2024
…usiness-process

# Conflicts:
#	src/Core/Content/Flow/Dispatching/FlowDispatcher.php
#	tests/unit/Core/Content/Flow/Dispatching/FlowDispatcherTest.php
@Benedikt-Brunner Benedikt-Brunner changed the title [Hacktoberfest] Isolate flow execution from business process [Hacktoberfest] Isolate flow execution from business process and add flow execution log Dec 3, 2024
@Benedikt-Brunner Benedikt-Brunner marked this pull request as ready for review December 3, 2024 18:12
Copy link
codecov bot commented Dec 3, 2024

Codecov Report

Attention: Patch coverage is 78.88889% with 19 lines in your changes missing coverage. Please review.

Project coverage is 50.94%. Comparing base (675607f) to head (76d3504).
Report is 17 commits behind head on trunk.

Files with missing lines Patch % Lines
...ontent/Flow/Exception/ExecuteSequenceException.php 0.00% 13 Missing ⚠️
...patching/BufferedFlowExecutionTriggersListener.php 71.42% 4 Missing ⚠️
...c/Core/Content/Flow/Dispatching/FlowDispatcher.php 75.00% 1 Missing ⚠️
src/Core/Content/Flow/Dispatching/FlowExecutor.php 96.55% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##            trunk    #5744   +/-   ##
=======================================
  Coverage   50.94%   50.94%           
=======================================
  Files        4754     4757    +3     
  Lines      161732   161818   +86     
  Branches     8817     8817           
=======================================
+ Hits        82387    82436   +49     
- Misses      76824    76861   +37     
  Partials     2521     2521           
Flag Coverage Δ
jest-admin 60.17% <ø> (ø)
jest-storefront 67.60% <ø> (ø)
phpunit-migration 39.11% <ø> (ø)
phpunit-unit 42.43% <78.88%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@patzick patzick changed the title [Hacktoberfest] Isolate flow execution from business process and add flow execution log feat: isolate flow execution from business process and add flow execution log Dec 19, 2024
@MalteJanz
Copy link
Contributor
MalteJanz commented Jan 7, 2025

Hi @Benedikt-Brunner , thanks for your big PR 🎉 .

before we start with code review, I checked out your changes and talked with our design department about them. We had the following questions:

  1. What does Failed action and context menu item Show failed action in the executions tab do? Can you provide an example on how to trigger a failed flow execution and see something here? We tried with a invalid state transition flow, but that only showed up in the Error message column.
  2. The Time column in the execution log shows 59 Minutes from now, even though the flow just executed and the hover tooltip correctly displays the current time.
  3. I guess you are aware of our commercial feature "delayed flows" and that it also add a tab Scheduled actions to a flow detail page, which is a little similar, as it can also show a failure state for a delayed flow action. Just something to keep in mind and not cause confusion between the two.
  4. Regarding your change to isolate the flow execution from ongoing processes: If I create a flow which does an illegal state transition (e.g. trigger on order transaction status paid, add an action that assigns the shipping status to returned). If I then change an order in the administration to the transaction status paid, that request will still fail. I didn't expect this with your change, maybe an oversight or did I do something wrong?
  5. A shop has likely many flows (e.g. 53 by default in commercial), how is the shop owner supposed to discover his failed flow executions? Does he need to look into every 53 flows daily and search for any failed execution there? It might make more sense to lift that executions tab out of the detail page into the general flow builder page.
  6. In general there could be many flows in a shop, which can execute many times (e.g. for each order that comes in). Is it really necessary to store all these executions? How are they cleaned up? Maybe it makes more sense to store only the failed ones.

We are also having some internal discussions about "the error handling and discovery for the merchant" right now, which isn't an easy topic. Maybe it would make sense to leave the "logging" part out of this PR for now. But to at least get somewhat notified you could maybe use the symfony logger for now. Do you have any other ideas?🤔

@Benedikt-Brunner
Copy link
Contributor Author

Hi @MalteJanz, thanks for the thorough feedback.

We would like to follow the proposal of splitting up the PR into two. This one will only contain the technical changes (moving the flow execution), while the follow up PR will contain all UI/Logging changes. This way we can incorporate the ideas of the design team without impeding the important technical changes "under the hood".

I have tested the implementation again to check that it still works (regarding your questions), in production builds everything works as expected (i have attached a video showing the functionality below). However i discovered that if symfony is running in debug mode, there is an inconsistency in the handling of the KernelEvents::Terminate-Event, which leads to malformed response payloads if a flow crashes. To address this i have opened an issue in the symfony repository, see symfony/symfony#59447.

However as we do not know how symfony will respond to this or how long a fix will take, we might need to work around the issue in Shopware. Specifically, the https://github.com/shopware/shopware/blob/trunk/src/Core/Framework/Api/EventListener/ResponseExceptionListener.php should not write to the response after the KernelEvents::Response/KernelEvents::Finish_Request-Event.

I would be happy to implement a fix, however i wanted to hear the shopware teams opinion on the matter first.

Video

Screen.Recording.2025-01-08.at.13.41.44.mov

@Benedikt-Brunner
Copy link
Contributor Author

PS:

regarding question 4:

Regarding your change to isolate the flow execution from ongoing processes: If I create a flow which does an illegal state transition (e.g. trigger on order transaction status paid, add an action that assigns the shipping status to returned). If I then change an order in the administration to the transaction status paid, that request will still fail. I didn't expect this with your change, maybe an oversight or did I do something wrong?

The changes to the flow execution are behind the V6.7.0.0 major feature flag, they only take effect when one enables the feature flag / removes the checks. Did you activate the feature flag?

Signed-off-by: Benedikt Brunner <benedikt.brunner@pickware.de>
@MalteJanz MalteJanz self-requested a review January 31, 2025 11:54
8000
Signed-off-by: Benedikt Brunner <benedikt.brunner@pickware.de>
Signed-off-by: Benedikt Brunner <benedikt.brunner@pickware.de>
@Benedikt-Brunner
Copy link
Contributor Author

Hi @MalteJanz,

first of all i want to thank the team for being open to exploring the proposed changes even though their main selling point is now void. I have implemented steps 1-3, i hope i got it all relatively correct. Regarding the proposed Github discussion, should i open that or will shopware handle this?

@MalteJanz
Copy link
Contributor

@Benedikt-Brunner If you want you can also start the GH discussion, I guess you will be quicker than me and are already driving this topic 💪 . It should be an RFC to change the default flow execution model for 6.8 and basically summarize all our discussed points:

  • Status quo and what disadvantages it has
  • Proposed change, can be checked out behind feature flag XY
  • Benefits of that change
  • Other considered alternatives, like using the message queue and it's disadvantages
  • link to ADR, this PR discussion
  • Call for further discussion with focus on discovering concerns that speak against making this change or finding better alternatives

Co-authored-by: Malte Janz <service.malte.j@protonmail.com>
@Benedikt-Brunner
Copy link
Contributor Author

@MalteJanz i seem to lack the necessary permissions for creating a discussion 😅.

@MalteJanz
Copy link
Contributor

@Benedikt-Brunner Alright, then I will create one. Let's first try to get this merged 🙂

< 67E6 div data-view-component="true" class="TimelineItem"> AydinHassan
@MalteJanz MalteJanz added this pull request to the merge queue Feb 5, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Feb 5, 2025
@Benedikt-Brunner
Copy link
Contributor Author

@MalteJanz the failed status checks seem unrelated, should i adjust something?

@MalteJanz
Copy link
Contributor

@Benedikt-Brunner no worries, I also think they were unrelated. We had some issues with our pipelines lately + a lot of stuff is getting merged for the major and because this change isn't urgent we might wait until next week, before trying again ✌️

@MalteJanz MalteJanz self-assigned this Feb 12, 2025
@MalteJanz MalteJanz added this pull request to the merge queue Feb 12, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Feb 12, 2025
@AydinHassan
Copy link
Contributor

@MalteJanz i'll put this back in the queue, I think it just failed in between two dependent platform/commercial prs being merged.

@AydinHassan AydinHassan added this pull request to the merge queue Feb 12, 2025
@Benedikt-Brunner Benedikt-Brunner changed the title feat: isolate flow execution from business process and add flow execution log feat: isolate flow execution from business process Feb 12, 2025
Merged via the queue into shopware:trunk with commit 59f4999 Feb 12, 2025
39 checks passed
@Benedikt-Brunner Benedikt-Brunner deleted the hacktoberfest/isolate-flow-execution-from-business-process branch February 12, 2025 11:37
@MalteJanz
Copy link
Contributor

@Benedikt-Brunner as promised I created the discussion here: #6750

I hope it's fine that I reused parts of your ADR to kickstart the discussion with some context. Let me know if the discussion description is missing something important that I should edit. Otherwise let's see what feedback we get and thanks again for your effort pushing this topic forward 🚀 🙂

@Benedikt-Brunner
Copy link
Contributor Author

Hey @MalteJanz, thanks for opening the Discussion 🥳. The description looks very nice overall, however i would like to distinguish the point about the impact on the business process. While the main error, regarding transaction handling, is now no longer relevant, there are still some similar pain points. Specifically, the flow code still runs during the business process allowing unexpected state mutations during business process execution. Further fatal application crashes (while rare, still very possible in PHP) could still crash the application before the business process can be completed. So i would propose removing the dashed out portions of the text and mention both, that the transaction handling is no longer a concern and that there are still similar concerns we want to address with this change. I can provide text snippets if desired 👋

raffaelecarelle pushed a commit to raffaelecarelle/shopware that referenced this pull request Mar 1, 2025
Signed-off-by: Benedikt Brunner <benedikt.brunner@pickware.de>
Co-authored-by: Malte Janz <service.malte.j@protonmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants
0