8000 feat: Add support for multiplexed sessions by currantw · Pull Request #1383 · googleapis/python-spanner · GitHub
[go: up one dir, main page]

Skip to content

feat: Add support for multiplexed sessions #1383

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 41 commits into
base: main
Choose a base branch
from

Conversation

currantw
Copy link
@currantw currantw commented May 29, 2025

feat: Add support for multiplexed sessions:

  • Enable the use of multiplexed session for partitioned transactions.
  • Support disabling multiplexed for all client database instances in case of failure.
  • Support precommit tokens ingest and output in commit requests.
  • Add mutation key to begin transaction requests for mutations-only transactions.
  • Retry commit if commit response contains a precommit token.
  • Deprecate methods that do not support multiplexed sessions.
  • Cleanup: add type hints; change class attributes to instance attributes.

@currantw currantw requested review from a team as code owners May 29, 2025 19:42
@product-auto-label product-auto-label bot added the size: l Pull request size is large. label May 29, 2025
@product-auto-label product-auto-label bot added the api: spanner Issues related to the googleapis/python-spanner API. label May 29, 2025
@currantw currantw force-pushed the support-mux-session-2 branch from 57902b4 to c8e7a1c Compare May 29, 2025 19:45
@lszinv lszinv added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label May 29, 2025
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label May 29, 2025
environ[SessionOptions.ENV_VAR_ENABLE_MULTIPLEXED] = "true"
environ[SessionOptions.ENV_VAR_ENABLE_MULTIPLEXED_FOR_PARTITIONED] = "true"
environ[SessionOptions.ENV_VAR_ENABLE_MULTIPLEXED_FOR_READ_WRITE] = "true"
environ[SessionOptions.ENV_VAR_FORCE_DISABLE_MULTIPLEXED] = "false"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need to force disable multiplexed

Copy link
Author
@currantw currantw May 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need it at all? Is this environment variable is being completely removed from the scope of the project?

@currantw currantw force-pushed the support-mux-session-2 branch from 0bd4e7b to 98e6b13 Compare May 30, 2025 19:00
@product-auto-label product-auto-label bot added size: xl Pull request size is extra large. and removed size: l Pull request size is large. labels May 30, 2025
currantw added 18 commits June 4, 2025 13:22
…E_MULTIPLEXED_SESSIONS` and add unit tests.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
…` exception from `DatabaseSessionManager` and add unit tests.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
…essions, add unit tests.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
Signed-off-by: Taylor Curran <taylor.curran@improving.com>
Signed-off-by: Taylor Curran <taylor.curran@improving.com>
…shotBase` and update attributes and tests.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
…edundant as transaction ID is set via `_restart_on_unavailable`)

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
…to `Client` so that multiplexed are disabled for _all_ databases.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
…Database.run_in_transaction` to not use it.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
… cleanup.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
…ession manager.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
…s from class attributes to instance attributes.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
…d `database.session()`

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
…s, add TODOs, and make `Session._transaction` default to None. Plus add some `Optional` typing hints.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
…hot` to `_SnapshotBase` and update unit tests.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
…nsaction`, move to base class, update tests.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
…ransactions.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
… checks, add `raises` documentation.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
currantw added 9 commits June 4, 2025 13:22
…that it doesn't depend on environment variable values.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
…`DatabaseSessionManager`.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
Signed-off-by: Taylor Curran <taylor.curran@improving.com>
…edentials.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
…for testing.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
Signed-off-by: Taylor Curran <taylor.curran@improving.com>
…nsert_then_rollback`.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
Signed-off-by: Taylor Curran <taylor.curran@improving.com>
…e class, simplify `_SnapshotBase` tests, remove redundant tests.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
@currantw currantw force-pushed the support-mux-session-2 branch from 2e8ccf8 to 72574d4 Compare June 4, 2025 20:23
…d_insert_then_rollback` and add `build_request_id` helper method, fix `test_snapshot` and `test_transaction` failures.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
@currantw currantw force-pushed the support-mux-session-2 branch from 9c6574b to 418edda Compare June 4, 2025 21:06
currantw added 5 commits June 4, 2025 15:15
…d by maintenance thread.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
…r `_SnapshotBase`.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
… constants for next step.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
…ew multiplexed behaviour.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
Signed-off-by: Taylor Curran <taylor.curran@improving.com>
@currantw currantw changed the title feat: Add support for multiplexed sessions - partitioned feat: Add support for multiplexed sessions Jun 5, 2025
@@ -49,6 +49,10 @@ def with_request_id(
return all_metadata


def build_request_id(client_id, channel_id, nth_request, attempt):
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added helper method to reduce duplication in tests.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
Comment on lines 88 to 93
elif transaction_type is TransactionType.READ_WRITE:
return False
return (
self._getenv(self.ENV_VAR_ENABLE_MULTIPLEXED)
and self._getenv(self.ENV_VAR_ENABLE_MULTIPLEXED_FOR_READ_WRITE)
and self._is_multiplexed_enabled[transaction_type]
)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

< 9E7A div class="comment-body markdown-body js-comment-body soft-wrap css-overflow-wrap-anywhere user-select-contain d-block">

Read/write transaction are supported here, but are disabled in DatabaseSessionManager. Will be turning that on in the next PR.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Helpers methods. Added to help simply unit tests.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
@rahul2393 rahul2393 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jun 5, 2025
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jun 5, 2025
currantw added 3 commits June 5, 2025 13:06
…multiplexed sessions due to runtime failures.

Signed-off-by: Taylor Curran <taylor.curran@improving.com>
Signed-off-by: Taylor Curran <taylor.curran@improving.com>
Signed-off-by: Taylor Curran <taylor.curran@improving.com>
@rahul2393 rahul2393 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jun 10, 2025
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jun 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: spanner Issues related to the googleapis/python-spanner API. size: xl Pull request size is extra large.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants
0