8000 protoype for forceOneShardAttributeValue by jsteemann · Pull Request #14707 · arangodb/arangodb · GitHub
[go: up one dir, main page]

Skip to content

protoype for forceOneShardAttributeValue #14707

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Sep 7, 2021

Conversation

jsteemann
Copy link
Contributor
@jsteemann jsteemann commented Aug 26, 2021

Scope & Purpose

Backport of #14701

Enterprise companion PR: https://github.com/arangodb/enterprise/pull/745

(Enterprise Edition only): added query option forceOneShardAttributeValue to explicitly set a shard key value that will be used during query snippet distribution to limit the query to a specific server in the cluster.

This query option can be used in complex queries in case the query optimizer cannot automatically detect that the query can be limited to only a single server (e.g. in a disjoint smart graph case).
When the option is set to the correct shard key value, the query will be limited to the target server determined by the shard key value. It thus requires that all collections in the query use the same distribution (i.e. distributeShardsLike attribute via disjoint SmartGraphs).

Limiting the query to a single DB server is a performance optimization and may make complex queries run a lot faster because of the reduced setup and teardown costs and the reduced cluster-internal traffic during query execution.

If the option is set incorrectly, i.e. to a wrong shard key value, then the query may be shipped to a wrong DB server and may not return results (i.e. empty result set). It is thus the caller's responsibility to set the forceOneShardAttributeValue correctly or not use it.

The forceOneShardAttributeValue option will only honor string values. All other values as well as the empty string will be ignored and treated as if the option is not set.

If the option is set and the query satisfies the requirements for using the option, the query's execution plan will contain the "cluster-one-shard" optimizer rule.

  • 💩 Bugfix (requires CHANGELOG entry)
  • 🍕 New feature (requires CHANGELOG entry, feature documentation and release notes)
  • 🔥 Performance improvement
  • 🔨 Refactoring/simplification
  • 📖 CHANGELOG entry made

Backports:

  • No backports required

Related Information

Testing & Verification

  • The behavior in this PR was manually tested
  • This PR adds tests that were used to verify all changes:
    • Added new integration tests (shell_server, in enterprise repository)

goedderz and others added 4 commits August 26, 2021 16:44
@jsteemann jsteemann added this to the 3.8 milestone Aug 26, 2021
@KVS85 KVS85 added 3 Cluster 3 Optimizer AQL query optimizer is involved labels Aug 26, 2021
@KVS85 KVS85 modified the milestones: 3.8, 3.8.2 Aug 26, 2021
@jsteemann jsteemann marked this pull request as draft August 26, 2021 17:00
@hkernbach
Copy link
Member

Jenkins run green

Copy link
Member
@mchacki mchacki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the Part that is supposed for this forceOneShard is good. (Would approve)

However this PR seems to contain unrelated parts (especially request lanes in AQL)
I just want to avoid merging issues for the other PR, or having something included unter the radar). So maybe we need to make sure to merge in correct order.
If this comment is sorted out count this as approved!

@jsteemann jsteemann marked this pull request as ready for review September 7, 2021 08:51
Copy link
Member
@mchacki mchacki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
My last comments have been sorted out, as the unrelated changes are part of BaseBranch now 👍

@KVS85 KVS85 merged commit 49be0ea into 3.8 Sep 7, 2021
@KVS85 KVS85 deleted the feature-3.8/forceOneShardAttributeValue branch September 7, 2021 10:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3 Cluster 3 Optimizer AQL query optimizer is involved 9 Ready to Merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants
0