8000 DOC-434 | [APM-283] Add release notes and update documentation for pa… · larsborn/arangodb-docs@dcb7c7f · GitHub
[go: up one dir, main page]
Skip to content

Commit dcb7c7f

Browse files
mpoeterSimran-Bnerpaula
authored
DOC-434 | [APM-283] Add release notes and update documentation for parallel gather (arangodb#1308)
* Add release notes and update documentation * Update release notes * Review --------- Co-authored-by: Simran Spiller <simran@arangodb.com> Co-authored-by: Simran <Simran-B@users.noreply.github.com> Co-authored-by: Paula Mihu <97217318+nerpaula@users.noreply.github.com>
1 parent 281b31c commit dcb7c7f

File tree

3 files changed

+45
-6
lines changed

3 files changed

+45
-6
lines changed

‎3.11/aql/execution-and-performance-optimizer.md

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -718,12 +718,11 @@ The following optimizer rules may appear in the `rules` attribute of
718718
directly works with the documents on the DB-Servers.
719719

720720
- `parallelize-gather`:
721-
Appear if an optimization to execute Coordinator `GatherNodes` in
722-
parallel is applied. `GatherNode`s go into parallel mode only if the
723-
DB-Server query part above it (in terms of query execution plan layout) is a
724-
terminal part of the query. To trigger the optimization, there must not be
725-
other nodes of type `ScatterNode`, `GatherNode` or `DistributeNode` present
726-
in the query.
721+
Appears if an optimization to execute Coordinator `GatherNode`s in
722+
parallel is applied. `GatherNodes`s cannot be parallelized if they depend on a
723+
`TraversalNode`, except for certain Disjoint SmartGraph traversals where the
724+
traversal can run completely on the local DB-Server (only available in the
725+
Enterprise Edition).
727726

728727
- `push-subqueries-to-dbserver` _(Enterprise Edition only)_:
729728
Appears if a subquery is determined to be executable entirely on a database

‎3.11/release-notes-new-features311.md

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,37 @@ example query, but you can also specify your preferred method explicitly.
8888

8989
See the [`COLLECT` options](aql/operations-collect.html#method) for details.
9090

91+
### Parallel gather
92+
93+
On Coordinators in cluster deployments, results from different DB-Servers are
94+
combined into a stream of results. This process is called gathering. It shows as
95+
`GatherNode` nodes in the execution plan of AQL queries.
96+
97+
Previously, a cluster AQL query could only parallelize a `GatherNode` if the
98+
DB-Server query part above it (in terms of query execution plan layout) was a
99+
terminal part of the query. That means that it was not allowed for other nodes of
100+
type `ScatterNode`, `GatherNode`, or `DistributeNode` to be present in the query.
101+
102+
Modification queries were also not allowed to use parallel gather unless the
103+
`--query.parallelize-gather-writes` startup option was enabled, which defaulted
104+
to `false`.
105+
106+
From v3.11.0 onward, these limitations are removed so that parallel gather can be
107+
used in almost all queries. As a result, the feature is enabled by default and
108+
the `--query.parallelize-gather-writes` startup option is now obsolete. You can
109+
still disable the optimization by disabling the `parallelize-gather` AQL
110+
optimizer rule.
111+
112+
The only case where parallel gather is not supported is when using traversals,
113+
although there are some exceptions for Disjoint SmartGraphs where the traversal
114+
can run completely on the local DB-Server (only available in the Enterprise Edition).
115+
116+
The parallel gather optimization can not only speed up queries quite significantly,
117+
but also overcome issues with the previous serial processing within `GatherNode`
118+
nodes, which could lead to high memory usage on Coordinators caused by buffering
119+
of documents for other shards, and timeouts on some DB-Servers because query parts
120+
were idle for too long.
121+
91122
### Extended peak memory usage reporting
92123

93124
The peak memory usage of AQL queries is now also reported for running queries

‎3.11/release-notes-upgrading-changes311.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -109,6 +109,15 @@ version of ArangoDB. Setting the option to anything but the value of
109109
From v3.11.0 onwards, this option is deprecated, and setting it to a value
110110
different than the value of `--agency.size` leads to a startup error.
111111

112+
### `--query.parallelize-gather-writes` obsoleted
113+
114+
Parallel gather is now enabled by default and supported for most queries.
115+
The `--query.parallelize-gather-writes` startup option has no effect anymore,
116+
but specifying it still tolerated.
117+
118+
See [Features and Improvements in ArangoDB 3.11](release-notes-new-features311.html#parallel-gather)
119+
for details.
120+
112121
### `--pregel.memory-mapped-files*` obsoleted
113122

114123
Pregel no longer supports use memory-mapped files as a backing storage.

0 commit comments

Comments
 (0)
0