8000 APM-84 | Add `spillOverThreshold*` options to Cursor API DocuBlock by Simran-B · Pull Request #17700 · arangodb/arangodb · GitHub
[go: up one dir, main page]
Skip to content

APM-84 | Add spillOverThreshold* options to Cursor API DocuBlock #17700

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jan 9, 2023
Merged

APM-84 | Add spillOverThreshold* options to Cursor API DocuBlock #17700

Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
1fe6dfc
Cursor API: Add spillOverThreshold* options to DocuBlock
Simran-B Nov 30, 2022
d844611
Query spillover startup option: fix typo, add unit
Simran-B Nov 30, 2022
cc3e85c
Fix note
Simran-B Dec 2, 2022
4e6c95b
Cursor API supports a maxNumberOfPlans option, not maxPlans
Simran-B Dec 2, 2022
06a7d6d
Document allowDirtyReads query options of Cursor API
Simran-B Dec 2, 2022
13d2507
Merge branch 'devel' of https://github.com/arangodb/arangodb into doc…
Simran-B Dec 28, 2022
e44e8a9
Merge branch 'devel' of https://github.com/arangodb/arangodb into doc…
KVS85 Jan 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Cursor API: Add spillOverThreshold* options to DocuBlock
  • Loading branch information
@Simran-B
Simran-B committed Nov 30, 2022
commit 1fe6dfc91bc9db968cc4ce9577c10cd910de6354
40 changes: 40 additions & 0 deletions Documentation/DocuBlocks/Rest/Cursors/post_api_cursor.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,46 @@ stores the full result in memory (on the contacted Coordinator if in a cluster).
All other resources are freed immediately (locks, RocksDB snapshots). The query
will fail before it returns results in case of a conflict.

@RESTSTRUCT{spillOverThresholdMemoryUsage,post_api_cursor_opts,integer,optional,}
This option allows queries to store intermediate and final results temporarily
on disk if the amount of memory used (in bytes) exceeds the specified value.
This is used for decreasing the memory usage during the query execution.

This option only has an effect on queries that use the `SORT` operation but
without a `LIMIT`, and if you enable the spillover feature by setting a path
for the directory to store the temporary data in with the
`--temp.intermediate-results-path` startup option.

Default value: 128MB.

Note:
Spilling data from RAM onto disk is an experimental feature and is turned off
by default. The query results are still built up entirely in RAM on Coordinators
and single servers for non-streaming queries. To avoid the buildup of
the entire query result in RAM, use a streaming query (see the `stream` option).

@RESTSTRUCT{spillOverThresholdNumRows,post_api_cursor_opts,integer,optional,}
This option allows queries to store intermediate and final results temporarily
on disk if the number of rows produced by the query exceeds the specified value.
This is used for decreasing the memory usage during the query execution. In a
query that iterates over a collection that contains documents, each row is a
document, and in a query that iterates over temporary values
(i.e. `FOR i IN 1..100`), each row is one of such temporary values.

**Note**:
This option only has an effect on queries that use the `SORT` operation but
without a `LIMIT`, and if you enable the spillover feature by setting a path
for the directory to store the temporary data in with the
`--temp.intermediate-results-path` startup option.

Default value: `5000000` rows.

**Note**:
Spilling data from RAM onto disk is an experimental feature and is turned off
by default. The query results are still built up entirely in RAM on Coordinators
and single servers for non-streaming queries. To avoid the buildup of
the entire query result in RAM, use a streaming query (see the `stream` option).

@RESTSTRUCT{optimizer,post_api_cursor_opts,object,optional,post_api_cursor_opts_optimizer}
Options related to the query optimizer.

Expand Down
0