10000 [APM-84] Support for external result sets by cpjulia · Pull Request #16421 · arangodb/arangodb · GitHub
[go: up one dir, main page]
Skip to content

[APM-84] Support for external result sets #16421

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 104 commits into from
Jul 17, 2022
Merged

[APM-84] Support for external result sets #16421

Changes from 1 commit
Commits
Show all changes
104 commits
Select commit Hold shift + click to select a range
acbd780
Started creating API for rocksdb temp storage of query entries and mo…
cpjulia Jun 17, 2022
3fa9f08
Started a rocksdb instance using a feature as API wrapper instead of …
cpjulia Jun 20, 2022
cc18c9e
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
cpjulia Jun 20, 2022
770ca3f
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
cpjulia Jun 21, 2022
1e713e1
"Added sort executor to execution list"
cpjulia Jun 22, 2022
5384607
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
cpjulia Jun 22, 2022
4c4b07b
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
cpjulia Jun 23, 2022
fd2bb53
Implemented rocksdb storage for SORT entries over threshold
cpjulia Jun 23, 2022
f09a4a1
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
cpjulia Jun 23, 2022
711d971
Updated tests with new signature
cpjulia Jun 23, 2022
b82e51f
Added deletion from rocksdb, fixed typo
cpjulia Jun 24, 2022
b5fd404
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
cpjulia Jun 24, 2022
90108c5
Started using RocksDBSstFileMethods for insertions with SstFileWriter…
cpjulia Jun 27, 2022
3de891e
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
cpjulia Jun 28, 2022
4368dac
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
cpjulia Jun 30, 2022
d8bca10
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
cpjulia Jul 1, 2022
b861d4a
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
cpjulia Jul 4, 2022
222e9d5
Updated sort executor
cpjulia Jul 6, 2022
3e3870d
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
cpjulia Jul 6, 2022
da85037
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
cpjulia Jul 6, 2022
070ea0d
Updated sort executor for compilation
cpjulia Jul 6, 2022
d53332b
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
jsteemann Jul 6, 2022
e1e5201
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
jsteemann Jul 7, 2022
992e592
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
jsteemann Jul 7, 2022
e4f5002
create a storage-engine-agnostic TemporaryStorage feature
jsteemann Jul 7, 2022
497a1ac
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
jsteemann Jul 7, 2022
10fd9bb
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
jsteemann Jul 7, 2022
015a0b3
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
jsteemann Jul 7, 2022
b902ad6
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
jsteemann Jul 7, 2022
fa9a627
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
jsteemann Jul 7, 2022
adcf42d
add RocksDB implementation for sorted rows storage backend
jsteemann Jul 8, 2022
6a53d27
added two-stage storage backend, with spillover from memory to rocksdb
jsteemann Jul 8, 2022
b933c43
fix staged backend
jsteemann Jul 8, 2022
3acc65b
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
jsteemann Jul 8, 2022
bab4078
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
jsteemann Jul 10, 2022
1685275
allow using process id ('$PID') in startup parameter
jsteemann Jul 11, 2022
ea0c73b
allow using process id ('$PID') in startup parameter
jsteemann Jul 11, 2022
0b80e37
adjust CHANGELOG
jsteemann Jul 11, 2022
3adc358
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
jsteemann Jul 11, 2022
3f94343
fix encryption
jsteemann Jul 11, 2022
c6f6a90
prevent duplicate cleanup
jsteemann Jul 11, 2022
81d69ca
make log message less verbose
jsteemann Jul 11, 2022
036f37c
set several RocksDB options
jsteemann Jul 11, 2022
008060b
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
jsteemann Jul 11, 2022
7fd9621
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
cpjulia Jul 11, 2022
53011e2
Merge branch 'feature/APM-84-rocksdb-store' of github.com:arangodb/ar…
cpjulia Jul 11, 2022
9c9f2a5
Added the startup option of temp dir storage --temp.intermediate-resu…
cpjulia Jul 11, 2022
ce8c665
turn off intermediate storage on agents
jsteemann Jul 11, 2022
88b22d7
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
cpjulia Jul 11, 2022
425bb86
Merge branch 'feature/APM-84-rocksdb-store' of github.com:arangodb/ar…
cpjulia Jul 11, 2022
c543892
Changed name of rocksdb comparator for temp storage in queries
cpjulia Jul 11, 2022
08c8e08
fix performance regression in in-memory sorter
jsteemann Jul 11, 2022
adbdeea
Merge branch 'feature/APM-84-rocksdb-store' of github.com:arangodb/ar…
jsteemann Jul 11, 2022
ace79d3
fix assertion failure
jsteemann Jul 11, 2022
055d1bf
Fixed comparator name
cpjulia Jul 11, 2022
eed8d1c
simplification
jsteemann Jul 12, 2022
fccb108
follow coding guidelines
jsteemann Jul 12, 2022
4ea41e7
micro optimization for serialization
jsteemann Jul 12, 2022
379219e
Added threshold memory usage parameter to query
cpjulia Jul 12, 2022
fdb7c9c
Added test, to be continued
cpjulia Jul 12, 2022
0615afb
Merge branch 'feature/APM-84-rocksdb-store' of github.com:arangodb/ar…
cpjulia Jul 12, 2022
8878517
fix broken tests
jsteemann Jul 12, 2022
c8a7247
split tests
jsteemann Jul 12, 2022
c0c3a76
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
jsteemann Jul 12, 2022
b9a9761
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
cpjulia Jul 12, 2022
1db6ac4
Merge branch 'feature/APM-84-rocksdb-store' of github.com:arangodb/ar…
cpjulia Jul 12, 2022
fe2580b
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
jsteemann Jul 12, 2022
cc4346b
Merge branch 'feature/APM-84-rocksdb-store' of github.com:arangodb/ar…
jsteemann Jul 12, 2022
3515aa9
small cleanup
jsteemann Jul 12, 2022
7dc1d60
fix test
jsteemann Jul 12, 2022
b99fe7e
cleanup
jsteemann Jul 12, 2022
efa8197
added to do
jsteemann Jul 12, 2022
427c105
cleanup, failure points
jsteemann Jul 12, 2022
ba502b0
renamed "thresholdNumRows" to "spillOverThresholdNumRows"
jsteemann Jul 12, 2022
9b0515a
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
jsteemann Jul 13, 2022
f4cdae4
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
cpjulia Jul 13, 2022
0d97c8a
Merge branch 'feature/APM-84-rocksdb-store' of github.com:arangodb/ar…
jsteemann Jul 13, 2022
b89bd10
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
jsteemann Jul 13, 2022
5f29b39
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
jsteemann Jul 13, 2022
e7a9bc0
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
jsteemann Jul 13, 2022
c29c5aa
Added max disk capacity control and failure tests
cpjulia Jul 13, 2022
8c87fa5
Merge branch 'feature/APM-84-rocksdb-store' of github.com:arangodb/ar…
cpjulia Jul 13, 2022
38b9b10
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
cpjulia Jul 13, 2022
a76eb31
mark options as experimental
jsteemann Jul 14, 2022
8153763
added code comments
jsteemann Jul 14, 2022
2487416
properly honor capacity limit
jsteemann Jul 14, 2022
0da00c5
add more startup options for thresholds
jsteemann Jul 14, 2022
7b298b3
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
jsteemann Jul 14, 2022
00cb64c
fix compile errors
jsteemann Jul 14, 2022
44b2dc9
added CHANGELOG
jsteemann Jul 14, 2022
3c2e943
fix compile error on macOS
jsteemann Jul 14, 2022
4037b3e
fix another compile error
jsteemann Jul 14, 2022
b80fd3f
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
cpjulia Jul 14, 2022
bd49144
Merge branch 'feature/APM-84-rocksdb-store' of github.com:arangodb/ar…
cpjulia Jul 14, 2022
df96e65
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
jsteemann Jul 14, 2022
8c0f8e5
Update CHANGELOG
cpjulia Jul 15, 2022
a850fbd
Update CHANGELOG
cpjulia Jul 15, 2022
8ca2916
Update CHANGELOG
cpjulia Jul 15, 2022
6fa39b3
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM…
cpjulia Jul 15, 2022
3979d08
Merge branch 'feature/APM-84-rocksdb-store' of github.com:arangodb/ar…
cpjulia Jul 15, 2022
e5e4dc6
Addressed suggestion of reserve row indexes size
cpjulia Jul 15, 2022
b690358
Addressed hasReachedCapacityLimit comments
cpjulia Jul 15, 2022
290f2f1
Added comment explaining the bytes in rocksdb custom comparator
cpjulia Jul 15, 2022
6e7923a
Addressed suggestion to check if we actually have the prefixId's corr…
cpjulia Jul 15, 2022 8000
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge branch 'devel' of github.com:arangodb/arangodb into feature/APM… 
…-84-rocksdb-store
  • Loading branch information
@cpjulia
cpjulia committed Jul 1, 2022
commit d8bca1010a0e03ff664bc1351d6b61c0ea1cd51c
5 changes: 5 additions & 0 deletions CHANGELOG
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,11 @@ devel
in disck temporarily using rocksdb as a means to it, because then also the
entries can be stored when inserted on disk using a customized comparator.

* Add "AT LEAST" quantifier for array filters in AQL:

`RETURN [1,2,3][? AT LEAST (3) FILTER CURRENT > 42]`
`RETURN [1,2,3] AT LEAST (2) IN [1,2,3,4,5]`

* Introduce reading from followers in clusters. This works by offering
an additional HTTP header "x-arango-allow-dirty-read" for certain
read-only APIs. This header has already been used for active failover
Expand Down
You are viewing a condensed version of this merge commit. You can view the full changes here.
0