-
Notifications
You must be signed in to change notification settings - Fork 857
Cluster overwhelm countermeasures #13108
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 1 commit
Commits
Show all changes
106 commits
Select commit
Hold shift + click to select a range
91377a8
Configurable parameter for synchronous replication timeout and better…
42356c1
Some experiments
28c856e
Merge remote-tracking branch 'origin/devel' into feature/prevent-clus…
neunhoef 3b48bb6
Take out misuse of set_symmetric_difference.
neunhoef 677337e
Introduce _onGoing counter.
neunhoef f14b515
Merge remote-tracking branch 'origin/devel' into feature/prevent-clus…
neunhoef a57773b
Add metrics and implement throttle on coordinators.
neunhoef 2d1def6
Fix atomicity of metrics.
neunhoef ec55028
Fix metrics atomics.
neunhoef 64c2c8b
fixed gauge assignment operatorsd
kvahed d3a52a8
tests
kvahed aa854ed
tiny
kvahed 578a0e3
fixed metrics for correct floating point atomics
kvahed 349840b
Merge remote-tracking branch 'origin/devel' into feature/prevent-clus…
neunhoef 8a1f00d
devel merge
kvahed 44f45d1
better testing
kvahed 1b8ddc5
Scheduler revamp, lots of other changes.
neunhoef 12ef863
Essentially get rid of synchronous replication timeout.
neunhoef 2b2a95a
Take out unnecessary option.
neunhoef d437b6b
Replication of commit and abort transaction on high prio queue.
neunhoef 59c96ff
Merge remote-tracking branch 'origin/devel' into feature/prevent-clus…
neunhoef ee23e9d
Trigger RocksDB throttle earlier.
neunhoef 2f54ff5
Take out lots of debugging output.
neunhoef fc180c8
Merge branch 'devel' into feature/prevent-cluster-overwhelm2
dab1d46
Merge remote-tracking branch 'origin/devel' into feature/prevent-clus…
neunhoef 6280342
Do not change RocksDB write buffer number after all.
neunhoef 7582e38
Do not touch RocksDB throttle for now.
neunhoef a5f5020
Some cleanup w.r.t. ongoing throtteling.
neunhoef d354ab1
Permute some definitions, more like devel.
neunhoef c0a0cb0
Take out inflight again.
neunhoef 7cf76e2
Remove unused code.
neunhoef 7b82262
Whitespacing.
neunhoef c42fe2c
Rearrange stuff in SupervisedScheduler.
neunhoef 5231dbc
More rearranging.
neunhoef 6e1980f
Transaction time limit down to 10s again.
neunhoef e119ae9
Restore startLocalCluster.sh
neunhoef c19a9bb
Adjust a comment.
neunhoef 400c1fb
Adjust lower limit for replication timeout.
neunhoef aff9c2d
CHANGELOG.
neunhoef 9d466a1
Try to fix Windows compilation.
neunhoef 4f1bc5e
Merge branch 'feature/prevent-cluster-overwhelm2' of github.com:arang…
45acdb1
Merge branch 'devel' into feature/prevent-cluster-overwhelm2
3d5e9b8
Prepare for merge.
893783e
Merge other branch
dc8159a
Fix Mac compilation.
215831e
Move AQL continuations to new lane
0c1437f
Actually move to new lane
e463e39
Remove synchronous commit from document handler
1048caa
Merge branch 'devel' into feature/prevent-cluster-overwhelm2
ac5a331
Revert temporary workaround for issue that should be fixed elsewhere
d9b9da2
Ensure max is at least min
467822d
Make ongoing task accounting clearer
dea468d
Clarification
fc97d24
Fix plurality for consistency
5fe4b0d
Address review comment
70c6692
Fix compilation
92fbaf8
Update arangod/Replication/ReplicationMetricsFeature.cpp
cff195d
Update arangod/Replication/ReplicationMetricsFeature.cpp
983c57c
Update arangod/Scheduler/SchedulerFeature.cpp
d44c59b
Update arangod/Scheduler/SchedulerFeature.cpp
f1cfcbe
Cleanup new parameters
d5957f4
Update arangod/Scheduler/SupervisedScheduler.cpp
dc4bf54
Fix atomic ordering for gauge
c02ac9f
Merge branch 'feature/prevent-cluster-overwhelm2' of github.com:arang…
359fb16
Add introduced for new flag
bea1b10
Update arangod/Scheduler/SupervisedScheduler.h
21f4215
Update arangod/Scheduler/SupervisedScheduler.cpp
fb354cf
Update arangod/Scheduler/SupervisedScheduler.cpp
01d9f45
Update arangod/Scheduler/SupervisedScheduler.cpp
03b3d27
Update arangod/Scheduler/SupervisedScheduler.cpp
5f3999a
Rename some variables
43593e8
Merge branch 'feature/prevent-cluster-overwhelm2' of github.com:arang…
363ef2c
Add introduced for new parameter
dab2218
Deredundify some wording
df74bce
Set another introduced flag
acf47a0
Update arangod/Scheduler/SupervisedScheduler.cpp
db009e1
Save map lookup
6872713
Merge branch 'feature/prevent-cluster-overwhelm2' of github.com:arang…
c73af0d
Merge branch 'devel' into bug-fix/gauge-assignment-opera
neunhoef 685ea8b
Address review comment about integral efficiency
6a2ff56
Revert some changes to metrics that are covered in another PR.
7f1293f
Merge branch 'devel' into feature/prevent-cluster-overwhelm2
jsteemann 1a8bd32
Fix Windows and Mac compilation
1407d37
Merge branch 'feature/prevent-cluster-overwhelm2' of github.com:arang…
3171aec
relaxed precision on tests
kvahed a861533
keep memory order as default
kvahed 186c045
Merge branch 'devel' into bug-fix/gauge-assignment-opera
neunhoef 161410a
Merge remote-tracking branch 'origin/devel' into feature/prevent-clus…
neunhoef 3934fb4
Merge remote-tracking branch 'origin/bug-fix/gauge-assignment-opera' …
neunhoef 1f16b99
Fix compilation.
neunhoef 056d0f0
Add missing headers
e2d8d67
Address some review comments
7ce9059
Merge branch 'devel' into feature/prevent-cluster-overwhelm2
825ed38
Fix some issues
c12a833
Fix a dependency
f54d0bb
Fix feature dependency in mock
9398260
Fix feature re-add
65badf8
fix ubsan issues
jsteemann f27e69c
speed up shortest path unit tests
jsteemann 2938056
fix MSVC compile warning
jsteemann ab6ce5b
8000
Merge branch 'devel' of github.com:arangodb/arangodb into feature/clu…
jsteemann 449877b
Merge branch 'devel' of github.com:arangodb/arangodb into feature/pre…
jsteemann 0af7ff7
fix potential nullptr access
jsteemann 12f0e70
make APIs of network::Response safer to use
jsteemann 5a10472
Merge branch 'devel' of github.com:arangodb/arangodb into feature/pre…
jsteemann 24d3d0c
Merge branch 'devel' into feature/prevent-cluster-overwhelm2
jsteemann File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
fix potential nullptr access
- Loading branch information
commit 0af7ff72963be71f228b155a84cb0edfacaed6b3
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
<
8000
svg aria-hidden="true" height="16" viewBox="0 0 16 16" version="1.1" width="16" data-view-component="true" class="octicon octicon-fold-up">
|
@@ -297,10 +297,10 @@ arangodb::AsyncAgencyComm::FutureResult agencyAsyncSend(AsyncAgencyCommManager& | |
|
||
auto& req = result.request; | ||
auto& resp = result.response; | ||
auto& body = *req; | ||
|
||
switch (result.error) { | ||
case Error::NoError: | ||
case Error::NoError: { | ||
TRI_ASSERT(req != nullptr); | ||
|
||
LOG_TOPIC("aac88", TRACE, Logger::AGENCYCOMM) | ||
<< "agencyAsyncSend [" << meta.requestId | ||
|
@@ -325,6 +325,7 @@ arangodb::AsyncAgencyComm::FutureResult agencyAsyncSend(AsyncAgencyCommManager& | |
redirectOrError(man, endpoint, location); | ||
|
||
// send again | ||
auto& body = *req; | ||
return ::agencyAsyncSend(man, std::move(meta), std::move(body).moveBuffer()); | ||
} | ||
|
||
|
@@ -333,50 +334,64 @@ arangodb::AsyncAgencyComm::FutureResult agencyAsyncSend(AsyncAgencyCommManager& | |
return futures::makeFuture( | ||
AsyncAgencyCommResult{result.error, std::move(resp)}); | ||
} | ||
|
||
[[fallthrough]]; | ||
/* fallthrough */ | ||
case Error::ConnectionCanceled: | ||
if (man.server().isStopping()) { | ||
} | ||
[[fallthrough]]; | ||
|
||
case Error::ConnectionCanceled: { | ||
if (man.server().isStopping() || req == nullptr) { | ||
return futures::makeFuture( | ||
AsyncAgencyCommResult{result.error, std::move(resp)}); | ||
} | ||
[[fallthrough]]; | ||
|
||
TRI_ASSERT(req != nullptr); | ||
} | ||
[[fallthrough]]; | ||
|
||
case Error::RequestTimeout: | ||
case Error::ConnectionClosed: | ||
case Error::ProtocolError: | ||
case Error::WriteError: | ||
case Error::ReadError: | ||
case Error::CloseRequested: | ||
// inquiry the request | ||
case Error::CloseRequested: { | ||
TRI_ASSERT(req != nullptr); | ||
|
||
// inquire the request | ||
man.reportError(endpoint); | ||
// in case of a write transaction we have to do inquiry | ||
if (meta.isInquiryOnNoResponse()) { | ||
TRI_ASSERT(req != nullptr); | ||
auto& body = *req; | ||
return ::agencyAsyncInquiry(man, std::move(meta), | ||
std::move(body).moveBuffer()); | ||
} else if (!meta.isRetryOnNoResponse()) { | ||
// return error | ||
return futures::makeFuture( | ||
AsyncAgencyCommResult{result.error, std::move(resp)}); | ||
} | ||
// otherwise just send again | ||
[[fallthrough]]; | ||
} | ||
// otherwise just send again | ||
[[fallthrough]]; | ||
|
||
case Error::CouldNotConnect: | ||
case Error::CouldNotConnect: { | ||
LOG_TOPIC("aac90", TRACE, Logger::AGENCYCOMM) | ||
<< "agencyAsyncSend [" << meta.requestId << "] retry request soon"; | ||
// retry to send the request | ||
man.reportError(endpoint); | ||
[[fallthrough]]; | ||
} | ||
[[fallthrough]]; | ||
|
||
case Error::QueueCapacityExceeded: | ||
return ::agencyAsyncSend(man, std::move(meta), | ||
std::move(body).moveBuffer()); // retry always | ||
case Error::QueueCapacityExceeded: { | ||
TRI_ASSERT(req != nullptr); | ||
auto& body = *req; | ||
return ::agencyAsyncSend(man, std::move(meta), | ||
std::move(body).moveBuffer()); // retry always | ||
} | ||
|
||
case Error::VstUnauthorized: | ||
case Error::VstUnauthorized: { | ||
return futures::makeFuture( | ||
AsyncAgencyCommResult{result.error, | ||
std::move(resp)}); // unrecoverable error | ||
} | ||
} | ||
|
||
ADB_UNREACHABLE; | ||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.