E5E4 experiment to check if SyncWAL works on Windows by jsteemann · Pull Request #14551 · arangodb/arangodb · GitHub
[go: up one dir, main page]
Skip to content

experiment to check if SyncWAL works on Windows #14551

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 28, 2021
Merged

experiment to check if SyncWAL works on Windows #14551

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
0689b6e
enable SyncWAL code for WIN32 as well
jsteemann Jul 28, 2021
eb28e5c
Update arangod/RocksDBEngine/Methods/RocksDBTrxBaseMethods.cpp
jsteemann Jul 28, 2021
36d9ffd
Update arangod/RocksDBEngine/RocksDBSyncThread.cpp
jsteemann Jul 28, 2021
c55c389
Merge branch 'devel' of github.com:arangodb/arangodb into experimenta…
jsteemann Jul 28, 2021
9dd72dc
added CHANGELOG entry
jsteemann Jul 28, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
devel
-----

* Honor the value of startup option `--rocksdb.sync-interval` on Windows, too.
Previously, the value was ignored and WAL syncing on Windows was using a
different code paths than on the other supported platforms. Now syncing is
unified across all platforms, and they all call RocksDB's `SyncWAL()`.

* Updated ArangoDB Starter to 0.15.1-preview-4.

* APM-132: Clean up collection statuses
Expand Down
15 changes: 1 addition & 14 deletions arangod/RocksDBEngine/Methods/RocksDBTrxBaseMethods.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -348,17 +348,6 @@ arangodb::Result RocksDBTrxBaseMethods::doCommit() {
// if we fail during commit, make sure we remove blockers, etc.
auto cleanupCollTrx = scopeGuard([this]() { _state->cleanupCollections(); });

#ifdef _WIN32
// set wait for sync flag if required
// we do this only for Windows here, because all other platforms use the
// RocksDB SyncThread to do the syncing
if (_state->waitForSync()) {
rocksdb::WriteOptions wo;
wo.sync = true;
_rocksTransaction->SetWriteOptions(wo);
}
#endif

// total number of sequence ID consuming records
uint64_t numOps = _rocksTransaction->GetNumPuts() +
_rocksTransaction->GetNumDeletes() +
Expand Down Expand Up @@ -387,8 +376,7 @@ arangodb::Result RocksDBTrxBaseMethods::doCommit() {
_state->commitCollections(_lastWrittenOperationTick);
cleanupCollTrx.cancel();

#ifndef _WIN32
// wait for sync if required, for all other platforms but Windows
// wait for sync if required
if (_state->waitForSync()) {
auto& selector = _state->vocbase().server().getFeature<EngineSelectorFeature>();
auto& engine = selector.engine<RocksDBEngine>();
Expand All @@ -402,7 +390,6 @@ arangodb::Result RocksDBTrxBaseMethods::doCommit() {
return RocksDBSyncThread::sync(engine.db()->GetBaseDB());
}
}
#endif

return Result();
}
13 changes: 0 additions & 13 deletions arangod/RocksDBEngine/RocksDBEngine.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -184,12 +184,7 @@ RocksDBEngine::RocksDBEngine(application_features::ApplicationServer& server)
_pruneWaitTimeInitial(180.0),
_maxWalArchiveSizeLimit(0),
_releasedTick(0),
#ifdef _WIN32
// background syncing is not supported on Windows
_syncInterval(0),
#else
_syncInterval(100),
#endif
_syncDelayThreshold(5000),
_requiredDiskFreePercentage(0.01),
_requiredDiskFreeBytes(16 * 1024 * 1024),
Expand Down Expand Up @@ -410,14 +405,6 @@ void RocksDBEngine::validateOptions(std::shared_ptr<options::ProgramOptions> opt
}
}

#ifdef _WIN32
if (_syncInterval > 0) {
LOG_TOPIC("68301", WARN, arangodb::Logger::CONFIG)
<< "automatic syncing of RocksDB WAL via background thread is not "
<< " supported on this platform";
}
#endif

if (_pruneWaitTimeInitial < 10) {
LOG_TOPIC("a9667", WARN, arangodb::Logger::ENGINES)
<< "consider increasing the value for "
Expand Down
5 changes: 0 additions & 5 deletions arangod/RocksDBEngine/RocksDBSyncThread.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,17 +75,12 @@ Result RocksDBSyncThread::syncWal() {
}

Result RocksDBSyncThread::sync(rocksdb::DB* db) {
#ifndef _WIN32
// if called on Windows, we would get the following error from RocksDB:
// > Not implemented: SyncWAL() is not supported for this implementation of
// WAL file
LOG_TOPIC("a3978", TRACE, Logger::ENGINES) << "syncing RocksDB WAL";

rocksdb::Status status = db->SyncWAL();
if (!status.ok()) {
return rocksutils::convertStatus(status);
}
#endif
return Result();
}

Expand Down
0