diff --git a/CHANGELOG b/CHANGELOG index 08fed4c74491..79b70fee5c81 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -1,59 +1,16 @@ devel ----- -* reverted accidental change to error handling in geo index - - In previous versions, if non-valid geo coordinates were contained in the - indexed field of a document, the document was simply ignored an not indexed. - In 3.4.0, this was accidentally changed to generate an error, which caused - the upgrade procedure to break in some cases. - -* fixed TypeError being thrown instead of validation errors when Foxx manifest - validation fails - -* fix internal issue #2786: improved confirmation dialog when clicking the - Truncate button in the Web UI - -* make `--help-all` now also show all hidden program options - - Previously hidden program options were only returned when invoking arangod or - a client tool with the cryptic `--help-.` option. Now `--help-all` simply - retuns them as well. - - The program options JSON description returned by `--dump-options` was also - improved as follows: - - - the new boolean attribute "dynamic" indicates whether the option has a dynamic - default value, i.e. a value that depends on the target host capabilities or - configuration - - - the new boolean attribute "requiresValue" indicates whether a boolean option - requires a value of "true" or "false" when specified. If "requiresValue" is - false, then the option can be specified without a boolean value following it, - and the option will still be set to true, e.g. `--server.authentication` is - identical to `--server.authentication true`. - - - the new "category" attribute will contain a value of "command" for command-like - options, such as `--version`, `--dump-options`, `--dump-dependencies` etc., - and "option" for all others. - -* fixed issue #7586: a running query within the user interface was not shown - if the active view was `Running Queries` or `Slow Query History`. - * allow usage of floating point values in AQL without leading zeros, e.g. `.1234`. Previous versions of ArangoDB required a leading zero in front of the decimal separator, i.e `0.1234`. -* fixed Foxx complaining about valid `$schema` value in manifest.json - * Foxx `req.makeAbsolute` now will return meaningful values when ArangoDB is using a unix socket endpoint. URLs generated when using a unix socket follow the format http://unix:: used by other JS tooling. * Updated joi library (Web UI), improved foxx mount path validation -* fix for supervision, which started failing servers using old transient store - * do not create `_routing` collection for new installations/new databases, as it is not needed anymore. Redirects to the web interface's login screen, which were previously handled by entries in the `_routing` collection are now handled @@ -71,74 +28,78 @@ devel Existing modules in existing `_modules` collections will remain functional even after this change -* fixed Foxx queues not retrying jobs with infinite `maxFailures` - -* force connection timeout to be 7 seconds to allow libcurl time to retry lost DNS - queries. +* disable in-memory cache for edge and traversal data on agency nodes, as it + is not needed there -* fixes a routing issue within the web ui after the use of views +* removed bundled Valgrind headers, removed JavaScript variable `valgrind` + from the `internal` module -* fixes some graph data parsing issues in the ui, e.g. cleaning up duplicate - edges inside the graph viewer. +* upgraded JEMalloc version to 5.1.0 -* in a cluster environment, the arangod process now exits if wrong credentials - are used during the startup process +* use `-std=c++14` for ArangoDB compilation -* fixes validation of allowed or not allowed foxx service mount paths within - the Web UI -* fixed handling of broken Foxx services +v3.4.1 (XXXX-XX-XX) +------------------- - Installation now also fails when the service encounters an error when - executed. Upgrading or replacing with a broken service will still result - in the broken services being installed. +* fix thread shutdown in _WIN32 builds -* restored error pages for broken Foxx services + Previous versions used a wrong comparison logic to determine the current + thread id when shutting down a thread, leading to threads hanging in their + destructors on thread shutdown - Services that could not be executed will now show an error page (with helpful - information if development mode is enabled) instead of a generic 404 response. - Requests to the service that do not prefer HTML (i.e. not a browser window) - will receive a JSON formatted 503 error response instead. +* reverted accidental change to error handling in geo index -* added support for `force` flag when upgrading Foxx services + In previous versions, if non-valid geo coordinates were contained in the + indexed field of a document, the document was simply ignored an not indexed. + In 3.4.0, this was accidentally changed to generate an error, which caused + the upgrade procedure to break in some cases. - Using the `force` flag when upgrading or replacing a service falls back to - installing the service if it does not already exist. +* fixed TypeError being thrown instead of validation errors when Foxx manifest + validation fails -* the query editor within the web ui is now catching HTTP 501 responses - properly. +* make AQL REMOVE operations use less memory with the RocksDB storage engine -* Fixed an AQL bug where the optimize-traversals rule was falsely applied to - extensions with inline expressions and thereby ignoring them + the previous implementation of batch removals read everything to remove into + memory first before carrying out the first remove operation. The new version + will only read in about 1000 documents each time and then remove these. Queries + such as -* disallow empty LDAP password + FOR doc IN collection FILTER ... REMOVE doc IN collection -* The single database or single coordinator statistics in a cluster - environment within the Web UI sometimes got called way too often. - This caused artifacts in the graphs, which is now fixed. + will benefit from this change in terms of memory usage. -* An aardvark statistics route could not collect and sum up the statistics of - all coordinators if one of them was ahead and had more results than the others +* make `--help-all` now also show all hidden program options -* Web UI now checks if server statistics are enabled before it sends its first - request to the statistics API + Previously hidden program options were only returned when invoking arangod or + a client tool with the cryptic `--help-.` option. Now `--help-all` simply + retuns them as well. -* disable in-memory cache for edge and traversal data on agency nodes, as it - is not needed there + The program options JSON description returned by `--dump-options` was also + improved as follows: -* removed bundled Valgrind headers, removed JavaScript variable `valgrind` - from the `internal` module + - the new boolean attribute "dynamic" indicates whether the option has a dynamic + default value, i.e. a value that depends on the target host capabilities or + configuration -* upgraded JEMalloc version to 5.1.0 + - the new boolean attribute "requiresValue" indicates whether a boolean option + requires a value of "true" or "false" when specified. If "requiresValue" is + false, then the option can be specified without a boolean value following it, + and the option will still be set to true, e.g. `--server.authentication` is + identical to `--server.authentication true`. -* use `-std=c++14` for ArangoDB compilation + - the new "category" attribute will contain a value of "command" for command-like + options, such as `--version`, `--dump-options`, `--dump-dependencies` etc., + and "option" for all others. -* Add license key checking to enterprise version in Docker containers. +v3.4.0 (2018-12-06) +------------------- +* Add license key checking to enterprise version in Docker containers. -v3.4.0-rc.5 (XXXX-XX-XX) +v3.4.0-rc.5 (2018-11-29) ------------------------ * Persist and check default language (locale) selection. @@ -151,7 +112,7 @@ v3.4.0-rc.5 (XXXX-XX-XX) * fixed issue #7522: FILTER logic totally broke for my query in 3.4-rc4 * export version and storage engine in `_admin/cluster/health` for Coordinators - and DBServers. + and DBServers. * restrict the total amount of data to build up in all in-memory RocksDB write buffers by default to a certain fraction of the available physical RAM. This helps restricting @@ -206,6 +167,7 @@ v3.4.0-rc.5 (XXXX-XX-XX) * agency's supervision edited for advertised endpoints + v3.4.0-rc.4 (2018-11-04) ------------------------ @@ -232,7 +194,10 @@ v3.4.0-rc.4 (2018-11-04) * increase maximum number of collections/shards in an AQL query from 256 to 2048 -* do not rely on `_modules` collection being present at arangod startup +* don't rely on `_modules` collection being present and usable for arangod startup + +* force connection timeout to be 7 seconds to allow libcurl time to retry lost DNS + queries. * fixes a routing issue within the web ui after the use of views @@ -245,7 +210,7 @@ v3.4.0-rc.4 (2018-11-04) * added option `--rocksdb.total-write-buffer-size` to limit total memory usage across all RocksDB in-memory write buffers -* suppress warnings from statistics background threads such as +* suppress warnings from statistics background threads such as `WARNING caught exception during statistics processing: Expecting Object` during version upgrade @@ -271,16 +236,16 @@ v3.4.0-rc.3 (2018-10-23) Using the `force` flag when upgrading or replacing a service falls back to installing the service if it does not already exist. -* The order of JSON object attribute keys in JSON return values will now be +* The order of JSON object attribute keys in JSON return values will now be "random" in more cases. In JSON, there is no defined order for object attribute keys anyway, so ArangoDB is taking the freedom to return the attribute keys in a non-deterministic, seemingly unordered way. * Fixed an AQL bug where the `optimize-traversals` rule was falsely applied to extensions with inline expressions and thereby ignoring them - + * fix side-effects of sorting larger arrays (>= 16 members) of constant literal - values in AQL, when the array was not used only for IN-value filtering but also + values in AQL, when the array was used not only for IN-value filtering but also later in the query. The array values were sorted so the IN-value lookup could use a binary search instead of a linear search, but this did not take into account that the array @@ -288,6 +253,21 @@ v3.4.0-rc.3 (2018-10-23) will create a copy of the array and sort the copy, leaving the original array untouched. +* disallow empty LDAP password + +* fixes validation of allowed or not allowed foxx service mount paths within + the Web UI + +* The single database or single coordinator statistics in a cluster + environment within the Web UI sometimes got called way too often. + This caused artifacts in the graphs, which is now fixed. + +* An aardvark statistics route could not collect and sum up the statistics of + all coordinators if one of them was ahead and had more results than the others + +* Web UI now checks if server statistics are enabled before it sends its first + request to the statistics API + * fix internal issue #486: immediate deletion (right after creation) of a view with a link to one collection and indexed data reports failure but removes the link @@ -516,18 +496,17 @@ v3.4.0-rc.2 (2018-09-30) * fixed a bug in cleanOutServer which could lead to a cleaned out server still being a follower for some shard - v3.4.0-rc.1 (2018-09-06) ------------------------ * Release Candidate for 3.4.0, please check the `ReleaseNotes/KnownIssues34.md` - file for a list of known issues + file for a list of known issues. * upgraded bundled RocksDB version to 5.16.0 * upgraded bundled Snappy compression library to 1.1.7 -* fixed issue #5941 if using breadth first search in traversals uniqueness checks +* fixed issue #5941: if using breadth first search in traversals uniqueness checks on path (vertices and edges) have not been applied. In SmartGraphs the checks have been executed properly. @@ -1048,9 +1027,178 @@ v3.4.0-rc.1 (2018-09-06) * only load Plan and Current from agency when actually needed -v3.3.18 (XXXX-XX-XX) +v3.3.21 (XXXX-XX-XX) -------------------- +* fixed TypeError being thrown instead of validation errors when Foxx manifest + validation fails + +* fixed issue #7586: a running query within the user interface was not shown + if the active view was `Running Queries` or `Slow Query History`. + +* improve Windows installer error messages, fix Windows installer backup routine + and exit code handling + +* make AQL REMOVE operations use less memory with the RocksDB storage engine + + the previous implementation of batch removals read everything to remove into + memory first before carrying out the first remove operation. The new version + will only read in about 1000 documents each time and then remove these. Queries + such as + + FOR doc IN collection FILTER ... REMOVE doc IN collection + + will benefit from this change in terms of memory usage. + + +v3.3.20 (2018-11-28) +-------------------- + +* upgraded arangodb starter version to 0.13.9 + +* Added RocksDB option `--rocksdb.total-write-buffer-size` to limit total memory + usage across all RocksDB in-memory write buffers + + The total amount of data to build up in all in-memory buffers (backed by log + files). This option, together with the block cache size configuration option, + can be used to limit memory usage. If set to 0, the memory usage is not limited. + This is the default setting in 3.3. The default setting may be adjusted in + future versions of ArangoDB. + + If set to a value greater than 0, this will cap the memory usage for write buffers, + but may have an effect on write performance. + +* Added RocksDB configuration option `--rocksdb.enforce-block-cache-size-limit` + + Whether or not the maximum size of the RocksDB block cache is strictly enforced. + This option can be set to limit the memory usage of the block cache to at most the + specified size. If then inserting a data block into the cache would exceed the + cache's capacity, the data block will not be inserted. If the flag is not set, + a data block may still get inserted into the cache. It is evicted later, but the + cache may temporarily grow beyond its capacity limit. + +* Export version and storage engine in cluster health + +* Potential fix for issue #7407: arangorestore very slow converting from + mmfiles to rocksdb + +* Updated joi library (Web UI), improved foxx mount path validation + +* fix internal issue #2786: improved confirmation dialog when clicking the + Truncate button in the Web UI + +* fix for supervision, which started failing servers using old transient store + +* fixed Foxx queues not retrying jobs with infinite `maxFailures` + +* Fixed a race condition in a coordinator, it could happen in rare cases and + only with the maintainer mode enabled if the creation of a collection is in + progress and at the same time a deletion is forced. + +* disable startup warning for Linux kernel variable `vm.overcommit_memory` settings + values of 0 or 1. + Effectively `overcommit_memory` settings value of 0 or 1 fix two memory-allocation + related issues with the default memory allocator used in ArangoDB release builds on + 64bit Linux. + The issues will remain when running with an `overcommit_memory` settings value of 2, + so this is now discouraged. + Setting `overcommit_memory` to 0 or 1 (0 is the Linux kernel's default) fixes issues + with increasing numbers of memory mappings for the arangod process (which may lead + to an out-of-memory situation if the kernel's maximum number of mappings threshold + is hit) and an increasing amount of memory that the kernel counts as "committed". + With an `overcommit_memory` setting of 0 or 1, an arangod process may either be + killed by the kernel's OOM killer or will die with a segfault when accessing memory + it has allocated before but the kernel could not provide later on. This is still + more acceptable than the kernel not providing any more memory to the process when + there is still physical memory left, which may have occurred with an `overcommit_memory` + setting of 2 after the arangod process had done lots of allocations. + + In summary, the recommendation for the `overcommit_memory` setting is now to set it + to 0 or 1 (0 is kernel default) and not use 2. + +* force connection timeout to be 7 seconds to allow libcurl time to retry lost DNS + queries. + +* increase maximum number of collections/shards in an AQL query from 256 to 2048 + +* don't rely on `_modules` collection being present and usable for arangod startup + +* optimizes the web ui's routing which could possibly led to unwanted events. + +* fixes some graph data parsing issues in the ui, e.g. cleaning up duplicate + edges inside the graph viewer. + +* in a cluster environment, the arangod process now exits if wrong credentials + are used during the startup process. + +* Fixed an AQL bug where the optimize-traversals rule was falsely applied to + extensions with inline expressions and thereby ignoring them + +* fix side-effects of sorting larger arrays (>= 16 members) of constant literal + values in AQL, when the array was not used only for IN-value filtering but also + later in the query. + The array values were sorted so the IN-value lookup could use a binary search + instead of a linear search, but this did not take into account that the array + could have been used elsewhere in the query, e.g. as a return value. The fix + will create a copy of the array and sort the copy, leaving the original array + untouched. + +* fixed a bug when cluster indexes were usable for queries, while + still being built on db servers + +* fix move leader shard: wait until all but the old leader are in sync. + This fixes some unstable tests. + +* cluster health features more elaborate agent records + + +v3.3.19 (2018-10-20) +-------------------- + +* fixes validation of allowed or not allowed foxx service mount paths within + the Web UI + +* The single database or single coordinator statistics in a cluster + environment within the Web UI sometimes got called way too often. + This caused artifacts in the graphs, which is now fixed. + +* An aardvark statistics route could not collect and sum up the statistics of + all coordinators if one of them was ahead and had more results than the others + +* upgraded arangodb starter version to 0.13.6 + +* turn on intermediate commits in replication applier in order to decrease + the size of transactional operations on replication (issue #6821) + +* fixed issue #6770: document update: ignoreRevs parameter ignored + +* when returning memory to the OS, use the same memory protection flags as + when initializing the memory + + this prevents "hole punching" and keeps the OS from splitting one memory + mapping into multiple mappings with different memory protection settings + +* fix internal issue #2770: the Query Profiling modal dialog in the Web UI + was slightly malformed. + +* fix internal issue #2035: the Web UI now updates its indices view to check + whether new indices exist or not. + +* fix internal issue #6808: newly created databases within the Web UI did not + appear when used Internet Explorer 11 as a browser. + +* fix internal issue #2688: the Web UI's graph viewer created malformed node + labels if a node was expanded multiple times. + +* fix internal issue #2957: the Web UI was not able to display more than 1000 + documents, even when it was set to a higher amount. + +* fix internal issue #2785: web ui's sort dialog sometimes got rendered, even + if it should not. + +* fix internal issue #2764: the waitForSync property of a satellite collection + could not be changed via the Web UI + * improved logging in case of replication errors * recover short server id from agency after a restart of a cluster node @@ -1069,6 +1217,8 @@ v3.3.18 (XXXX-XX-XX) * fixed internal issue #1983: the Web UI was showing a deletion confirmation multiple times. +* fixed agents busy looping gossip + * handle missing `_frontend` collections gracefully the `_frontend` system collection is not required for normal ArangoDB operations, @@ -1076,6 +1226,12 @@ v3.3.18 (XXXX-XX-XX) on. +v3.3.18 +------- + +* not released + + v3.3.17 (2018-10-04) -------------------- diff --git a/lib/Basics/threads-win32.cpp b/lib/Basics/threads-win32.cpp index 1ecd6b3426e7..6624003e4b7d 100644 --- a/lib/Basics/threads-win32.cpp +++ b/lib/Basics/threads-win32.cpp @@ -101,6 +101,7 @@ bool TRI_StartThread(TRI_thread_t* thread, TRI_tid_t* threadId, //////////////////////////////////////////////////////////////////////////////// int TRI_JoinThread(TRI_thread_t* thread) { + TRI_ASSERT(thread != nullptr); DWORD result = WaitForSingleObject(*thread, INFINITE); switch (result) { @@ -136,7 +137,8 @@ int TRI_JoinThread(TRI_thread_t* thread) { int TRI_DetachThread(TRI_thread_t* thread) { // If the function succeeds, the return value is nonzero. // If the function fails, the return value is zero. To get extended error information, call GetLastError. - BOOL res = CloseHandle(thread); + TRI_ASSERT(thread != nullptr); + BOOL res = CloseHandle(*thread); if (res == 0) { DWORD result = GetLastError(); @@ -157,7 +159,8 @@ bool TRI_IsSelfThread(TRI_thread_t* thread) { // TODO: Change the TRI_thread_t into a structure which stores the thread id // as well as the thread handle. This can then be passed around // ........................................................................... - return (GetCurrentThreadId() == GetThreadId(thread)); + TRI_ASSERT(thread != nullptr); + return (GetCurrentThreadId() == GetThreadId(*thread)); } ////////////////////////////////////////////////////////////////////////////////