8000 Add load balancing support to cursor API in 3.3 (#5797) · mnemosdev/arangodb@1897fd0 · GitHub
[go: up one dir, main page]
Skip to content

Commit 1897fd0

Browse files
Dan Larkin-Yorkjsteemann
Dan Larkin-York
authored and
jsteemann
committed
Add load balancing support to cursor API in 3.3 (arangodb#5797)
1 parent 9a639e0 commit 1897fd0

27 files changed

+1354
-244
lines changed

‎CHANGELOG

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,13 @@
11
v3.3.13 (XXXX-XX-XX)
22
--------------------
33

4+
* Added load balancer support and user-restriction to cursor API.
5+
6+
If a cursor is accessed on a different coordinator than where it was created,
7+
the requests will be forwarded to the correct coordinator. If a cursor is
8+
accessed by a different user than the one who created it, the request will
9+
be denied.
10+
411
* keep failed follower in followers list in Plan.
512

613
This increases the changes of a failed follower getting back into sync if the
@@ -21,7 +28,7 @@ v3.3.13 (XXXX-XX-XX)
2128
* fixed issue #5827: Batch request handling incompatible with .NET's default
2229
ContentType format
2330

24-
* fixed agency's log compaction for internal issue #2249
31+
* fixed agency's log compaction for internal issue #2249
2532

2633
* inspector collects additionally disk data size and storage engine statistics
2734

‎Documentation/Books/HTTP/General/README.md

Lines changed: 69 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -11,20 +11,20 @@ can optionally be SSL-encrypted.
1111
ArangoDB uses the standard HTTP methods (e.g. *GET*, *POST*, *PUT*, *DELETE*) plus
1212
the *PATCH* method described in [RFC 5789](http://tools.ietf.org/html/rfc5789).
1313

14-
Most server APIs expect clients to send any payload data in [JSON](http://www.json.org)
14+
Most server APIs expect clients to send any payload data in [JSON](http://www.json.org)
1515
format. Details on the expected format and JSON attributes can be found in the
1616
documentation of the individual server methods.
1717

1818
Clients sending requests to ArangoDB must use either HTTP 1.0 or HTTP 1.1.
19-
Other HTTP versions are not supported by ArangoDB and any attempt to send
19+
Other HTTP versions are not supported by ArangoDB and any attempt to send
2020
a different HTTP version signature will result in the server responding with
2121
an HTTP 505 (HTTP version not supported) error.
2222

2323
ArangoDB will always respond to client requests with HTTP 1.1. Clients
24-
should therefore support HTTP version 1.1.
24+
should therefore support HTTP version 1.1.
2525

26-
Clients are required to include the *Content-Length* HTTP header with the
27-
correct content length in every request that can have a body (e.g. *POST*,
26+
Clients are required to include the *Content-Length* HTTP header with the
27+
correct content length in every request that can have a body (e.g. *POST*,
2828
*PUT* or *PATCH*) request. ArangoDB will not process requests without a
2929
*Content-Length* header - thus chunked transfer encoding for POST-documents
3030
is not supported.
@@ -33,10 +33,10 @@ HTTP Keep-Alive
3333
---------------
3434

3535
ArangoDB supports HTTP keep-alive. If the client does not send a *Connection*
36-
header in its request, and the client uses HTTP version 1.1, ArangoDB will assume
37-
the client wants to keep alive the connection.
36+
header in its request, and the client uses HTTP version 1.1, ArangoDB will assume
37+
the client wants to keep alive the connection.
3838
If clients do not wish to use the keep-alive feature, they should
39-
explicitly indicate that by sending a *Connection: Close* HTTP header in
39+
explicitly indicate that by sending a *Connection: Close* HTTP header in
4040
the request.
4141

4242
ArangoDB will close connections automatically for clients that send requests
@@ -58,24 +58,24 @@ Blocking vs. Non-blocking HTTP Requests
5858

5959
ArangoDB supports both blocking and non-blocking HTTP requests.
6060

61-
ArangoDB is a multi-threaded server, allowing the processing of multiple
62-
client requests at the same time. Request/response handling and the actual
61+
ArangoDB is a multi-threaded server, allowing the processing of multiple
62+
client requests at the same time. Request/response handling and the actual
6363
work are performed on the server in parallel by multiple worker threads.
6464

6565
Still, clients need to wait for their requests to be processed by the server,
6666
and thus keep one connection of a pool occupied.
67-
By default, the server will fully process an incoming request and then return
68-
the result to the client when the operation is finished. The client must
69-
wait for the server's HTTP response before it can send additional requests over
70-
the same connection. For clients that are single-threaded and/or are
71-
blocking on I/O themselves, waiting idle for the server response may be
67+
By default, the server will fully process an incoming request and then return
68+
the result to the client when the operation is finished. The client must
69+
wait for the server's HTTP response before it can send additional requests over
70+
the same connection. For clients that are single-threaded and/or are
71+
blocking on I/O themselves, waiting idle for the server response may be
7272
non-optimal.
7373

7474
To reduce blocking on the client side, ArangoDB offers a generic mechanism for
7575
non-blocking, asynchronous execution: clients can add the
76-
HTTP header *x-arango-async: true* to any of their requests, marking
77-
them as to be executed asynchronously on the server. ArangoDB will put such
78-
requests into an in-memory task queue and return an *HTTP 202* (accepted)
76+
HTTP header *x-arango-async: true* to any of their requests, marking
77+
them as to be executed asynchronously on the server. ArangoDB will put such
78+
requests into an in-memory task queue and return an *HTTP 202* (accepted)
7979
response to the client instantly and thus finish this HTTP-request.
8080
The server will execute the tasks from the queue asynchronously as fast
8181
as possible, while clients can continue to do other work.
@@ -84,23 +84,23 @@ option ["--scheduler.maximal-queue-size"](../../Manual/Administration/Configurat
8484
then the request will be rejected instantly with an *HTTP 500* (internal
8585
server error) response.
8686

87-
Asynchronous execution decouples the request/response handling from the actual
88-
work to be performed, allowing fast server responses and greatly reducing wait
89-
time for clients. Overall this allows for much higher throughput than if
87+
Asynchronous execution decouples the request/response handling from the actual
88+
work to be performed, allowing fast server responses and greatly reducing wait
89+
time for clients. Overall this allows for much higher throughput than if
9090
clients would always wait for the server's response.
9191

92-
Keep in mind that the asynchronous execution is just "fire and forget".
93-
Clients will get any of their asynchronous requests answered with a generic
94-
HTTP 202 response. At the time the server sends this response, it does not
95-
know whether the requested operation can be carried out successfully (the
96-
actual operation execution will happen at some later point). Clients therefore
97-
cannot make a decision based on the server response and must rely on their
92+
Keep in mind that the asynchronous execution is just "fire and forget".
93+
Clients will get any of their asynchronous requests answered with a generic
94+
HTTP 202 response. At the time the server sends this response, it does not
95+
know whether the requested operation can be carried out successfully (the
96+
actual operation execution will happen at some later point). Clients therefore
97+
cannot make a decision based on the server response and must rely on their
9898
requests being valid and processable by the server.
9999

100-
Additionally, the server's asynchronous task queue is an in-memory data
101-
structure, meaning not-yet processed tasks from the queue might be lost in
102-
case of a crash. Clients should therefore not use the asynchronous feature
103-
when they have strict durability requirements or if they rely on the immediate
100+
Additionally, the server's asynchronous task queue is an in-memory data
101+
structure, meaning not-yet processed tasks from the queue might be lost in
102+
case of a crash. Clients should therefore not use the asynchronous feature
103+
when they have strict durability requirements or if they rely on the immediate
104104
result of the request they send.
105105

106106
For details on the subsequent processing
@@ -124,17 +124,17 @@ The response to an HTTP OPTIONS request will be generic and not expose any priva
124124

125125
There is an additional option to control authentication for custom Foxx apps. The option
126126
[--server.authentication-system-only](../../Manual/Administration/Configuration/GeneralArangod.html)
127-
controls whether authentication is required only for requests to the internal database APIs and the admin interface.
127+
controls whether authentication is required only for requests to the internal database APIs and the admin interface.
128128
It is turned on by default, meaning that other APIs (this includes custom Foxx apps) do not require authentication.
129129

130130
The default values allow exposing a public custom Foxx API built with ArangoDB to the outside
131131
world without the need for HTTP authentication, but still protecting the usage of the
132132
internal database APIs (i.e. */_api/*, */_admin/*) with HTTP authentication.
133133

134-
If the server is started with the *--server.authentication-system-only* option set
135-
to *false*, all incoming requests will need HTTP authentication if the server is configured
136-
to require HTTP authentication (i.e. *--server.authentication true*).
137-
Setting the option to *true* will make the server require authentication only for requests to the
134+
If the server is started with the *--server.authentication-system-only* option set
135+
to *false*, all incoming requests will need HTTP authentication if the server is configured
136+
to require HTTP authentication (i.e. *--server.authentication true*).
137+
Setting the option to *true* will make the server require authentication only for requests to the
138138
internal database APIs and will allow unauthenticated requests to all other URLs.
139139

140140
Here's a short summary:
@@ -146,12 +146,12 @@ Here's a short summary:
146146
authentication for all requests (including custom Foxx apps).
147147
* `--server.authentication false`: authentication disabled for all requests
148148

149-
Whenever authentication is required and the client has not yet authenticated,
149+
Whenever authentication is required and the client has not yet authenticated,
150150
ArangoDB will return *HTTP 401* (Unauthorized). It will also send the *WWW-Authenticate*
151-
response header, indicating that the client should prompt the user for username and
151+
response header, indicating that the client should prompt the user for username and
152152
password if supported. If the client is a browser, then sending back this header will
153153
normally trigger the display of the browser-side HTTP authentication dialog.
154-
As showing the browser HTTP authentication dialog is undesired in AJAX requests,
154+
As showing the browser HTTP authentication dialog is undesired in AJAX requests,
155155
ArangoDB can be told to not send the *WWW-Authenticate* header back to the client.
156156
Whenever a client sends the *X-Omit-WWW-Authenticate* HTTP header (with an arbitrary value)
157157
to ArangoDB, ArangoDB will only send status code 401, but no *WWW-Authenticate* header.
@@ -194,7 +194,7 @@ Error Handling
194194
The following should be noted about how ArangoDB handles client errors in its
195195
HTTP layer:
196196

197-
* client requests using an HTTP version signature different than *HTTP/1.0* or
197+
* client requests using an HTTP version signature different than *HTTP/1.0* or
198198
*HTTP/1.1* will get an *HTTP 505* (HTTP version not supported) error in return.
199199
* ArangoDB will reject client requests with a negative value in the
200200
*Content-Length* request header with *HTTP 411* (Length Required).
@@ -356,11 +356,35 @@ PUT, DELETE, PATCH) of a request using one of the following custom HTTP headers:
356356
* *x-http-method*
357357
* *x-method-override*
358358

359-
This allows using HTTP clients that do not support all "common" HTTP methods such as
360-
PUT, PATCH and DELETE. It also allows bypassing proxies and tools that would otherwise
361-
just let cer 10000 tain types of requests (e.g. GET and POST) pass through.
359+
This allows using HTTP clients that do not support all "common" HTTP methods such as
360+
PUT, PATCH and DELETE. It also allows bypassing proxies and tools that would otherwise
361+
just let certain types of requests (e.g. GET and POST) pass through.
362362

363-
Enabling this option may impose a security risk, so it should only be used in very
364-
controlled environments. Thus the default value for this option is *false* (no method
363+
Enabling this option may impose a security risk, so it should only be used in very
364+
controlled environments. Thus the default value for this option is *false* (no method
365365
overriding allowed). You need to enable it explicitly if you want to use this
366366
feature.
367+
368+
Load-balancer support
369+
---------------------
370+
371+
When running in cluster mode, ArangoDB exposes some APIs which store request
372+
state data on specific coordinator nodes, and thus subsequent requests which
373+
require access to this state must be served by the coordinator node which owns
374+
this state data. In order to support function behind a load-balancer, ArangoDB
375+
can transparently forward requests within the cluster to the correct node. If a
376+
request is forwarded, the response will contain the following custom HTTP header
377+
whose value will be the ID of the node which actually answered the request:
378+
379+
* *x-arango-request-served-by*
380+
381+
The following APIs may use request forwarding:
382+
383+
* `/_api/cursor`
384+
385+
Note: since forwarding such requests require an additional cluster-internal HTTP
386+
request, they should be avoided when possible for best performance. Typically
387+
this is accomplished either by directing the requests to the correct coordinator
388+
at a client-level or by enabling request "stickiness" on a load balancer. Since
389+
these approaches are not always possible in a given environment, we support the
390+
request forwarding as a fall-back solution.

‎arangod/Cluster/ClusterComm.cpp

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -636,7 +636,7 @@ ClusterCommResult const ClusterComm::wait(
636636
ClusterCommTimeout endTime = TRI_microtime() + timeout;
637637

638638
TRI_ASSERT(timeout >= 0.0);
639-
639+
640640
// if we cannot find the sought operation, we will return the status
641641
// DROPPED. if we get into the timeout while waiting, we will still return
642642
// CL_COMM_TIMEOUT.
@@ -1214,8 +1214,9 @@ std::pair<ClusterCommResult*, HttpRequest*> ClusterComm::prepareRequest(std::str
12141214
}
12151215

12161216
void ClusterComm::addAuthorization(std::unordered_map<std::string, std::string>* headers) {
1217-
if (_authenticationEnabled) {
1218-
headers->emplace("Authorization", _jwtAuthorization);
1217+
if (_authenticationEnabled &&
1218+
headers->find(StaticStrings::Authorization) == headers->end()) {
1219+
headers->emplace(StaticStrings::Authorization, _jwtAuthorization);
12191220
}
12201221
}
12211222

0 commit comments

Comments
 (0)
0