8000 Improve endpoint handling. by ObiWahn · Pull Request #11236 · arangodb/arangodb · GitHub
[go: up one dir, main page]

Skip to content

Conversation

@ObiWahn
Copy link
Contributor
@ObiWahn ObiWahn commented Mar 5, 2020
  • add tests

@ObiWahn ObiWahn requested a review from dothebart March 5, 2020 12:43
@ObiWahn ObiWahn self-assigned this Mar 5, 2020
@ObiWahn ObiWahn requested a review from Simran-B March 5, 2020 12:43
@ObiWahn
Copy link
Contributor Author
ObiWahn commented Mar 5, 2020

@Simran-B
Copy link
Contributor
Simran-B commented Mar 5, 2020

There is an inconsistency between ssl:// and https://, namely the port has to be explicitly added to the request in the latter case:

arangosh --javascript.endpoints-whitelist "ssl://www\.arangodb\.com:443\$"
require("internal").download("https://www.arangodb.com")     // allowed
require("internal").download("https://www.arangodb.com:443") // allowed

arangosh --javascript.endpoints-whitelist "https://www\.arangodb\.com:443\$"
require("internal").download("https://www.arangodb.com")     // NOT allowed!
require("internal").download("https://www.arangodb.com:443") // allowed

Not directly related, but odd: there is a pseudo-protocol(?) src:// for which we seem to omit the protocol and port internally?!

127.0.0.1:8529@_system> require("internal").download("http://asdf.foo/path")
{
  "code" : 500,
  "message" : "Could not connect to 'http+tcp://asdf.foo:80' 'getaddrinfo for host 'asdf.foo': Name does not resolve'"
}
127.0.0.1:8529@_system> require("internal").download("srv://asdf.foo/path")
{
  "code" : 500,
  "message" : "Could not connect to 'asdf.foo' ''"
}

@ObiWahn
Copy link
Contributor Author
ObiWahn commented Mar 9, 2020

There is an inconsistency between ssl:// and https://, namely the port has to be explicitly added to the request in the latter case:

arangosh --javascript.endpoints-whitelist "ssl://www\.arangodb\.com:443\$"
require("internal").download("https://www.arangodb.com")     // allowed
require("internal").download("https://www.arangodb.com:443") // allowed

arangosh --javascript.endpoints-whitelist "https://www\.arangodb\.com:443\$"
require("internal").download("https://www.arangodb.com")     // NOT allowed!
require("internal").download("https://www.arangodb.com:443") // allowed

You could make the port optional in the 2nd regular expression! The first rule apparently works with the endpoint check only and is important to be backward compatible. We should really encourage users just to use the http/https variants and deprecate other combinations like udp+vst.

Not directly related, but odd: there is a pseudo-protocol(?) src:// for which we seem to omit the protocol and port internally?!

127.0.0.1:8529@_system> require("internal").download("http://asdf.foo/path")
{
  "code" : 500,
  "message" : "Could not connect to 'http+tcp://asdf.foo:80' 'getaddrinfo for host 'asdf.foo': Name does not resolve'"
}
127.0.0.1:8529@_system> require("internal").download("srv://asdf.foo/path")
{
  "code" : 500,
  "message" : "Could not connect to 'asdf.foo' ''"
}

The failure shown here is not an permission error. Therefor it will not be fixed in this PR.

@Simran-B
Copy link
Contributor

Copied from original #11232 which was changed so something else since then:


Requests may get redirected to a different endpoint. Let the user know the endpoint in case it is disallowed by endpoint black/whitelist for easier diagnosis, e.g. --javascript.endpoints-whitelist "^ssl://arangodb\.com:443$" and the following request:

// redirects to disallowed www subdomain
require("internal").download("https://arangodb.com")

Related: arangodb/docs#92

simran@c9 ~/o/w/ArangoDB> ./build/bin/arangosh --javascript.endpoints-whitelist "ssl://arangodb\.com:443\$"

127.0.0.1:8529@_system> require("internal").download("https://arangodb.com/file.zip")
JavaScript exception: ArangoError 11: not allowed to connect to this endpoint: ssl://www.arangodb.com:443
!require("internal").download("https://arangodb.com/file.zip")
!                    ^
stacktrace: ArangoError: not allowed to connect to this endpoint: ssl://www.arangodb.com:443
    at <shell command>:1:21

Don 8000 e in 912fe72#diff-cf342f102b1a80552625f5193a113b4bR999 and eb6d153

ObiWahn added 3 commits March 25, 2020 12:52
…telisting

* origin/devel: (80 commits)
  Feature/aql subquery execution block impl execute implementation batch sub queries (#11318)
  Fix isAdminUser. (#11326)
  Bug fix/fixes 20200318 (#11319)
  updated CHANGELOG
  Feature/out of search in range (#11324)
  fix "fix" for collection figures (#11323)
  updated CHANGELOG
  compilation fixes for clang-10s more strict checking (#11316)
  fix failing query (#11317)
  KShortestPathsExecutor must reset its KShortestPathFinder, including all caches. (#11312)
  Feature/aql subquery execution block impl execute implementation expected number of rows (#11274)
  Add DTRACE points to measure request timings. (#11245)
  USE_STRICT_OPENSSL is Off by default
  Fix usesRevisionAsDocumentId population and add syncByRevision flag (#11314)
  Traversal Bugfix  (#11310)
  Bug fix/issue 11275 (#11299)
  added simple test (#11301)
  Fix some typos (#10173)
  Documentation/typos 2020-01-24 (#10975)
  Update CHANGELOG
  ...
@ObiWahn
Copy link
Contributor Author
ObiWahn commented Mar 25, 2020

@ObiWahn ObiWahn requested a review from jsteemann March 25, 2020 12:06
if (endpoint.find(':') == std::string::npos) {
endpoint.append(":80");
}
endpoint = "tcp://" + endpoint;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this be

Suggested change
endpoint = "tcp://" + endpoint;
endpoint = "ssl://" + endpoint;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure the code is just moved up and it was in the original code like that. Maybe h2c does the initial connection over http. Maybe @graetzer has an opinion on this.

Copy link
Contributor Author
@ObiWahn ObiWahn Mar 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jsteemann

simon:arango:  15:48
kp kann man wahrscheinlich ssl:// draus machen wenn das h2s vorher war
15:48
h2s ist eh nur unser propiertäres ding um http2 einzuschalten

I am not sure what to make out of this. Do you want me switch it to ssl?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, please!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the port be changed as well?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

@ObiWahn ObiWahn requested a review from graetzer March 25, 2020 14:37
@ObiWahn
Copy link
Contributor Author
ObiWahn commented Mar 25, 2020

relative = "/" + relative;
}
if (endpoint.find(':') == std::string::npos) {
endpoint.append(":80");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't we also move to port 443 here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ObiWahn
Copy link
Contributor Author
ObiWahn commented Mar 25, 2020

@jsteemann
Copy link
Contributor

@jsteemann jsteemann marked this pull request as ready for review March 25, 2020 21:13
@jsteemann jsteemann merged commit fbd3151 into devel Mar 25, 2020
@jsteemann jsteemann deleted the feature/endpoint-whitelisting branch March 25, 2020 21:14
ObiWahn added a commit that referenced this pull request Mar 27, 2020
…ql-functions

* origin/devel:
  Bug fix/schema validation return code (#11341)
  Fix explainer output when restricting collections (#11338)
  remove tabstops
  Improve endpoint handling. (#11236)
  revert HTTP return code change for user API, add tests (#11331)
  remove unused header files (#11320)
  Feature/aql subquery execution block impl execute implementation batch sub queries (#11318)
  Fix isAdminUser. (#11326)
Simran-B added a commit to arangodb/docs that referenced this pull request Oct 12, 2021
tcp:// and ssl:// are still supported but somehow buggy
67DE
. Updated docs only describe http:// and https:// that can now be used thanks to arangodb/arangodb#11236

h2://, h2s://, srv:// and unix:// are supported as well but not left uncovered in the docs.
Simran-B added a commit to arangodb/docs that referenced this pull request Oct 12, 2021
…st options (#352)

Adjusted to actual behavior of endpoint filtering, some other adjustments.

tcp:// and ssl:// are still supported but somehow buggy. Updated docs only describe http:// and https:// that can now be used thanks to arangodb/arangodb#11236

h2://, h2s://, srv:// and unix:// are supported as well but not left uncovered in the docs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

0