8000 Add optional scheme support for push gateway urls by thekuffs · Pull Request #103 · prometheus/client_python · GitHub
[go: up one dir, main page]

Skip to content

Add optional scheme support for push gateway urls #103

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Oct 1, 2016

Conversation

thekuffs
Copy link
Contributor

We are using a push gateway behind an https reverse proxy. push_gateway and related functions previously assumed http and prepended the scheme no matter the circumstances. This patch checks to see if the user has specified a scheme. And, if not, it defaults to http.

I ran the tests on both python 2.7.10 and 3.5.2 and they pass just fine. So we've got backward-compatibility. But I didn't see a very obvious way to get an https server into the test.

Thanks @brian-brazil

@@ -105,6 +106,8 @@ def write_to_textfile(path, registry):
def push_to_gateway(gateway, job, registry, grouping_key=None, timeout=None):
'''Push metrics to the given pushgateway.

`gateway` is a url, but will assume http if no other scheme is provided.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"scheme defaults to http if none is provided" would be a bit clearer.

If we're documenting one arg, they all should get documented. Exactly what bits of the URL are needed should be specified.

@thekuffs
Copy link
Contributor Author

Looks like tests failed on 2.6. My gut feeling is that urlparse probably assumes http in the absence of a scheme on 2.6. It's '' on all others.

@thekuffs
Copy link
Contributor Author

And that feeling would be incorrect: https://docs.python.org/2.6/library/urlparse.html#urlparse.urlparse

@thekuffs
Copy link
Contributor Author
thekuffs commented Sep 28, 2016

I also meant to followup with a question about the /metric/ path segment. That's configurable on the push gateway (https://github.com/prometheus/pushgateway/blob/master/main.go#L40). But the python client always assumes /metric. It doesn't affect me, but might be annoying to others in the future. It would be a bit difficult to make assumptions about the provided string given that RFC 1808 specifies that a scheme is required to interpret it as a netloc (https://github.com/prometheus/pushgateway/blob/master/main.go#L40)

@brian-brazil
Copy link
Contributor

/metrics is what Prometheus scrapes, the pushgateway only supports /metric/ for incoming metrics.

@thekuffs
Copy link
Contributor Author

Oh wait, sorry. I understand what you mean now. You're right.

@thekuffs
Copy link
Contributor Author

Welp, urlparse is actually bugged in 2.6.9

(Pdb) gateway
u'127.0.0.1:59541'
(Pdb) urlparse(gateway)
ParseResult(scheme=u'127.0.0.1', netloc='', path=u'59541', params='', query='', fragment='')
(Pdb) from urlparse import urlsplit
(Pdb) urlsplit(gateway)
SplitResult(scheme=u'127.0.0.1', netloc='', path=u'59541', query='', fragment='')

Looking at this bug https://bugs.python.org/issue754016 , it's been broken since Python 2.3 at least. Someone submitted a patch for some release in the meantime that wasn't accepted. And then it didn't actually get fixed until 2.7.

How would you like to handle this? I can do a cheap if '//' not in gateway: ... or I guess drop 2.6 support?

@brian-brazil
Copy link
Contributor

We should continue to support 2.6

@thekuffs
Copy link
Contributor Author
thekuffs commented Sep 30, 2016

I guess this is bettter

if not (gateway.startswith('http://') or gateway.startswith('https://')):

I feel like regex is probably a little too much to bring in to solve this.

I could also split the port number off and reattach it after urlparse?

@@ -106,7 +106,13 @@ def write_to_textfile(path, registry):
def push_to_gateway(gateway, job, registry, grouping_key=None, timeout=None):
'''Push metrics to the given pushgateway.

`gateway` is a url, but will assume http if no other scheme is provided.
`gateway` is a url. Scheme defaults to 'http' if none is provided
`job` is the name of the local routine pushing metrics
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Job is the job label to use

@brian-brazil
Copy link
Contributor

Why would you need to remove the port number?

@@ -105,6 +106,14 @@ def write_to_textfile(path, registry):
def push_to_gateway(gateway, job, registry, grouping_key=None, timeout=None):
'''Push metrics to the given pushgateway.

`gateway` is a url. Scheme defaults to 'http' if none is provided
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make it clear how much of the pgw url you need to provide

@thekuffs
Copy link
Contributor Author

Python 2.6.9 doesn't correctly parse urls of the scheme 127.0.0.1:23523, which is why it got tripped up in the tests. The issue is fixed in all other pythons. So I could test and split on ':' and reattach it after the call to urlparse.

@thekuffs
Copy link
Contributor Author

I elected to just do a gateway.startswith() test. It solves the problem.

8000

@brian-brazil brian-brazil merged commit 4197142 into prometheus:master Oct 1, 2016
@brian-brazil
Copy link
Contributor

Thanks!

@thekuffs thekuffs deleted the add_https_url_support branch October 3, 2016 14:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
0