8000 Stop duplicating otel data to Grafana Cloud (#178) · opensafely-core/sysadmin@23e6328 · GitHub
[go: up one dir, main page]
Skip to content

Commit 23e6328

Browse files
madwortmadwort
authored
Stop duplicating otel data to Grafana Cloud (#178)
* Remove Grafana otel tests * Revert "feat: duplicate otel data to Grafana" This reverts commit caee385. * fix old typo
1 parent 86916ed commit 23e6328

File tree

7 files changed

+9
-92
lines changed

7 files changed

+9
-92
lines changed

‎services/otel-gateway/Dockerfile

-3
Original file line numberDiff line numberDiff line change
@@ -13,15 +13,12 @@ LABEL org.opencontainers.image.authors="tech@opensafely.org" \
1313

1414
# default config, can be overridden at runtime
1515
ENV HONEYCOMB_ENDPOINT="https://api.honeycomb.io"
16-
ENV GRAFANA_ENDPOINT="https://otlp-gateway-prod-gb-south-0.grafana.net/otlp"
1716
ENV METRICS_DATASET="jobrunner-metrics"
1817
ENV LOG_LEVEL="info"
1918

2019
# these must be provided at runtime
2120
# ENV HONEYCOMB_KEY
2221
# ENV BASIC_AUTH_USER
2322
# ENV BASIC_AUTH_PASSWORD
24-
# ENV GRAFANA_INSTANCE_ID
25-
# ENV GRAFANA_API_TOKEN
2623

2724
COPY config.yaml /etc/otelcol-contrib/config.yaml

‎services/otel-gateway/README.md

+3-5
Original file line numberDiff line numberDiff line change
@@ -24,16 +24,14 @@ traffic and basic auth.
2424

2525
### Integration Tests
2626

27-
You will need valid API keys for the development environment
27+
You will need a valid API key for the development environment
2828

2929
```
3030
export HONEYCOMB_KEY=...
31-
export GRAFANA_INSTANCE_ID=...
32-
export GRAFANA_API_TOKEN=...
3331
just test-integration
3432
```
3533

36-
This will print a link where you can go and view the data in the Honeycomb UI. It should also send data to Grafana.
34+
This will print a link where you can go and view the data in the Honeycomb UI.
3735

3836

3937
## Deploy to dokku
@@ -64,7 +62,7 @@ dokku proxy:ports-add otel-gateway https:443:4318
6462
dokku letsencrypt:enable otel-gateway
6563

6664
# secrets
67-
dokku config:set otel-gateway HONEYCOMB_KEY=... BASIC_AUTH_USER=... BASIC_AUTH_PASSWORD=... GRAFANA_INSTANCE_ID=... GRAFANA_API_TOKEN=...
65+
dokku config:set otel-gateway HONEYCOMB_KEY=... BASIC_AUTH_USER=... BASIC_AUTH_PASSWORD=...
6866

6967
```
7068

‎services/otel-gateway/config.yaml

+3-12
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,6 @@ extensions:
55
htpasswd:
66
inline: |
77
${BASIC_AUTH_USER}:${BASIC_AUTH_PASSWORD}
8-
basicauth/otlp:
9-
client_auth:
10-
username: ${GRAFANA_INSTANCE_ID}
11-
password: ${GRAFANA_API_TOKEN}
128
139
receivers:
1410
otlp:
@@ -38,22 +34,17 @@ exporters:
3834
x-honeycomb-team: "${HONEYCOMB_KEY}"
3935
x-honeycomb-dataset: "${METRICS_DATASET}"
4036

41-
otlphttp/grafana:
42-
auth:
43-
authenticator: basicauth/otlp
44-
endpoint: "${GRAFANA_ENDPOINT}"
45-
4637
service:
4738
telemetry:
4839
logs:
4940
level: "${LOG_LEVEL}"
50-
extensions: [basicauth/server, basicauth/otlp]
41+
extensions: [basicauth/server]
5142
pipelines:
5243
traces:
5344
receivers: [otlp]
5445
processors: [batch]
55-
exporters: [otlphttp/traces, otlphttp/grafana, logging]
46+
exporters: [otlphttp/traces, logging]
5647
metrics:
5748
receivers: [otlp]
5849
processors: [batch]
59-
exporters: [otlphttp/metrics, otlphttp/grafana, logging]
50+
exporters: [otlphttp/metrics, logging]

‎services/otel-gateway/dotenv-sample

-2
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,3 @@
11
BASIC_AUTH_USER="test"
22
BASIC_AUTH_PASSWORD="test"
33
HONEYCOMB_KEY="testkey"
4-
GRAFANA_INSTANCE_ID="012345"
5-
GRAFANA_API_TOKEN="glc_aoeuhtnsqjkxbmwv1234567890="

‎services/otel-gateway/justfile

+3-29
Original file line numberDiff line numberDiff line change
@@ -19,19 +19,17 @@ run *args: build
1919
-e BASIC_AUTH_USER=$BASIC_AUTH_USER \
2020
-e BASIC_AUTH_PASSWORD=$BASIC_AUTH_PASSWORD \
2121
-e HONEYCOMB_KEY \
22-
-e GRAFANA_INSTANCE_ID \
23-
-e GRAFANA_API_TOKEN \
2422
{{ args }} {{ IMAGE_NAME }}
2523

2624

27-
# run integration test. You will need keys for Honeycomb & Grafana set in the environment
25+
# run integration test. You will need a HONEYCOMB_KEY set in the environment
2826
test-integration: venv
2927
#!/bin/bash
3028
set -euo pipefail
3129

3230
{{ just_executable() }} run -d -e LOG_LEVEL=debug -p 4318:4318
3331
{{ just_executable() }} run-python tests.py
34-
echo "Data sent to honeycomb and grafana"
32+
echo "Data sent to honeycomb"
3533
echo "https://ui.honeycomb.io/bennett-institute-for-applied-data-science/environments/development/datasets/otel-gateway-tests?query=%7B%22time_range%22%3A600%2C%22granularity%22%3A0%2C%22breakdowns%22%3A%5B%5D%2C%22calculations%22%3A%5B%5D%2C%22orders%22%3A%5B%5D%2C%22havings%22%3A%5B%5D%2C%22limit%22%3A100%7D"
3634

3735

@@ -53,40 +51,16 @@ _mock_honeycomb_start:
5351
_mock_honeycomb_stop:
5452
docker stop mock-honeycomb
5553

56-
_mock_grafana_start:
57-
#!/bin/bash
58-
set -euo pipefail
59-
60-
mkdir -p exported/grafana
61-
62-
# run a different instance of a collector as a test endpoint
63-
docker rm --force mock-grafana 2>/dev/null || true
64-
docker run -d -p 4320:4318 --name mock-grafana -u "$(id -u):$(id -g)" \
65-
-v $PWD/mock-grafana-config.yaml:/etc/otelcol-contrib/config.yaml \
66-
-v $PWD/exported/grafana:/exported \
67-
-e GRAFANA_INSTANCE_ID \
68-
-e GRAFANA_API_TOKEN \
69-
otel/opentelemetry-collector-contrib:0.62.1
70-
71-
test "$(docker inspect mock-grafana -f '{{{{.State.Status}}')" == "running" || { docker logs mock-grafana; exit 1; }
72-
73-
_mock_grafana_stop:
74-
docker stop mock-grafana
75-
7654
# run tests against mock upstream servers
77-
test-ci: venv _mock_honeycomb_start _mock_grafana_start && _mock_honeycomb_stop _mock_grafana_stop
55+
test-ci: venv _mock_honeycomb_start && _mock_honeycomb_stop
7856
#!/bin/bash
7957
set -euo pipefail
8058

8159
export HONEYCOMB_ENDPOINT="http://host.docker.internal:4319"
82-
export GRAFANA_ENDPOINT="http://host.docker.internal:4320"
8360

8461
# run otel-gateway pointing at the test endpoint
8562
{{ just_executable() }} run -d \
8663
-e HONEYCOMB_ENDPOINT \
87-
-e GRAFANA_ENDPOINT \
88-
-e GRAFANA_INSTANCE_ID \
89-
-e GRAFANA_API_TOKEN \
9064
-e LOG_LEVEL=debug \
9165
-p 4318:4318 \
9266
--add-host=host.docker.internal:host-gateway

‎services/otel-gateway/mock-grafana-config.yaml

-39
This file was deleted.

‎services/otel-gateway/tests.py

-2
Original file line numberDiff line numberDiff line change
@@ -18,11 +18,9 @@
1818
import pytest
1919

2020
trace_files = [
21-
Path("exported/grafana/traces.json"),
2221
Path("exported/honeycomb/traces.json"),
2322
]
2423
metric_files = [
25-
Path("exported/grafana/metrics.json"),
2624
Path("exported/honeycomb/metrics.json"),
2725
]
2826

0 commit comments

Comments
 (0)
0