PrometheusReporter doesn't sanitize names before pushing to prometheus #96
Description
I am trying to add the Prometheus reporter in the Cadence Java SDK v3.0. When I do so I run into exceptions because invalid names are being passed to prometheus.
"java.lang.IllegalArgumentException: Invalid metric name: cadence-PollForDecisionTask.cadence-latency\n\t
at io.prometheus.client.Collector.checkMetricName(Collector.java:218)\n\t
at io.prometheus.client.SimpleCollector.<init>(SimpleCollector.java:169)\n\t
at io.prometheus.client.Summary.<init>(Summary.java:88)\n\t
at io.prometheus.client.Summary$Builder.create(Summary.java:136)\n\t
at io.prometheus.client.Summary$Builder.create(Summary.java:95)\n\t
at io.prometheus.client.SimpleCollector$Builder.register(SimpleCollector.java:258)\n\t
at com.uber.m3.tally.prometheus.PrometheusReporter.lambda$reportTimerSummary$5(PrometheusReporter.java:254)\n\t
at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1705)\n\t
at com.uber.m3.tally.prometheus.PrometheusReporter.reportTimerSummary(PrometheusReporter.java:246)\n\t
at com.uber.m3.tally.prometheus.PrometheusReporter.reportTimer(PrometheusReporter.java:169)\n\t
at com.uber.m3.tally.TimerImpl.record(TimerImpl.java:55)\n\t
at com.uber.m3.tally.TimerImpl.recordStopwatch(TimerImpl.java:69)\n\t
at com.uber.m3.tally.Stopwatch.stop(Stopwatch.java:48)\n\t
at com.uber.cadence.serviceclient.WorkflowServiceTChannel.measureRemoteCall(WorkflowServiceTChannel.java:356)\n\t
at com.uber.cadence.serviceclient.WorkflowServiceTChannel.PollForDecisionTask(WorkflowServiceTChannel.java:680)\n\t
at com.uber.cadence.internal.worker.WorkflowPollTask.poll(WorkflowPollTask.java:77)\n\t
at com.uber.cadence.internal.worker.WorkflowPollTask.poll(WorkflowPollTask.java:37)\n\t
at com.uber.cadence.internal.worker.Poller$PollExecutionTask.run(Poller.java:269)\n\t
at com.uber.cadence.internal.worker.Poller$PollLoopTask.run(Poller.java:240)\n\t
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\t
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\t
at java.base/java.lang.Thread.run(Thread.java:834)\n"
The instantiation is fairly simple kotlin code:
val scope = RootScopeBuilder().reporter(
PrometheusReporter.builder().apply {
this.registry(collectorRegistry)
this.defaultBuckets(PromProcessor.DEFAULT_TIMER_HISTOGRAM_BUCKETS)
}.build()
).reportEvery(Duration.ofSeconds(1.0))
val tcOptions = ClientOptions.newBuilder().apply {
this.setHost(config[CadenceConfigSpec.serverHostname])
this.setPort(config[CadenceConfigSpec.serverPort])
this.setMetricsScope(scope)
}.build()