AI/ML platform for Jaeger #1639
Description
Summary
At the moment doing ML/AI analysis with Jaeger is hard. There is no direct integration with ML/AI platforms and we do not have much knowledge on what models we could build.
- Create Community/SIG for doing ML/AI with tracing/telemetry data.
- Build ML/AI integration with Jaeger to make it easy for data scientists write and evaluate models (e.g Jupyter notebooks).
- Create a registry of models/post-processing pipelines which derive useful information out of tracing data.
Proposal
Placeholder issue for any discussion related to ML/AI integration with Jaeger. On the recent Jaeger bi-weekly meetings we have talked about doing ML/AI on tracing data (and also with combination with other telemetry data like metrics and logs).
For the completion, I will list existing ML/post-processing integrations:
- https://github.com/jaegertracing/spark-dependencies - apache spark batch post-processing to calculate dependencies for dependency tab
- https://github.com/jaegertracing/jaeger-analytics - apache flink stream processing jobs: dependencies, trace-quality
- @tiffon experiments with trace analysis APIs https://gist.github.com/tiffon/c0998457099bd433c2e5d84a4ebee18d
cc) @annanay25