feat: autoload filebeat related kibana dashboards and objects #54
Conversation
feat: autoload filebeat related kibana dashboards and objects
| Although Pulumi supports many programming languages, Python was chosen as | ||
| the language for this project. The reimplementation of the deployments | ||
| definitions here should be reproducible in other languages. | ||
| This project illustrates the end to end stand up of an AWS VPC cluster, Elastic Kubernetes Service (EKS), NGINX |
There was a problem hiding this comment.
What's changed in the readme? Did we redo the line width?
There was a problem hiding this comment.
I pulled out the piece that explained how to manually load the dashboards/queries/etc
Then I did the reformat code, which did the line width since that is apparently how Pycharm thinks md should be formatted.
| "enabled": True, | ||
| "filebeatConfig": { | ||
| "filebeat.yml": "filebeat.autodiscover:\n providers:\n - type: kubernetes\n hints.enabled: true\n hints.default_config:\n type: container\n paths:\n - /var/lib/docker/containers/${data.kubernetes.container.id}/*.log\noutput.elasticsearch:\n host: '${NODE_NAME}'\n hosts: 'elastic-coordinating-only.logstore.svc.cluster.local:9200'\n" | ||
| "filebeat.yml": "setup.kibana.host: 'http://elastic-kibana.logstore.svc.cluster.local:5601'\nsetup.dashboards.enabled: true\nfilebeat.autodiscover:\n providers:\n - type: kubernetes\n hints.enabled: true\n hints.default_config:\n type: container\n paths:\n - /var/lib/docker/containers/${data.kubernetes.container.id}/*.log\noutput.elasticsearch:\n host: '${NODE_NAME}'\n hosts: 'elastic-coordinating-only.logstore.svc.cluster.local:9200'\n" |
There was a problem hiding this comment.
Does it make sense to hard code the port here to 9200?
There was a problem hiding this comment.
Yeah, I actually want to hardcode all the kibana/elasticsearch information in here (at least for now). I don't see a lot of value in allowing people to change from 5601/9200 in terms of ports, and since this is designed to work with our log store deployment I want to keep the service names in this file. My theory is that 99.9% of people should never touch this, and the 0.1% that need to change it will either know how / why to change it (or will learn pretty quick).
Proposed changes
We have been manually loading dashboards, index templates, and other objects into kibana following standup. This process automates the loading of these dashboards by loading them from the filebeat deployment.
README has been updated with the new details, and the code has been reformatted.
Checklist
Before creating a PR, run through this checklist and mark each as complete.