Databricks djm exclude logs docs (#29398)

warrierr · urseberry · web-flow · commit bdcf7eba8aa1 · 2025-05-19T19:20:09.000Z
* databricks djm exclude logs docs

Show users how to exclude all log collection from an individual cluster

* Update content/en/data_jobs/databricks.md

Co-authored-by: Ursula Chen &lt;58821586+urseberry@users.noreply.github.com&gt;

* Update databricks.md

---------

Co-authored-by: Ursula Chen &lt;58821586+urseberry@users.noreply.github.com&gt;
diff --git a/content/en/data_jobs/databricks.md b/content/en/data_jobs/databricks.md
@@ -229,6 +229,14 @@ In Datadog, view the [Data Jobs Monitoring][6] page to see a list of all your Da
 
 ## Advanced Configuration
 
+### Filter log collection on clusters
+
+#### Exclude all log collection from an individual cluster
+Configure the following environment variable in the Advanced Configuration section of your cluster in the Databricks UI or as a [Spark environment variable][18] in the Databricks API.
+```bash
+DD_LOGS_CONFIG_PROCESSING_RULES=[{\"type\": \"exclude_at_match\",\"name\": \"drop_all_logs\",\"pattern\": \".*\"}]
+```
+
 ### Tag spans at runtime
 
 {{% djm-runtime-tagging %}}
@@ -291,3 +299,4 @@ If you are using [Databricks Private Connectivity][14], reach out to the Datadog
 [15]: https://www.datadoghq.com/support/
 [16]: https://docs.databricks.com/en/security/network/front-end/ip-access-list-workspace.html
 [17]: https://docs.databricks.com/aws/en/security/auth/entitlements#entitlements-overview
+[18]: https://docs.databricks.com/api/workspace/clusters/edit#spark_env_vars
