8000 [DOCS-9847] Add note about OP data directory (#29017) · DataDog/documentation@ba68746 · GitHub
[go: up one dir, main page]
Skip to content

Commit ba68746

Browse files
maycmleealdrick.castro
maycmlee
authored and
aldrick.castro
committed
[DOCS-9847] Add note about OP data directory (#29017)
* add note * update all instances * update text * add advanced configuration links * remove go files
1 parent e0794a4 commit ba68746

File tree

13 files changed

+54
-20
lines changed

13 files changed

+54
-20
lines changed

‎layouts/shortcodes/observability_pipelines/destination_settings/amazon_security_lake.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
1. Enter your S3 bucket name.
22
1. Enter the AWS region.
3-
1. Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required:
3+
1. Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required.<br>**Note**: All file paths are made relative to the configuration data directory, which is `/var/lib/observability-pipelines-worker/config/` by default. See [Advanced Configurations][10082] for more information. The file must be owned by the `observability-pipelines-worker group` and `observability-pipelines-worker` user, or at least readable by the group or user.
44
- `Server Certificate Path`: The path to the certificate file that has been signed by your Certificate Authority (CA) Root File in DER or PEM (X.509).
55
- `CA Certificate Path`: The path to the certificate file that is your Certificate Authority (CA) Root File in DER or PEM (X.509).
66
- `Private Key Path`: The path to the `.key` private key file that belongs to your Server Certificate Path in DER or PEM (PKCS#8) format.
@@ -9,4 +9,5 @@
99
- When you add the Amazon Security Lake destination, the OCSF processor is automatically added so that you can convert your logs to Parquet before they are sent to Amazon Security Lake. See [Remap to OCSF documentation][10081] for setup instructions.
1010
- Only logs formatted by the OCSF processor are converted to Parquet.
1111

12-
[10081]: /observability_pipelines/processors/remap_ocsf
12+
[10081]: /observability_pipelines/processors/remap_ocsf
13+
[10082]: /observability_pipelines/advanced_configurations/

‎layouts/shortcodes/observability_pipelines/destination_settings/crowdstrike_ng_siem.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,11 @@ To use the CrowdStrike NG-SIEM destination, you need to set up a CrowdStrike dat
22

33
1. Select **JSON** or **Raw** encoding in the dropdown menu.
44
1. Optionally, enable compressions and select an algorithm (**gzip** or **zlib**) in the dropdown menu.
5-
1. Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required:
5+
1. Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required.<br>**Note**: All file paths are made relative to the configuration data directory, which is `/var/lib/observability-pipelines-worker/config/` by default. See [Advanced Configurations][10172] for more information. The file must be owned by the `observability-pipelines-worker group` and `observability-pipelines-worker` user, or at least readable by the group or user.
66
- `Server Certificate Path`: The path to the certificate file that has been signed by your Certificate Authority (CA) Root File in DER or PEM (X.509).
77
- `CA Certificate Path`: The path to the certificate file that is your Certificate Authority (CA) Root File in DER or PEM (X.509).
88
- `Private Key Path`: The path to the `.key` private key file that belongs to your Server Certificate Path in DER or PEM (PKCS#8) format.
99

10-
[10171]: https://falcon.us-2.crowdstrike.com/documentation/page/bdded008/hec-http-event-connector-guide
10+
[10171]: https://falcon.us-2.crowdstrike.com/documentation/page/bdded008/hec-http-event-connector-guide
11+
[10172]: /observability_pipelines/advanced_configurations/
12+
<!-- 10172 link is used in multiple shortcodes, so if it is changed, make sure to update those shortcodes using find and replace -->

‎layouts/shortcodes/observability_pipelines/processors/enrichment_table.en.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ To set up the enrichment table processor:
66
3. Enter the target attribute. The target attribute's value stores, as a JSON object, the information found in the reference table.
77
4. Select the type of reference table you want to use, **File** or **GeoIP**.
88
- For the **File** type:
9-
1. Enter the file path.
9+
1. Enter the file path.<br>**Note**: All file paths are made relative to the configuration data directory, which is `/var/lib/observability-pipelines-worker/config/` by default. See [Advanced Configurations][10172] for more information. The file must be owned by the `observability-pipelines-worker group` and `observability-pipelines-worker` user, or at least readable by the group or user.
1010
1. Enter the column name. The column name in the enrichment table is used for matching the source attribute value. See the [Enrichment file example](#enrichment-file-example).
1111
- For the **GeoIP** type, enter the GeoIP path.
1212

@@ -36,3 +36,6 @@ merchant_info {
3636
"state":"Colorado"
3737
}
3838
```
39+
40+
[10172]: /observability_pipelines/advanced_configurations/
41+
<!-- 10172 link is used in multiple shortcodes, so if it is changed, make sure to update those shortcodes using find and replace -->

‎layouts/shortcodes/observability_pipelines/source_settings/amazon_data_firehose.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,7 @@
1-
Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required:
1+
Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required.<br>**Note**: All file paths are made relative to the configuration data directory, which is `/var/lib/observability-pipelines-worker/config/` by default. See [Advanced Configurations][10172] for more information. The file must be owned by the `observability-pipelines-worker group` and `observability-pipelines-worker` user, or at least readable by the group or user.
22
- `Server Certificate Path`: The path to the certificate file that has been signed by your Certificate Authority (CA) Root File in DER or PEM (X.509).
33
- `CA Certificate Path`: The path to the certificate file that is your Certificate Authority (CA) Root File in DER or PEM (X.509).
4-
- `Private Key Path`: The path to the `.key` private key file that belongs to your Server Certificate Path in DER or PEM (PKCS#8) format.
4+
- `Private Key Path`: The path to the `.key` private key file that belongs to your Server Certificate Path in DER or PEM (PKCS#8) format.
5+
6+
[10172]: /observability_pipelines/advanced_configurations/
7+
<!-- 10172 link is used in multiple shortcodes, so if it is changed, make sure to update those shortcodes using find and replace -->

‎layouts/shortcodes/observability_pipelines/source_settings/amazon_s3.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
11
1. Enter the AWS region.
2-
1. Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required:
2+
1. Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required.<br>**Note**: All file paths are made relative to the configuration data directory, which is `/var/lib/observability-pipelines-worker/config/` by default. See [Advanced Configurations][10172] for more information. The file must be owned by the `observability-pipelines-worker group` and `observability-pipelines-worker` user, or at least readable by the group or user.
33
- `Server Certificate Path`: The path to the certificate file that has been signed by your Certificate Authority (CA) Root File in DER or PEM (X.509).
44
- `CA Certificate Path`: The path to the certificate file that is your Certificate Authority (CA) Root File in DER or PEM (X.509).
55
- `Private Key Path`: The path to the `.key` private key file that belongs to your Server Certificate Path in DER or PEM (PKCS#8) format.
6+
7+
[10172]: /observability_pipelines/advanced_configurations/
8+
<!-- 10172 link is used in multiple shortcodes, so if it is changed, make sure to update those shortcodes using find and replace -->

‎layouts/shortcodes/observability_pipelines/source_settings/fluent.en.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,7 @@
1-
Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required:
1+
Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required.<br>**Note**: All file paths are made relative to the configuration data directory, which is `/var/lib/observability-pipelines-worker/config/` by default. See [Advanced Configurations][10172] for more information. The file must be owned by the `observability-pipelines-worker group` and `observability-pipelines-worker` user, or at least readable by the group or user.
22
- `Server Certificate Path`: The path to the certificate file that has been signed by your Certificate Authority (CA) Root File in DER or PEM (X.509) format.
33
- `CA Certificate Path`: The path to the certificate file that is your Certificate Authority (CA) Root File in DER or PEM (X.509) format.
44
- `Private Key Path`: The path to the `.key` private key file that belongs to your Server Certificate Path in DER or PEM (PKCS#8) format.
5+
6+
[10172]: /observability_pipelines/advanced_configurations/
7+
<!-- 10172 link is used in multiple shortcodes, so if it is changed, make sure to update those shortcodes using find and replace -->

‎layouts/shortcodes/observability_pipelines/source_settings/google_pubsub.en.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,10 @@
22
1. Enter the path to the Google Developer Service Account Credential JSON file. See [Prerequisites](#prerequisites) for more information.
33
1. Enter the subscription name.
44
1. Select the decoder you want to use (Bytes, GELF, JSON, syslog).
5-
1. Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required:
5+
1. Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required.<br>**Note**: All file paths are made relative to the configuration data directory, which is `/var/lib/observability-pipelines-worker/config/` by default. See [Advanced Configurations][10172] for more information. The file must be owned by the `observability-pipelines-worker group` and `observability-pipelines-worker` user, or at least readable by the group or user.
66
- `Server Certificate Path`: The path to the certificate file that has been signed by your Certificate Authority (CA) Root File in DER or PEM (X.509).
77
- `CA Certificate Path`: The path to the certificate file that is your Certificate Authority (CA) Root File in DER or PEM (X.509).
8-
- `Private Key Path`: The path to the `.key` private key file that belongs to your Server Certificate Path in DER or PEM (PKCS #8) format.
8+
- `Private Key Path`: The path to the `.key` private key file that belongs to your Server Certificate Path in DER or PEM (PKCS #8) format.
9+
10+
[10172]: /observability_pipelines/advanced_configurations/
11+
<!-- 10172 link is used in multiple shortcodes, so if it is changed, make sure to update those shortcodes using find and replace -->

‎layouts/shortcodes/observability_pipelines/source_settings/http_client.en.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,14 @@ To configure your HTTP/S Client source:
22

33
1. Select your authorization strategy.
44
2. Select the decoder you want to use on the HTTP messages. Logs pulled from the HTTP source must be in this format.
5-
3. Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required:
5+
3. Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required.<br>**Note**: All file paths are made relative to the configuration data directory, which is `/var/lib/observability-pipelines-worker/config/` by default. See [Advanced Configurations][10172] for more information. The file must be owned by the `observability-pipelines-worker group` and `observability-pipelines-worker` user, or at least readable by the group or user.
66
- `Server Certificate Path`: The path to the certificate file that has been signed by your Certificate Authority (CA) Root File in DER or PEM (X.509) format.
77
- `CA Certificate Path`: The path to the certificate file that is your Certificate Authority (CA) Root File in DER or PEM (X.509) format.
88
- `Private Key Path`: The path to the `.key` private key file that belongs to your Server Certificate Path in DER or PEM (PKCS#8) format.
99
4. Enter the interval between scrapes.
1010
- Your HTTP Server must be able to handle GET requests at this interval.
1111
- Since requests run concurrently, if a scrape takes longer than the interval given, a new scrape is started, which can consume extra resources. Set the timeout to a value lower than the scrape interval to prevent this from happening.
1212
5. Enter the timeout for each scrape request.
13+
14+
[10172]: /observability_pipelines/advanced_configurations/
15+
<!-- 10172 link is used in multiple shortcodes, so if it is changed, make sure to update those shortcodes using find and replace -->

‎layouts/shortcodes/observability_pipelines/source_settings/http_server.en.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,10 @@ To configure your HTTP/S Server source, enter the following:
22

33
1. Select your authorization strategy.
44
1. Select the decoder you want to use on the HTTP messages. Your HTTP client logs must be in this format. **Note**: If you select `bytes` decoding, the raw log is stored in the `message` field.
5-
1. Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required:
5+
1. Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required.<br>**Note**: All file paths are made relative to the configuration data directory, which is `/var/lib/observability-pipelines-worker/config/` by default. See [Advanced Configurations][10172] for more information. The file must be owned by the `observability-pipelines-worker group` and `observability-pipelines-worker` user, or at least readable by the group or user.
66
- `Server Certificate Path`: The path to the certificate file that has been signed by your Certificate Authority (CA) Root File in DER or PEM (X.509).
77
- `CA Certificate Path`: The path to the certificate file that is your Certificate Authority (CA) Root File in DER or PEM (X.509).
88
- `Private Key Path`: The path to the `.key` private key file that belongs to your Server Certificate Path in DER or PEM (PKCS #8) format.
9+
10+
[10172]: /observability_pipelines/advanced_configurations/
11+
<!-- 10172 link is used in multiple shortcodes, so if it is changed, make sure to update those shortcodes using find and replace -->

‎layouts/shortcodes/observability_pipelines/source_settings/kafka.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
1. Enter the group ID.
22
1. Enter the topic name. If there is more than one, click **Add Field** to add additional topics.
33
1. Optionally, toggle the switch to enable SASL Authentication and select the mechanism (**PLAIN**, **SCHRAM-SHA-256**, or **SCHRAM-SHA-512**) in the dropdown menu.
4-
1. Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required:
4+
1. Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required.<br>**Note**: All file paths are made relative to the configuration data directory, which is `/var/lib/observability-pipelines-worker/config/` by default. See [Advanced Configurations][10132] for more information. The file must be owned by the `observability-pipelines-worker group` and `observability-pipelines-worker` user, or at least readable by the group or user.
55
- `Server Certificate Path`: The path to the certificate file that has been signed by your Certificate Authority (CA) Root File in DER or PEM (X.509).
66
- `CA Certificate Path`: The path to the certificate file that is your Certificate Authority (CA) Root File in DER or PEM (X.509).
77
- `Private Key Path`: The path to the `.key` private key file that belongs to your Server Certificate Path in DER or PEM (PKCS#8) format.
@@ -11,4 +11,5 @@
1111
1. Check your values against the [librdkafka documentation][10131] to make sure they have the correct type and are within the set range.
1212
1. Click **Add Option** to add another librdkafka option.
1313

14-
[10131]: https://github.com/confluentinc/librdkafka/blob/master/CONFIGURATION.md
14+
[10131]: https://github.com/confluentinc/librdkafka/blob/master/CONFIGURATION.md
15+
[10132]: /observability_pipelines/advanced_configurations/

‎layouts/shortcodes/observability_pipelines/source_settings/logstash.en.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,7 @@
1-
Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required:
1+
Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required.<br>**Note**: All file paths are made relative to the configuration data directory, which is `/var/lib/observability-pipelines-worker/config/` by default. See [Advanced Configurations][10172] for more information. The file must be owned by the `observability-pipelines-worker group` and `observability-pipelines-worker` user, or at least readable by the group or user.
22
- `Server Certificate Path`: The path to the certificate file that has been signed by your Certificate Authority (CA) Root File in DER or PEM (X.509).
33
- `CA Certificate Path`: The path to the certificate file that is your Certificate Authority (CA) Root File in DER or PEM (X.509).
4-
- `Private Key Path`: The path to the `.key` private key file that belongs to your Server Certificate Path in DER or PEM (PKCS #8) format.
4+
- `Private Key Path`: The path to the `.key` private key file that belongs to your Server Certificate Path in DER or PEM (PKCS #8) format.
5+
6+
[10172]: /observability_pipelines/advanced_configurations/
7+
<!-- 10172 link is used in multiple shortcodes, so if it is changed, make sure to update those shortcodes using find and replace -->

‎layouts/shortcodes/observability_pipelines/source_settings/splunk_hec.en.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,7 @@
1-
Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required:
1+
Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required.<br>**Note**: All file paths are made relative to the configuration data directory, which is `/var/lib/observability-pipelines-worker/config/` by default. See [Advanced Configurations][10172] for more information. The file must be owned by the `observability-pipelines-worker group` and `observability-pipelines-worker` user, or at least readable by the group or user.
22
- `Server Certificate Path`: The path to the certificate file that has been signed by your Certificate Authority (CA) Root File in DER or PEM (X.509).
33
- `CA Certificate Path`: The path to the certificate file that is your Certificate Authority (CA) Root File in DER or PEM (X.509).
44
- `Private Key Path`: The path to the `.key` private key file that belongs to your Server Certificate Path in DER or PEM (PKCS#8) format.
5+
6+
[10172]: /observability_pipelines/advanced_configurations/
7+
<!-- 10172 link is used in multiple shortcodes, so if it is changed, make sure to update those shortcodes using find and replace -->

‎layouts/shortcodes/observability_pipelines/source_settings/syslog.en.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,10 @@
11
To configure your Syslog source:
22

33
1. In the **Socket Type** dropdown menu, select the communication protocol you want to use: **TCP** or **UDP**.
4-
2. Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required:
4+
2. Optionally, toggle the switch to enable TLS. If you enable TLS, the following certificate and key files are required.<br>**Note**: All file paths are made relative to the configuration data directory, which is `/var/lib/observability-pipelines-worker/config/` by default. See [Advanced Configurations][10172] for more information. The file must be owned by the `observability-pipelines-worker group` and `observability-pipelines-worker` user, or at least readable by the group or user.
55
- `Server Certificate Path`: The path to the certificate file that has been signed by your Certificate Authority (CA) Root File in DER or PEM (X.509) format.
66
- `CA Certificate Path`: The path to the certificate file that is your Certificate Authority (CA) Root File in DER or PEM (X.509) format.
7-
- `Private Key Path`: The path to the `.key` private key file that belongs to your Server Certificate Path in DER or PEM (PKCS#8) format.
7+
- `Private Key Path`: The path to the `.key` private key file that belongs to your Server Certificate Path in DER or PEM (PKCS#8) format.
8+
9+
[10172]: /observability_pipelines/advanced_configurations/
10+
<!-- 10172 link is used in multiple shortcodes, so if it is changed, make sure to update those shortcodes using find and replace -->

0 commit comments

Comments
 (0)
0