Concat filter : multiline_end_regexp value is not formatted properly in Fluentd config #2071
Description
Describe the bug:
When using the Concat Filter, I think the regexp value for the field multiline_end_regexp is not properly formatted into the generated Fluentd config :
My Flow configuration:
apiVersion: v1
- apiVersion: logging.banzaicloud.io/v1beta1
kind: Flow
metadata:
name: my-flow
namespace: xxx
spec:
filters:
- concat:
key: log
multiline_end_regexp: /\n$/
separator: ""
use_first_timestamp: true
- grep:
regexp:
- key: log
pattern: /^\{.*\}/
- parser:
key_name: log
parse:
type: json
remove_key_name_field: true
reserve_data: true
localOutputRefs:
- my-output
match:
- select:
container_names:
- xxxThe resulting FluentdConfig (taken from the logs of the pod rancher-logging-root-fluentd-configcheck) has the value "/\\n$/" for the multiline_end_regexp field :
<label @2b0669f6af9ddff877bae45460fe9d59>
<filter **>
@type concat
@id flow:xxx:my-flow:0
key "log"
multiline_end_regexp "/\\n$/"
use_first_timestamp true
</filter>
<filter **>
@type grep
@id flow:xxx:my-flow:1
<regexp>
key "log"
pattern /^\{.*\}/
</regexp>
</filter>
<filter **>
@type parser
@id flow:xxx:my-flow:2
key_name "log"
remove_key_name_field true
reserve_data true
<parse>
@type "json"
</parse>
</filter>
<match **>
@type elasticsearch_data_streamExpected behaviour:
I expect to obtain the value /\n$/ in the Fluentd config :
<filter **>
@type concat
@id flow:xxx:my-flow:0
key "log"
multiline_end_regexp /\n$/
use_first_timestamp true
</filter>Steps to reproduce the bug:
- Create a flow with the above Flow configuration
- Check the generated Fluentd configuration
Additional context:
I'm dealing with JSON logs which have a size greater than 16 Kib an Docker splits them in multiples logs. I was trying to solve this problem by adding this Fluentd configuration but when porting it to my Flow resource, the regexp for the field multiline_end_regexp is not formatted properly in the resulting Fluentd config.
Environment details:
- Kubernetes version: v1.25.14
- Cloud-provider/provisioner: RKE1
- logging-operator version: 3.17.10
- Install method (e.g. helm or static manifests): Rancher Logging plugin (chart is
rancher-logging:102.0.3+up3.17.10) - Logs from the misbehaving component (and any other relevant logs): /
- Resource definition (possibly in YAML format) that caused the issue, without sensitive data: see above
/kind bug