Extended appsec request/response headers collection #8724

jandro996 · 2025-04-23T10:15:18Z

What Does This Do

Adds the DD_APPSEC_COLLECT_ALL_HEADERS flag, which enables collection of all request and response headers. This feature is disabled by default.
Adds the DD_APPSEC_HEADER_COLLECTION_REDACTION_ENABLED flag, which enabled header redaction. This feature is true by deafult. (The redaction is out of the scope, right now we only want to collect the headers without redaction)
To enable this feature we need DD_APPSEC_COLLECT_ALL_HEADERS = true and DD_APPSEC_HEADER_COLLECTION_REDACTION_ENABLED = false ( a future RFC should establish how to deal with redaction)
Introduces the DD_APPSEC_MAX_COLLECTED_HEADERS setting to limit the maximum number of headers collected.
Updates the writeHeaders logic to collect all headers when DD_APPSEC_COLLECT_ALL_HEADERS is enabled. Allowed headers are prioritized and must be collected if present.
If the number of headers exceeds DD_APPSEC_MAX_COLLECTED_HEADERS, the following tags are added to the span indicating the number of discarded headers:
- dd.appsec.request.header_collection.discarded
- dd.appsec.response.header_collection.discarded

Motivation

Additional Notes

RFC

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: APPSEC-57269

pr-commenter · 2025-04-23T11:31:23Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	alejandro.gonzalez/WaPo-headers
git_commit_date	1747808555	1747817164
git_commit_sha	`0f42e0a`	`1989174`
release_version	1.50.0-SNAPSHOT~0f42e0a463	1.50.0-SNAPSHOT~1989174a13

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1747819573	1747819573
ci_job_id	947383022	947383022
ci_pipeline_id	65693993	65693993
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-aszuydir-project-304-concurrent-0-746g8sv0 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-aszuydir-project-304-concurrent-0-746g8sv0 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 57 metrics, 14 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.50.0-SNAPSHOT~1989174a13, baseline=1.50.0-SNAPSHOT~0f42e0a463

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.021 s) : 0, 1021283
Total [baseline] (10.563 s) : 0, 10562812
Agent [candidate] (1.019 s) : 0, 1018591
Total [candidate] (10.542 s) : 0, 10542307
section appsec
Agent [baseline] (1.164 s) : 0, 1164055
Total [baseline] (10.701 s) : 0, 10700529
Agent [candidate] (1.164 s) : 0, 1164330
Total [candidate] (10.703 s) : 0, 10703045
section iast
Agent [baseline] (1.15 s) : 0, 1150020
Total [baseline] (10.908 s) : 0, 10908235
Agent [candidate] (1.151 s) : 0, 1150743
Total [candidate] (10.882 s) : 0, 10882471
section profiling
Agent [baseline] (1.277 s) : 0, 1276635
Total [baseline] (10.847 s) : 0, 10847246
Agent [candidate] (1.28 s) : 0, 1280174
Total [candidate] (10.831 s) : 0, 10831204

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.021 s	-
Agent	appsec	1.164 s	142.772 ms (14.0%)
Agent	iast	1.15 s	128.737 ms (12.6%)
Agent	profiling	1.277 s	255.351 ms (25.0%)
Total	tracing	10.563 s	-
Total	appsec	10.701 s	137.717 ms (1.3%)
Total	iast	10.908 s	345.423 ms (3.3%)
Total	profiling	10.847 s	284.434 ms (2.7%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.019 s	-
Agent	appsec	1.164 s	145.739 ms (14.3%)
Agent	iast	1.151 s	132.152 ms (13.0%)
Agent	profiling	1.28 s	261.584 ms (25.7%)
Total	tracing	10.542 s	-
Total	appsec	10.703 s	160.739 ms (1.5%)
Total	iast	10.882 s	340.164 ms (3.2%)
Total	profiling	10.831 s	288.898 ms (2.7%)

gantt
    title petclinic - break down per module: candidate=1.50.0-SNAPSHOT~1989174a13, baseline=1.50.0-SNAPSHOT~0f42e0a463

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (682.145 ms) : 0, 682145
BytebuddyAgent [candidate] (682.614 ms) : 0, 682614
GlobalTracer [baseline] (240.494 ms) : 0, 240494
GlobalTracer [candidate] (239.899 ms) : 0, 239899
AppSec [baseline] (55.268 ms) : 0, 55268
AppSec [candidate] (54.169 ms) : 0, 54169
Debugger [baseline] (8.3 ms) : 0, 8300
Debugger [candidate] (6.162 ms) : 0, 6162
Remote Config [baseline] (714.598 µs) : 0, 715
Remote Config [candidate] (698.767 µs) : 0, 699
Telemetry [baseline] (10.796 ms) : 0, 10796
Telemetry [candidate] (11.46 ms) : 0, 11460
section appsec
BytebuddyAgent [baseline] (703.478 ms) : 0, 703478
BytebuddyAgent [candidate] (702.704 ms) : 0, 702704
GlobalTracer [baseline] (236.65 ms) : 0, 236650
GlobalTracer [candidate] (236.654 ms) : 0, 236654
AppSec [baseline] (175.53 ms) : 0, 175530
AppSec [candidate] (176.477 ms) : 0, 176477
Debugger [baseline] (5.949 ms) : 0, 5949
Debugger [candidate] (5.964 ms) : 0, 5964
Remote Config [baseline] (627.355 µs) : 0, 627
Remote Config [candidate] (625.072 µs) : 0, 625
Telemetry [baseline] (7.401 ms) : 0, 7401
Telemetry [candidate] (7.42 ms) : 0, 7420
IAST [baseline] (21.583 ms) : 0, 21583
IAST [candidate] (21.726 ms) : 0, 21726
section iast
BytebuddyAgent [baseline] (802.714 ms) : 0, 802714
BytebuddyAgent [candidate] (803.712 ms) : 0, 803712
GlobalTracer [baseline] (230.807 ms) : 0, 230807
GlobalTracer [candidate] (230.577 ms) : 0, 230577
AppSec [baseline] (48.516 ms) : 0, 48516
AppSec [candidate] (53.623 ms) : 0, 53623
Debugger [baseline] (5.91 ms) : 0, 5910
Debugger [candidate] (5.874 ms) : 0, 5874
Remote Config [baseline] (589.857 µs) : 0, 590
Remote Config [candidate] (584.968 µs) : 0, 585
Telemetry [baseline] (7.892 ms) : 0, 7892
Telemetry [candidate] (7.887 ms) : 0, 7887
IAST [baseline] (29.334 ms) : 0, 29334
IAST [candidate] (24.98 ms) : 0, 24980
section profiling
BytebuddyAgent [baseline] (672.36 ms) : 0, 672360
BytebuddyAgent [candidate] (674.436 ms) : 0, 674436
GlobalTracer [baseline] (374.151 ms) : 0, 374151
GlobalTracer [candidate] (374.241 ms) : 0, 374241
AppSec [baseline] (61.694 ms) : 0, 61694
AppSec [candidate] (62.036 ms) : 0, 62036
Debugger [baseline] (6.223 ms) : 0, 6223
Debugger [candidate] (6.295 ms) : 0, 6295
Remote Config [baseline] (649.033 µs) : 0, 649
Remote Config [candidate] (671.533 µs) : 0, 672
Telemetry [baseline] (8.122 ms) : 0, 8122
Telemetry [candidate] (8.205 ms) : 0, 8205
ProfilingAgent [baseline] (102.487 ms) : 0, 102487
ProfilingAgent [candidate] (103.396 ms) : 0, 103396
Profiling [baseline] (102.511 ms) : 0, 102511
Profiling [candidate] (103.42 ms) : 0, 103420

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.50.0-SNAPSHOT~1989174a13, baseline=1.50.0-SNAPSHOT~0f42e0a463

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.03 s) : 0, 1030253
Total [baseline] (8.675 s) : 0, 8675163
Agent [candidate] (1.042 s) : 0, 1041855
Total [candidate] (8.735 s) : 0, 8734881
section iast
Agent [baseline] (1.158 s) : 0, 1157835
Total [baseline] (9.258 s) : 0, 9257838
Agent [candidate] (1.149 s) : 0, 1148850
Total [candidate] (9.247 s) : 0, 9246672
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.148 s) : 0, 1147822
Total [baseline] (9.216 s) : 0, 9215585
Agent [candidate] (1.153 s) : 0, 1153215
Total [candidate] (9.27 s) : 0, 9270282
section iast_TELEMETRY_OFF
Agent [baseline] (1.146 s) : 0, 1146490
Total [baseline] (9.239 s) : 0, 9239125
Agent [candidate] (1.144 s) : 0, 1144447
Total [candidate] (9.245 s) : 0, 9245190

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.03 s	-
Agent	iast	1.158 s	127.581 ms (12.4%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.148 s	117.569 ms (11.4%)
Agent	iast_TELEMETRY_OFF	1.146 s	116.237 ms (11.3%)
Total	tracing	8.675 s	-
Total	iast	9.258 s	582.675 ms (6.7%)
Total	iast_HARDCODED_SECRET_DISABLED	9.216 s	540.422 ms (6.2%)
Total	iast_TELEMETRY_OFF	9.239 s	563.962 ms (6.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.042 s	-
Agent	iast	1.149 s	106.994 ms (10.3%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.153 s	111.359 ms (10.7%)
Agent	iast_TELEMETRY_OFF	1.144 s	102.592 ms (9.8%)
Total	tracing	8.735 s	-
Total	iast	9.247 s	511.791 ms (5.9%)
Total	iast_HARDCODED_SECRET_DISABLED	9.27 s	535.401 ms (6.1%)
Total	iast_TELEMETRY_OFF	9.245 s	510.308 ms (5.8%)

gantt
    title insecure-bank - break down per module: candidate=1.50.0-SNAPSHOT~1989174a13, baseline=1.50.0-SNAPSHOT~0f42e0a463

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (689.11 ms) : 0, 689110
BytebuddyAgent [candidate] (697.372 ms) : 0, 697372
GlobalTracer [baseline] (241.958 ms) : 0, 241958
GlobalTracer [candidate] (244.725 ms) : 0, 244725
AppSec [baseline] (55.04 ms) : 0, 55040
AppSec [candidate] (56.721 ms) : 0, 56721
Debugger [baseline] (10.595 ms) : 0, 10595
Debugger [candidate] (8.455 ms) : 0, 8455
Remote Config [baseline] (720.123 µs) : 0, 720
Remote Config [candidate] (707.471 µs) : 0, 707
Telemetry [baseline] (9.144 ms) : 0, 9144
Telemetry [candidate] (9.885 ms) : 0, 9885
section iast
BytebuddyAgent [baseline] (808.491 ms) : 0, 808491
BytebuddyAgent [candidate] (802.005 ms) : 0, 802005
GlobalTracer [baseline] (231.938 ms) : 0, 231938
GlobalTracer [candidate] (230.558 ms) : 0, 230558
IAST [baseline] (29.12 ms) : 0, 29120
IAST [candidate] (26.013 ms) : 0, 26013
AppSec [baseline] (50.069 ms) : 0, 50069
AppSec [candidate] (49.36 ms) : 0, 49360
Debugger [baseline] (5.949 ms) : 0, 5949
Debugger [candidate] (5.886 ms) : 0, 5886
Remote Config [baseline] (598.922 µs) : 0, 599
Remote Config [candidate] (580.772 µs) : 0, 581
Telemetry [baseline] (7.97 ms) : 0, 7970
Telemetry [candidate] (7.842 ms) : 0, 7842
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (800.515 ms) : 0, 800515
BytebuddyAgent [candidate] (803.867 ms) : 0, 803867
GlobalTracer [baseline] (230.196 ms) : 0, 230196
GlobalTracer [candidate] (232.233 ms) : 0, 232233
IAST [baseline] (27.75 ms) : 0, 27750
IAST [candidate] (29.364 ms) : 0, 29364
AppSec [baseline] (51.359 ms) : 0, 51359
AppSec [candidate] (49.78 ms) : 0, 49780
Debugger [baseline] (5.962 ms) : 0, 5962
Debugger [candidate] (5.905 ms) : 0, 5905
Remote Config [baseline] (603.347 µs) : 0, 603
Remote Config [candidate] (583.409 µs) : 0, 583
Telemetry [baseline] (7.965 ms) : 0, 7965
Telemetry [candidate] (7.925 ms) : 0, 7925
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (800.717 ms) : 0, 800717
BytebuddyAgent [candidate] (798.121 ms) : 0, 798121
GlobalTracer [baseline] (229.643 ms) : 0, 229643
GlobalTracer [candidate] (230.312 ms) : 0, 230312
IAST [baseline] (23.219 ms) : 0, 23219
IAST [candidate] (23.083 ms) : 0, 23083
AppSec [baseline] (55.162 ms) : 0, 55162
AppSec [candidate] (55.064 ms) : 0, 55064
Debugger [baseline] (5.946 ms) : 0, 5946
Debugger [candidate] (5.929 ms) : 0, 5929
Remote Config [baseline] (602.284 µs) : 0, 602
Remote Config [candidate] (621.468 µs) : 0, 621
Telemetry [baseline] (7.707 ms) : 0, 7707
Telemetry [candidate] (7.799 ms) : 0, 7799

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-05-21T08:54:37	2025-05-21T09:02:24
git_branch	master	alejandro.gonzalez/WaPo-headers
git_commit_date	1747808555	1747817164
git_commit_sha	`0f42e0a`	`1989174`
release_version	1.50.0-SNAPSHOT~0f42e0a463	1.50.0-SNAPSHOT~1989174a13
start_time	2025-05-21T08:54:23	2025-05-21T09:02:10

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1747818546	1747818546
ci_job_id	947383023	947383023
ci_pipeline_id	65693993	65693993
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-yh26g23a-project-304-concurrent-0-whsch0ss 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-yh26g23a-project-304-concurrent-0-whsch0ss 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 18 unstable metrics.

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~1989174a13, baseline=1.50.0-SNAPSHOT~0f42e0a463
    dateFormat X
    axisFormat %s
section baseline
no_agent (393.345 µs) : 374, 413
.   : milestone, 393,
iast (532.94 µs) : 510, 556
.   : milestone, 533,
iast_FULL (746.185 µs) : 723, 770
.   : milestone, 746,
iast_GLOBAL (566.998 µs) : 545, 589
.   : milestone, 567,
iast_HARDCODED_SECRET_DISABLED (532.141 µs) : 510, 555
.   : milestone, 532,
iast_INACTIVE (466.234 µs) : 444, 489
.   : milestone, 466,
iast_TELEMETRY_OFF (510.858 µs) : 487, 535
.   : milestone, 511,
tracing (468.226 µs) : 446, 490
.   : milestone, 468,
section candidate
no_agent (392.924 µs) : 373, 413
.   : milestone, 393,
iast (537.83 µs) : 516, 560
.   : milestone, 538,
iast_FULL (737.709 µs) : 716, 760
.   : milestone, 738,
iast_GLOBAL (573.006 µs) : 551, 595
.   : milestone, 573,
iast_HARDCODED_SECRET_DISABLED (535.9 µs) : 513, 558
.   : milestone, 536,
iast_INACTIVE (469.47 µs) : 448, 491
.   : milestone, 469,
iast_TELEMETRY_OFF (512.81 µs) : 490, 536
.   : milestone, 513,
tracing (462.919 µs) : 441, 485
.   : milestone, 463,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	393.345 µs [373.659 µs, 413.03 µs]	-
iast	532.94 µs [509.501 µs, 556.38 µs]	139.596 µs (35.5%)
iast_FULL	746.185 µs [722.791 µs, 769.579 µs]	352.84 µs (89.7%)
iast_GLOBAL	566.998 µs [545.198 µs, 588.798 µs]	173.653 µs (44.1%)
iast_HARDCODED_SECRET_DISABLED	532.141 µs [509.598 µs, 554.684 µs]	138.796 µs (35.3%)
iast_INACTIVE	466.234 µs [443.822 µs, 488.647 µs]	72.889 µs (18.5%)
iast_TELEMETRY_OFF	510.858 µs [486.989 µs, 534.726 µs]	117.513 µs (29.9%)
tracing	468.226 µs [446.008 µs, 490.445 µs]	74.881 µs (19.0%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	392.924 µs [373.048 µs, 412.8 µs]	-
iast	537.83 µs [515.823 µs, 559.836 µs]	144.905 µs (36.9%)
iast_FULL	737.709 µs [715.628 µs, 759.79 µs]	344.785 µs (87.7%)
iast_GLOBAL	573.006 µs [551.023 µs, 594.988 µs]	180.082 µs (45.8%)
iast_HARDCODED_SECRET_DISABLED	535.9 µs [513.388 µs, 558.413 µs]	142.976 µs (36.4%)
iast_INACTIVE	469.47 µs [447.755 µs, 491.186 µs]	76.546 µs (19.5%)
iast_TELEMETRY_OFF	512.81 µs [489.852 µs, 535.767 µs]	119.885 µs (30.5%)
tracing	462.919 µs [440.641 µs, 485.198 µs]	69.995 µs (17.8%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~1989174a13, baseline=1.50.0-SNAPSHOT~0f42e0a463
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.366 ms) : 1346, 1386
.   : milestone, 1366,
appsec (1.746 ms) : 1721, 1770
.   : milestone, 1746,
appsec_no_iast (1.731 ms) : 1708, 1754
.   : milestone, 1731,
code_origins (1.691 ms) : 1664, 1718
.   : milestone, 1691,
iast (1.521 ms) : 1496, 1546
.   : milestone, 1521,
profiling (1.53 ms) : 1505, 1554
.   : milestone, 1530,
tracing (1.51 ms) : 1488, 1533
.   : milestone, 1510,
section candidate
no_agent (1.364 ms) : 1343, 1384
.   : milestone, 1364,
appsec (1.746 ms) : 1722, 1769
.   : milestone, 1746,
appsec_no_iast (1.746 ms) : 1723, 1769
.   : milestone, 1746,
code_origins (1.682 ms) : 1656, 1709
.   : milestone, 1682,
iast (1.533 ms) : 1509, 1557
.   : milestone, 1533,
profiling (1.522 ms) : 1499, 1546
.   : milestone, 1522,
tracing (1.498 ms) : 1473, 1523
.   : milestone, 1498,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.366 ms [1.346 ms, 1.386 ms]	-
appsec	1.746 ms [1.721 ms, 1.77 ms]	379.382 µs (27.8%)
appsec_no_iast	1.731 ms [1.708 ms, 1.754 ms]	364.646 µs (26.7%)
code_origins	1.691 ms [1.664 ms, 1.718 ms]	324.778 µs (23.8%)
iast	1.521 ms [1.496 ms, 1.546 ms]	154.765 µs (11.3%)
profiling	1.53 ms [1.505 ms, 1.554 ms]	163.563 µs (12.0%)
tracing	1.51 ms [1.488 ms, 1.533 ms]	144.229 µs (10.6%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.364 ms [1.343 ms, 1.384 ms]	-
appsec	1.746 ms [1.722 ms, 1.769 ms]	382.277 µs (28.0%)
appsec_no_iast	1.746 ms [1.723 ms, 1.769 ms]	382.505 µs (28.1%)
code_origins	1.682 ms [1.656 ms, 1.709 ms]	318.887 µs (23.4%)
iast	1.533 ms [1.509 ms, 1.557 ms]	169.383 µs (12.4%)
profiling	1.522 ms [1.499 ms, 1.546 ms]	158.827 µs (11.6%)
tracing	1.498 ms [1.473 ms, 1.523 ms]	134.82 µs (9.9%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	alejandro.gonzalez/WaPo-headers
git_commit_date	1747808555	1747817164
git_commit_sha	`0f42e0a`	`1989174`
release_version	1.50.0-SNAPSHOT~0f42e0a463	1.50.0-SNAPSHOT~1989174a13

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1747819124	1747819124
ci_job_id	947383024	947383024
ci_pipeline_id	65693993	65693993
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-aszuydir-project-304-concurrent-1-pm4qf3db 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-aszuydir-project-304-concurrent-1-pm4qf3db 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~1989174a13, baseline=1.50.0-SNAPSHOT~0f42e0a463
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.473 ms) : 1461, 1484
.   : milestone, 1473,
appsec (2.428 ms) : 2376, 2480
.   : milestone, 2428,
iast (2.214 ms) : 2149, 2280
.   : milestone, 2214,
iast_GLOBAL (2.266 ms) : 2200, 2332
.   : milestone, 2266,
profiling (2.535 ms) : 2353, 2717
.   : milestone, 2535,
tracing (2.027 ms) : 1976, 2078
.   : milestone, 2027,
section candidate
no_agent (1.472 ms) : 1461, 1484
.   : milestone, 1472,
appsec (2.438 ms) : 2386, 2490
.   : milestone, 2438,
iast (2.218 ms) : 2152, 2283
.   : milestone, 2218,
iast_GLOBAL (2.263 ms) : 2197, 2329
.   : milestone, 2263,
profiling (2.063 ms) : 2010, 2116
.   : milestone, 2063,
tracing (2.037 ms) : 1986, 2088
.   : milestone, 2037,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.473 ms [1.461 ms, 1.484 ms]	-
appsec	2.428 ms [2.376 ms, 2.48 ms]	955.528 µs (64.9%)
iast	2.214 ms [2.149 ms, 2.28 ms]	741.654 µs (50.4%)
iast_GLOBAL	2.266 ms [2.2 ms, 2.332 ms]	792.963 µs (53.8%)
profiling	2.535 ms [2.353 ms, 2.717 ms]	1.062 ms (72.1%)
tracing	2.027 ms [1.976 ms, 2.078 ms]	553.845 µs (37.6%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.472 ms [1.461 ms 8000 , 1.484 ms]	-
appsec	2.438 ms [2.386 ms, 2.49 ms]	965.787 µs (65.6%)
iast	2.218 ms [2.152 ms, 2.283 ms]	745.279 µs (50.6%)
iast_GLOBAL	2.263 ms [2.197 ms, 2.329 ms]	790.699 µs (53.7%)
profiling	2.063 ms [2.01 ms, 2.116 ms]	591.083 µs (40.1%)
tracing	2.037 ms [1.986 ms, 2.088 ms]	564.61 µs (38.4%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~1989174a13, baseline=1.50.0-SNAPSHOT~0f42e0a463
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.954 s) : 14954000, 14954000
.   : milestone, 14954000,
appsec (15.309 s) : 15309000, 15309000
.   : milestone, 15309000,
iast (19.133 s) : 19133000, 19133000
.   : milestone, 19133000,
iast_GLOBAL (18.18 s) : 18180000, 18180000
.   : milestone, 18180000,
profiling (14.91 s) : 14910000, 14910000
.   : milestone, 14910000,
tracing (15.037 s) : 15037000, 15037000
.   : milestone, 15037000,
section candidate
no_agent (15.606 s) : 15606000, 15606000
.   : milestone, 15606000,
appsec (15.156 s) : 15156000, 15156000
.   : milestone, 15156000,
iast (18.624 s) : 18624000, 18624000
.   : milestone, 18624000,
iast_GLOBAL (18.289 s) : 18289000, 18289000
.   : milestone, 18289000,
profiling (15.429 s) : 15429000, 15429000
.   : milestone, 15429000,
tracing (14.879 s) : 14879000, 14879000
.   : milestone, 14879000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.954 s [14.954 s, 14.954 s]	-
appsec	15.309 s [15.309 s, 15.309 s]	355.0 ms (2.4%)
iast	19.133 s [19.133 s, 19.133 s]	4.179 s (27.9%)
iast_GLOBAL	18.18 s [18.18 s, 18.18 s]	3.226 s (21.6%)
profiling	14.91 s [14.91 s, 14.91 s]	-44.0 ms (-0.3%)
tracing	15.037 s [15.037 s, 15.037 s]	83.0 ms (0.6%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.606 s [15.606 s, 15.606 s]	-
appsec	15.156 s [15.156 s, 15.156 s]	-450.0 ms (-2.9%)
iast	18.624 s [18.624 s, 18.624 s]	3.018 s (19.3%)
iast_GLOBAL	18.289 s [18.289 s, 18.289 s]	2.683 s (17.2%)
profiling	15.429 s [15.429 s, 15.429 s]	-177.0 ms (-1.1%)
tracing	14.879 s [14.879 s, 14.879 s]	-727.0 ms (-4.7%)

smola · 2025-05-20T13:24:20Z

dd-java-agent/appsec/src/main/java/com/datadog/appsec/gateway/GatewayBridge.java

-        writeRequestHeaders(traceSeg, REQUEST_HEADERS_ALLOW_LIST, ctx.getRequestHeaders());
-        writeResponseHeaders(traceSeg, RESPONSE_HEADERS_ALLOW_LIST, ctx.getResponseHeaders());
+        boolean collectAll =
+            Config.get().isAppSecCollectAllHeaders()


Adds the DD_APPSEC_HEADER_COLLECTION_REDACTION_ENABLED flag, which enabled header redaction. This feature is true by deafult. (The redaction is out of the scope, right now we only want to collect the headers without redaction)

So can we also clarify in the PR description that, right now, if redaction is enabled, we do not collect headers at all?

Yes! I'm going to clarify this. Until a new RFC establish how to deal with redaction on those cases we prefer to avoid collection of potential sensitive data. So long story sort to enable this feature we need DD_APPSEC_COLLECT_ALL_HEADERS = true and DD_APPSEC_HEADER_COLLECTION_REDACTION_ENABLED = false

dd-trace-api/src/main/java/datadog/trace/api/config/AppSecConfig.java

…ig.java Co-authored-by: Santiago M. Mola <santiago.mola@datadoghq.com>

| Package | Type | Package file | Manager | Update | Change | |---|---|---|---|---|---| | [com.datadoghq:dd-trace-api](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.49.0` -> `1.50.0` | --- ### Release Notes <details> <summary>datadog/dd-trace-java (com.datadoghq:dd-trace-api)</summary> ### [`v1.50.0`](https://github.com/DataDog/dd-trace-java/releases/tag/v1.50.0): 1.50.0 ### Deprecation Notice > \[!NOTE] > `DD_RUNTIME_ID_ENABLED` has been deprecated and will be removed in future releases. Please use `DD_RUNTIME_METRICS_RUNTIME_ID_ENABLED` instead. ### Components #### Application Security Management (WAF) - 🐛 Add String length truncation limit to ObjectIntrospector and update truncation metrics ([#8825](DataDog/dd-trace-java#8825) - [@jandro996](https://github.com/jandro996)) - 🐛 Adapt standalone ASM to support API Security ([#8804](DataDog/dd-trace-java#8804) - [@jandro996](https://github.com/jandro996)) - ✨ Add appsec.waf.input\_truncated metric ([#8791](DataDog/dd-trace-java#8791) - [@jandro996](https://github.com/jandro996)) - ✨ Extended appsec request body collection ([#8748](DataDog/dd-trace-java#8748) - [@jandro996](https://github.com/jandro996)) - ✨ Extended appsec request/response headers collection ([#8724](DataDog/dd-trace-java#8724) - [@jandro996](https://github.com/jandro996)) #### Build & Tooling - ✨ Add artifacts to public s3 bucket ([#8947](DataDog/dd-trace-java#8947) - [@randomanderson](https://github.com/randomanderson)) #### Continuous Integration Visibility - ✨ Improve PR information building ([#8908](DataDog/dd-trace-java#8908) - [@daniel-mohedano](https://github.com/daniel-mohedano)) - ✨ Truncate span stack traces when Test Optimization is enabled ([#8903](DataDog/dd-trace-java#8903) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🐛 Ensure auto-detected service name is the same for every process in the same build ([#8902](DataDog/dd-trace-java#8902) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🐛 Use tag as fallback in api requests if no branch is available ([#8876](DataDog/dd-trace-java#8876) - [@daniel-mohedano](https://github.com/daniel-mohedano)) - ✨ Add support for JUnit 5.13-RC1 ([#8865](DataDog/dd-trace-java#8865), [#8871](DataDog/dd-trace-java#8871) - [@daniel-mohedano](https://github.com/daniel-mohedano)) - ✨ Implement attempt to fix v3 and v4 and bump capability version ([#8824](DataDog/dd-trace-java#8824) - [@daniel-mohedano](https://github.com/daniel-mohedano)) - 🧹 Align retry logic for all test framework instrumentations ([#8803](DataDog/dd-trace-java#8803) - [@daniel-mohedano](https://github.com/daniel-mohedano)) - 🐛 Always build ci workspace without trailing separator ([#8788](DataDog/dd-trace-java#8788) - [@daniel-mohedano](https://github.com/daniel-mohedano)) - ✨ Add commit discrepancies telemetry when building repository git information ([#8763](DataDog/dd-trace-java#8763) - [@daniel-mohedano](https://github.com/daniel-mohedano)) #### Data Streams Monitoring - 💡 Surface process tags in dsm payloads and use them for base hash calculation ([#8836](DataDog/dd-trace-java#8836) - [@amarziali](https://github.com/amarziali)) #### Dynamic Instrumentation - ✨ Optimized allocations for collection filter functions ([#8896](DataDog/dd-trace-java#8896) - [@jpbempel](https://github.com/jpbempel)) - 🐛 Fix SymDB upload size check ([#8887](DataDog/dd-trace-java#8887) - [@jpbempel](https://github.com/jpbempel)) - 🐛 Add support for Set in filter function ([#8873](DataDog/dd-trace-java#8873) - [@jpbempel](https://github.com/jpbempel)) - 🐛 Add support for isDefined in log template ([#8859](DataDog/dd-trace-java#8859) - [@jpbempel](https://github.com/jpbempel)) - 🐛 Fix Max captured frames for Exception Replay ([#8856](DataDog/dd-trace-java#8856) - [@jpbempel](https://github.com/jpbempel)) - 🐛 Remove static inherited fields collection ([#8832](DataDog/dd-trace-java#8832) - [@jpbempel](https://github.com/jpbempel)) - 💡 Add process tags to dynamic instrumentation intake payload ([#8779](DataDog/dd-trace-java#8779) - [@amarziali](https://github.com/amarziali)) #### GraalVM native-image - ✨ Add support for GraalVM Native GC metrics ([#8913](DataDog/dd-trace-java#8913) - [@ygree](https://github.com/ygree)) - ✨ Add JMXFetch support for GraalVM Native ([#8569](DataDog/dd-trace-java#8569) - [@ygree](https://github.com/ygree)) #### JMX fetch - ✨ Add support for GraalVM Native GC metrics ([#8913](DataDog/dd-trace-java#8913) - [@ygree](https://github.com/ygree)) #### Library Injection - ✨ Deny oracle db jvm based tools ([#8909](DataDog/dd-trace-java#8909) - [@bric3](https://github.com/bric3)) #### OpenTracing - 🐛 Fix OT packaging for exception replay ([#8912](DataDog/dd-trace-java#8912) - [@jpbempel](https://github.com/jpbempel)) #### Profiling - ✨ Bump ddprof to 1.27.0 ([#8893](DataDog/dd-trace-java#8893) - [@jbachorik](https://github.com/jbachorik)) - Properly handle the adaptive sampling interval overflow by [@jbachorik](https://github.com/jbachorik) in DataDog/java-profiler#213 - Fix [#200](DataDog/dd-trace-java#200) Crash related to aligned\_alloc and free in context by [@yanglong1010](https://github.com/yanglong1010) in DataDog/java-profiler#208 - Explicitly initialize empty context page by [@jbachorik](https://github.com/jbachorik) in DataDog/java-profiler#210 - Re-connect crash recursion protection with VM stackwalker by [@jbachorik](https://github.com/jbachorik) in DataDog/java-profiler#214 - ✨ Enable ZSTD compression for profiling ([#8862](DataDog/dd-trace-java#8862) - [@MattAlp](https://github.com/MattAlp)) - ✨ Extend JPS re-implementation to J9 family ([#8813](DataDog/dd-trace-java#8813) - [@MattAlp](https://github.com/MattAlp)) - 💡 Collect process tags for profiling upload requests ([#8780](DataDog/dd-trace-java#8780) - [@amarziali](https://github.com/amarziali)) #### Telemetry - 💡 Surface process tags on telemetry payloads ([#8837](DataDog/dd-trace-java#8837) - [@amarziali](https://github.com/amarziali)) #### Trace context propagation - ✨ Migrating all HttpClient Instrumentations to Inject Full Context ([#8826](DataDog/dd-trace-java#8826) - [@mhlidd](https://github.com/mhlidd)) - ✨ Migrating all HttpServer Instrumentations to Extract full Context ([#8820](DataDog/dd-trace-java#8820) - [@mhlidd](https://github.com/mhlidd)) - ✨ Add context API support OTel propagators ([#8770](DataDog/dd-trace-java#8770) - [@PerfectSlayer](https://github.com/PerfectSlayer)) #### Tracer core - ✨⚡ Skip JAXB generated classes classloader ([#9003](DataDog/dd-trace-java#9003) - [@bric3](https://github.com/bric3)) - ✨ Add DD\_RUNTIME\_METRICS\_RUNTIME\_ID\_ENABLED alias for runtime id generation ([#8981](DataDog/dd-trace-java#8981) - [@amarziali](https://github.com/amarziali)) - 🐛 Use resolved address for peer.hostname when available without hitting the cache ([#8915](DataDog/dd-trace-java#8915) - [@amarziali](https://github.com/amarziali)) - 💡 Surface server name process tag for tomcat ([#8894](DataDog/dd-trace-java#8894) - [@amarziali](https://github.com/amarziali)) - 💡 Surface websphere cell and server name on process tags ([#8880](DataDog/dd-trace-java#8880) - [@amarziali](https://github.com/amarziali)) - ✨ Added special lightweight pre-main class that skips installation on incompatible JVMs. ([#8855](DataDog/dd-trace-java#8855) - [@AlexeyKuznetsov-DD](https://github.com/AlexeyKuznetsov-DD)) - 💡 Add entrypoint type to process tags ([#8839](DataDog/dd-trace-java#8839) - [@amarziali](https://github.com/amarziali)) - ✨ Extend JPS re-implementation to J9 family ([#8813](DataDog/dd-trace-java#8813) - [@MattAlp](https://github.com/MattAlp)) - ✨ Notify listeners when the scope top changes after switching scope stacks ([#8797](DataDog/dd-trace-java#8797) - [@mcculls](https://github.com/mcculls)) - ✨ Read hsperfdata for Java PIDs if jvmstat is unavailable ([#8792](DataDog/dd-trace-java#8792) - [@MattAlp](https://github.com/MattAlp)) - 🐛 Turn JDK socket support on by default ([#8752](DataDog/dd-trace-java#8752) - [@sarahchen6](https://github.com/sarahchen6)) - ✨ Simplify context propagation ([#8719](DataDog/dd-trace-java#8719) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - ✨ Add JSON parsing support ([#8579](DataDog/dd-trace-java#8579) - [@PerfectSlayer](https://github.com/PerfectSlayer)) #### Tracer internal logging - ✨ Fix printing format of span identifiers ([#8897](DataDog/dd-trace-java#8897) - [@vandonr](https://github.com/vandonr)) #### Tracer public API - 💡 Track the source of installation ([#8956](DataDog/dd-trace-java#8956) - [@mabdinur](https://github.com/mabdinur)) - ✨ Enforce size limit on application\_monitoring.yaml files ([#8789](DataDog/dd-trace-java#8789) - [@mtoffl01](https://github.com/mtoffl01)) - ✨ Enabling baggage cache to support limits and non-ascii characters ([#8713](DataDog/dd-trace-java#8713) - [@mhlidd](https://github.com/mhlidd)) ### Instrumentations #### AWS Lambda instrumentation - ✨ Pass Lambda Request ID to Extension ([#8814](DataDog/dd-trace-java#8814) - [@nhulston](https://github.com/nhulston)) #### Core Java language instrumentation - ✨ Ensure ClassloadingInstrumentation is always applied even with `DD_TRACE_ENABLED=false` ([#8863](DataDog/dd-trace-java#8863) - [@mcculls](https://github.com/mcculls)) #### Eclipse Vert.x instrumentation - 🐛 Do not override route with / in vertx instrumentation ([#8881](DataDog/dd-trace-java#8881) - [@vandonr](https://github.com/vandonr)) #### IBM Liberty - 🐛 Fix error mark on http status for IBM liberty ([#8822](DataDog/dd-trace-java#8822) - [@amarziali](https://github.com/amarziali)) #### JDBC instrumentation - 🐛 Do not prepend DBM <> APM trace comment in SQLCommenter if there is a pg plan hint ([#8864](DataDog/dd-trace-java#8864) - [@edengorevoy](https://github.com/edengorevoy)) #### JMS instrumentation - ✨ Add jms as an extra integration name where there is JMS involved ([#8933](DataDog/dd-trace-java#8933) - [@vandonr](https://github.com/vandonr)) #### Kotlin instrumentation - ✨ Enable kotlin\_coroutine integration by default ([#8848](DataDog/dd-trace-java#8848) - [@mcculls](https://github.com/mcculls)) - 🧹 Rework Kotlin coroutines instrumentation around coroutine context ([#8774](DataDog/dd-trace-java#8774) - [@mcculls](https://github.com/mcculls)) #### OpenTelemetry instrumentation - 🐛 Support WithSpan inheritContext attribute ([#8858](DataDog/dd-trace-java#8858) - [@amarziali](https://github.com/amarziali)) - ✨ Add context API support OTel propagators ([#8770](DataDog/dd-trace-java#8770) - [@PerfectSlayer](https://github.com/PerfectSlayer)) #### Play Framework instrumentation - 🐛 Fix the Play Framework's span resource name priority so that the client JAX-RS 404 cannot override it ([#8591](DataDog/dd-trace-java#8591) - [@ygree](https://github.com/ygree)) #### Quarkus Instrumentation - 🐛 Ignore quarkus jaxrs stubs and cdi wrapper proxies ([#8891](DataDog/dd-trace-java#8891) - [@amarziali](https://github.com/amarziali)) #### ServiceTalk - ✨ Improve ServiceTalk Captured Context API Instrumentation for v0.42.56+ ([#8821](DataDog/dd-trace-java#8821) - [@ygree](https://github.com/ygree)) #### Spring instrumentation - ✨ Supporting Baggage for Instrumentations used in Weblog Tests ([#8773](DataDog/dd-trace-java#8773) - [@mhlidd](https://github.com/mhlidd)) #### WebSocket Instrumentation - 💡 Trace websocket for spring webflux reactive handlers ([#8831](DataDog/dd-trace-java#8831) - [@amarziali](https://github.com/amarziali)) - 💡:test\_tube: WebSocket support for Netty ([#8632](DataDog/dd-trace-java#8632) - [@ValentinZakharov](https://github.com/ValentinZakharov)) #### Zio Instrumentation - 🧹 Cleanup Zio fiber instrumentation to avoid repeated activation of continuation ([#8798](DataDog/dd-trace-java#8798) - [@mcculls](https://github.com/mcculls)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 6pm every weekday,before 2am every weekday" in timezone Australia/Melbourne, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Never, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). GitOrigin-RevId: 9207366cdb6a1bd098082305d354a0a3c4622d7a

jandro996 added comp: asm waf Application Security Management (WAF) type: enhancement labels Apr 23, 2025

jandro996 force-pushed the alejandro.gonzalez/WaPo-headers branch from dc48fa1 to 761eade Compare April 30, 2025 11:32

jandro996 marked this pull request as ready for review May 6, 2025 11:06

jandro996 requested review from a team as code owners May 6, 2025 11:06

jandro996 requested review from smola, manuel-alvarez-alvarez, robertpi and sezen-datadog May 6, 2025 11:06

WIP

09b77c3

jandro996 force-pushed the alejandro.gonzalez/WaPo-headers branch from c117ce0 to 09b77c3 Compare May 19, 2025 07:07

smola reviewed May 20, 2025

View reviewed changes

dd-trace-api/src/main/java/datadog/trace/api/config/AppSecConfig.java 8000 Show resolved Hide resolved

dd-trace-api/src/main/java/datadog/trace/api/config/AppSecConfig.java Show resolved Hide resolved

dd-trace-api/src/main/java/datadog/trace/api/config/AppSecConfig.java Show resolved Hide resolved

jandro996 and others added 6 commits May 20, 2025 15:36

Update dd-trace-api/src/main/java/datadog/trace/api/config/AppSecConf…

97433cd

…ig.java Co-authored-by: Santiago M. Mola <santiago.mola@datadoghq.com>

Update dd-trace-api/src/main/java/datadog/trace/api/config/AppSecConf…

e6cb2dd

…ig.java Co-authored-by: Santiago M. Mola <santiago.mola@datadoghq.com>

Update dd-trace-api/src/main/java/datadog/trace/api/config/AppSecConf…

37a5094

…ig.java Co-authored-by: Santiago M. Mola <santiago.mola@datadoghq.com>

Merge branch 'master' into alejandro.gonzalez/WaPo-headers

bcac4de

fix smoke tests

4e995e9

rollback properties changes

1989174

smola approved these changes May 21, 2025

View reviewed changes

jandro996 merged commit 9f23747 into master May 21, 2025
587 of 590 checks passed

jandro996 deleted the alejandro.gonzalez/WaPo-headers branch May 21, 2025 10:18

github-actions bot added this to the 1.50.0 milestone May 21, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Extended appsec request/response headers collection #8724

Extended appsec request/response headers collection #8724

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Extended appsec request/response headers collection #8724

Extended appsec request/response headers collection #8724

Uh oh!

Conversation

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!