-
Notifications
You must be signed in to change notification settings - Fork 306
Fix Jackson nodes introspection for request/response schema extraction #8980
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clickin 8000 g “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Fix Jackson nodes introspection for request/response schema extraction #8980
Conversation
01f08ff
to
f2f2b01
Compare
1ee1d7a
to
119f166
Compare
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 42 metrics, 11 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.51.0-SNAPSHOT~5c3dbf86af, baseline=1.51.0-SNAPSHOT~aa9dd27535
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.032 s) : 0, 1032422
Total [baseline] (8.566 s) : 0, 8565834
Agent [candidate] (1.027 s) : 0, 1027041
Total [candidate] (8.566 s) : 0, 8566190
section iast
Agent [baseline] (1.157 s) : 0, 1157382
Total [baseline] (9.223 s) : 0, 9223095
Agent [candidate] (1.17 s) : 0, 1169568
Total [candidate] (9.262 s) : 0, 9262322
gantt
title insecure-bank - break down per module: candidate=1.51.0-SNAPSHOT~5c3dbf86af, baseline=1.51.0-SNAPSHOT~aa9dd27535
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (691.299 ms) : 0, 691299
BytebuddyAgent [candidate] (687.169 ms) : 0, 687169
GlobalTracer [baseline] (243.478 ms) : 0, 243478
GlobalTracer [candidate] (242.586 ms) : 0, 242586
AppSec [baseline] (59.723 ms) : 0, 59723
AppSec [candidate] (59.208 ms) : 0, 59208
Debugger [baseline] (7.338 ms) : 0, 7338
Debugger [candidate] (7.645 ms) : 0, 7645
Remote Config [baseline] (666.227 µs) : 0, 666
Remote Config [candidate] (650.748 µs) : 0, 651
Telemetry [baseline] (8.941 ms) : 0, 8941
Telemetry [candidate] (8.864 ms) : 0, 8864
section iast
BytebuddyAgent [baseline] (810.373 ms) : 0, 810373
BytebuddyAgent [candidate] (818.638 ms) : 0, 818638
GlobalTracer [baseline] (232.053 ms) : 0, 232053
GlobalTracer [candidate] (234.659 ms) : 0, 234659
AppSec [baseline] (55.079 ms) : 0, 55079
AppSec [candidate] (54.701 ms) : 0, 54701
Debugger [baseline] (5.891 ms) : 0, 5891
Debugger [candidate] (5.974 ms) : 0, 5974
Remote Config [baseline] (601.245 µs) : 0, 601
Remote Config [candidate] (620.286 µs) : 0, 620
Telemetry [baseline] (7.925 ms) : 0, 7925
Telemetry [candidate] (8.11 ms) : 0, 8110
IAST [baseline] (24.516 ms) : 0, 24516
IAST [candidate] (25.777 ms) : 0, 25777
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.51.0-SNAPSHOT~5c3dbf86af, baseline=1.51.0-SNAPSHOT~aa9dd27535
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.027 s) : 0, 1027388
Total [baseline] (10.687 s) : 0, 10686519
Agent [candidate] (1.026 s) : 0, 1025871
Total [candidate] (10.627 s) : 0, 10627267
section appsec
Agent [baseline] (1.181 s) : 0, 1180910
Total [baseline] (10.731 s) : 0, 10731033
Agent [candidate] (1.181 s) : 0, 1181445
Total [candidate] (10.733 s) : 0, 10732774
section iast
Agent [baseline] (1.162 s) : 0, 1162032
Total [baseline] (10.94 s) : 0, 10940459
Agent [candidate] (1.155 s) : 0, 1155444
Total [candidate] (10.863 s) : 0, 10862763
section profiling
Agent [baseline] (1.269 s) : 0, 1268939
Total [baseline] (11.005 s) : 0, 11004525
Agent [candidate] (1.268 s) : 0, 1267529
Total [candidate] (10.912 s) : 0, 10911889
gantt
title petclinic - break down per module: candidate=1.51.0-SNAPSHOT~5c3dbf86af, baseline=1.51.0-SNAPSHOT~aa9dd27535
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (687.558 ms) : 0, 687558
BytebuddyAgent [candidate] (686.29 ms) : 0, 686290
GlobalTracer [baseline] (243.021 ms) : 0, 243021
GlobalTracer [candidate] (242.744 ms) : 0, 242744
AppSec [baseline] (58.708 ms) : 0, 58708
AppSec [candidate] (60.12 ms) : 0, 60120
Debugger [baseline] (6.199 ms) : 0, 6199
Debugger [candidate] (6.95 ms) : 0, 6950
Remote Config [baseline] (667.003 µs) : 0, 667
Remote Config [candidate] (662.433 µs) : 0, 662
Telemetry [baseline] (10.313 ms) : 0, 10313
Telemetry [candidate] (8.174 ms) : 0, 8174
section appsec
BytebuddyAgent [baseline] (709.703 ms) : 0, 709703
BytebuddyAgent [candidate] (709.798 ms) : 0, 709798
GlobalTracer [baseline] (235.929 ms) : 0, 235929
GlobalTracer [candidate] (235.757 ms) : 0, 235757
AppSec [baseline] (178.809 ms) : 0, 178809
AppSec [candidate] (179.374 ms) : 0, 179374
Debugger [baseline] (5.792 ms) : 0, 5792
Debugger [candidate] (5.792 ms) : 0, 5792
Remote Config [baseline] (609.961 µs) : 0, 610
Remote Config [candidate] (613.336 µs) : 0, 613
Telemetry [baseline] (7.253 ms) : 0, 7253
Telemetry [candidate] (7.316 ms) : 0, 7316
IAST [baseline] (21.976 ms) : 0, 21976
IAST [candidate] (21.944 ms) : 0, 21944
section iast
BytebuddyAgent [baseline] (813.055 ms) : 0, 813055
BytebuddyAgent [candidate] (808.572 ms) : 0, 808572
GlobalTracer [baseline] (233.488 ms) : 0, 233488
GlobalTracer [candidate] (232.298 ms) : 0, 232298
AppSec [baseline] (53.497 ms) : 0, 53497
AppSec [candidate] (52.288 ms) : 0, 52288
Debugger [baseline] (5.992 ms) : 0, 5992
Debugger [candidate] (5.948 ms) : 0, 5948
Remote Config [baseline] (620.023 µs) : 0, 620
Remote Config [candidate] (595.616 µs) : 0, 596
Telemetry [baseline] (8.112 ms) : 0, 8112
Telemetry [candidate] (8.01 ms) : 0, 8010
IAST [baseline] (26.38 ms) : 0, 26380
IAST [candidate] (26.838 ms) : 0, 26838
section profiling
BytebuddyAgent [baseline] (675.994 ms) : 0, 675994
BytebuddyAgent [candidate] (676.916 ms) : 0, 676916
GlobalTracer [baseline] (361.169 ms) : 0, 361169
GlobalTracer [candidate] (360.302 ms) : 0, 360302
AppSec [baseline] (61.815 ms) : 0, 61815
AppSec [candidate] (61.971 ms) : 0, 61971
Debugger [baseline] (6.119 ms) : 0, 6119
Debugger [candidate] (6.105 ms) : 0, 6105
Remote Config [baseline] (693.602 µs) : 0, 694
Remote Config [candidate] (696.687 µs) : 0, 697
Telemetry [baseline] (8.065 ms) : 0, 8065
Telemetry [candidate] (8.095 ms) : 0, 8095
ProfilingAgent [baseline] (106.671 ms) : 0, 106671
ProfilingAgent [candidate] (104.848 ms) : 0, 104848
Profiling [baseline] (106.696 ms) : 0, 106696
Profiling [candidate] (104.872 ms) : 0, 104872
LoadParameters
See matching parameters
SummaryFound 2 performance improvements and 3 performance regressions! Performance is the same for 7 metrics, 12 unstable metrics.
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~5c3dbf86af, baseline=1.51.0-SNAPSHOT~83532cfd23
dateFormat X
axisFormat %s
section baseline
no_agent (4.327 ms) : 4279, 4375
. : milestone, 4327,
iast (9.181 ms) : 9032, 9330
. : milestone, 9181,
iast_FULL (14.145 ms) : 13869, 14421
. : milestone, 14145,
iast_GLOBAL (10.26 ms) : 10079, 10441
. : milestone, 10260,
profiling (8.559 ms) : 8422, 8696
. : milestone, 8559,
tracing (7.506 ms) : 7393, 7619
. : milestone, 7506,
section candidate
no_agent (4.362 ms) : 4311, 4413
. : milestone, 4362,
iast (9.362 ms) : 9205, 9518
. : milestone, 9362,
iast_FULL (13.76 ms) : 13482, 14038
. : milestone, 13760,
iast_GLOBAL (9.734 ms) : 9557, 9911
. : milestone, 9734,
profiling (8.964 ms) : 8824, 9104
. : milestone, 8964,
tracing (7.868 ms) : 7749, 7987
. : milestone, 7868,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~5c3dbf86af, baseline=1.51.0-SNAPSHOT~83532cfd23
dateFormat X
axisFormat %s
section baseline
no_agent (36.757 ms) : 36457, 37056
. : milestone, 36757,
appsec (46.234 ms) : 45812, 46655
. : milestone, 46234,
code_origins (45.362 ms) : 44988, 45736
. : milestone, 45362,
iast (44.639 ms) : 44268, 45011
. : milestone, 44639,
profiling (47.179 ms) : 46758, 47600
. : milestone, 47179,
tracing (43.687 ms) : 43332, 44042
. : milestone, 43687,
section candidate
no_agent (35.354 ms) : 35075, 35634
. : milestone, 35354,
appsec (47.531 ms) : 47105, 47956
. : milestone, 47531,
code_origins (45.642 ms) : 45256, 46028
. : milestone, 45642,
iast (46.186 ms) : 45786, 46586
. : milestone, 46186,
profiling (48.117 ms) : 47661, 48574
. : milestone, 48117,
tracing (43.691 ms) : 43330, 44051
. : milestone, 43691,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~5c3dbf86af, baseline=1.51.0-SNAPSHOT~aa9dd27535
dateFormat X
axisFormat %s
section baseline
no_agent (1.477 ms) : 1466, 1489
. : milestone, 1477,
appsec (2.402 ms) : 2353, 2451
. : milestone, 2402,
iast (2.176 ms) : 2114, 2237
. : milestone, 2176,
iast_GLOBAL (2.232 ms) : 2170, 2293
. : milestone, 2232,
profiling (2.02 ms) : 1972, 2069
. : milestone, 2020,
tracing (1.999 ms) : 1951, 2046
. : milestone, 1999,
section candidate
no_agent (1.473 ms) : 1462, 1484
. : milestone, 1473,
appsec (2.394 ms) : 2345, 2442
. : milestone, 2394,
iast (2.188 ms) : 2127, 2250
. : milestone, 2188,
iast_GLOBAL (2.231 ms) : 2170, 2293
. : milestone, 2231,
profiling (2.486 ms) : 2315, 2658
. : milestone, 2486,
tracing (2.005 ms) : 1957, 2052
. : milestone, 2005,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~5c3dbf86af, baseline=1.51.0-SNAPSHOT~aa9dd27535
dateFormat X
axisFormat %s
section baseline
no_agent (15.105 s) : 15105000, 15105000
. : milestone, 15105000,
appsec (14.841 s) : 14841000, 14841000
. : milestone, 14841000,
iast (18.464 s) : 18464000, 18464000
. : milestone, 18464000,
iast_GLOBAL (17.871 s) : 17871000, 17871000
. : milestone, 17871000,
profiling (15.426 s) : 15426000, 15426000
. : milestone, 15426000,
tracing (14.861 s) : 14861000, 14861000
. : milestone, 14861000,
section candidate
no_agent (15.461 s) : 15461000, 15461000
. : milestone, 15461000,
appsec (14.883 s) : 14883000, 14883000
. : milestone, 14883000,
iast (19.094 s) : 19094000, 19094000
. : milestone, 19094000,
iast_GLOBAL (18.148 s) : 18148000, 18148000
. : milestone, 18148000,
profiling (15.279 s) : 15279000, 15279000
. : milestone, 15279000,
tracing (15.035 s) : 15035000, 15035000
. : milestone, 15035000,
|
119f166
to
adf78fc
Compare
...appsec/src/test/groovy/com/datadog/appsec/event/data/ObjectIntrospectionSpecification.groovy
Show resolved
Hide resolved
...appsec/src/test/groovy/com/datadog/appsec/event/data/ObjectIntrospectionSpecification.groovy
Show resolved
Hide resolved
...appsec/src/test/groovy/com/datadog/appsec/event/data/ObjectIntrospectionSpecification.groovy
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
I've suggested some small changes in the tests to check that the Object result is properly truncated.
e427ebf
to
fc9e88c
Compare
578eb6f
to
5c3dbf8
Compare
@@ -178,6 +181,19 @@ private static Object doConversion(Object obj, int depth, State state) { | |||
return obj.toString(); | |||
} | |||
|
|||
// Jackson databind nodes (via reflection) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be nice to have a comment to remind us that this could happens with other data structures that need to be sent to the WAF
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
What Does This Do
Improved Jackson
JsonNode
conversion inObjectIntrospection
by using reflection and creating a 1:1 copy of the internal json.Motivation
Jackson
JsonNode
classes implementIterable
causing issues in the conversion and producing a different representation of the original json.Additional Notes
Contributor Checklist
type:
and (comp:
orinst:
) labels in addition to any usefull labelsclose
,fix
or any linking keywords when referencing an issue.Use
solves
instead, and assign the PR milestone to the issueJira ticket: APPSEC-57966