Skip to content

Cleanup InstrumenterModule enablement #9027

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

mcculls
Copy link
Contributor

@mcculls mcculls commented Jun 24, 2025

What Does This Do

  • All instrumentations now declare a target system which they fall under
  • We introduce COMMON as the system for instrumentations shared across all systems and SECURITY for instrumentations shared between APPSEC and IAST
  • We now log when an instrumentation has been disabled
  • Special enablement of security instrumentations shared between APPSEC and IAST has been moved to isEnabled which is now told the enabled systems

Motivation

Preparation for encoding the target system of each instrumentation in the instrumentation index. This will let us do early filtering without needing to load the instrumentation.

Contributor Checklist

Jira ticket: APMAPI-1290

@mcculls mcculls added comp: core Tracer core tag: no release notes Changes to exclude from release notes type: refactoring labels Jun 24, 2025
@mcculls mcculls force-pushed the mcculls/cleanup-instrumenter-enablement branch 2 times, most recently from ef6f9ce to e311183 Compare June 24, 2025 11:49
@pr-commenter
Copy link

pr-commenter bot commented Jun 24, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/cleanup-instrumenter-enablement
git_commit_date 1750788935 1750792614
git_commit_sha faeb62c 0e4a616
release_version 1.51.0-SNAPSHOT~faeb62cfb1 1.51.0-SNAPSHOT~0e4a61656a
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1750800145 1750800145
ci_job_id 996993849 996993849
ci_pipeline_id 68656549 68656549
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-6z7wq9aq-project-304-concurrent-0-b3f7n6i0 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-6z7wq9aq-project-304-concurrent-0-b3f7n6i0 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 45 metrics, 8 unstable metrics.

Startup time reports for insecure-bank
gantt
    title insecure-bank - global startup overhead: candidate=1.51.0-SNAPSHOT~0e4a61656a, baseline=1.51.0-SNAPSHOT~faeb62cfb1

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (993.343 ms) : 0, 993343
Total [baseline] (8.546 s) : 0, 8545927
Agent [candidate] (1.002 s) : 0, 1002392
Total [candidate] (8.568 s) : 0, 8567886
section iast
Agent [baseline] (1.131 s) : 0, 1131126
Total [baseline] (9.299 s) : 0, 9298696
Agent [candidate] (1.134 s) : 0, 1133831
Total [candidate] (9.25 s) : 0, 9249757
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 993.343 ms -
Agent iast 1.131 s 137.783 ms (13.9%)
Total tracing 8.546 s -
Total iast 9.299 s 752.768 ms (8.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.002 s -
Agent iast 1.134 s 131.439 ms (13.1%)
Total tracing 8.568 s -
Total iast 9.25 s 681.871 ms (8.0%)
gantt
    title insecure-bank - break down per module: candidate=1.51.0-SNAPSHOT~0e4a61656a, baseline=1.51.0-SNAPSHOT~faeb62cfb1

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (685.882 ms) : 0, 685882
BytebuddyAgent [candidate] (691.422 ms) : 0, 691422
GlobalTracer [baseline] (241.631 ms) : 0, 241631
GlobalTracer [candidate] (244.579 ms) : 0, 244579
AppSec [baseline] (30.116 ms) : 0, 30116
AppSec [candidate] (30.402 ms) : 0, 30402
Debugger [baseline] (6.082 ms) : 0, 6082
Debugger [candidate] (6.097 ms) : 0, 6097
Remote Config [baseline] (652.509 µs) : 0, 653
Remote Config [candidate] (653.164 µs) : 0, 653
Telemetry [baseline] (8.281 ms) : 0, 8281
Telemetry [candidate] (8.303 ms) : 0, 8303
section iast
BytebuddyAgent [baseline] (807.908 ms) : 0, 807908
BytebuddyAgent [candidate] (803.444 ms) : 0, 803444
GlobalTracer [baseline] (233.002 ms) : 0, 233002
GlobalTracer [candidate] (240.078 ms) : 0, 240078
IAST [baseline] (28.344 ms) : 0, 28344
IAST [candidate] (28.727 ms) : 0, 28727
AppSec [baseline] (26.757 ms) : 0, 26757
AppSec [candidate] (26.543 ms) : 0, 26543
Debugger [baseline] (5.819 ms) : 0, 5819
Debugger [candidate] (5.844 ms) : 0, 5844
Remote Config [baseline] (581.402 µs) : 0, 581
Remote Config [candidate] (571.539 µs) : 0, 572
Telemetry [baseline] (7.937 ms) : 0, 7937
Telemetry [candidate] (7.862 ms) : 0, 7862
Loading
Startup time reports for petclinic
gantt
    title petclinic - global startup overhead: candidate=1.51.0-SNAPSHOT~0e4a61656a, baseline=1.51.0-SNAPSHOT~faeb62cfb1

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (997.236 ms) : 0, 997236
Total [baseline] (10.641 s) : 0, 10641459
Agent [candidate] (994.692 ms) : 0, 994692
Total [candidate] (10.708 s) : 0, 10707550
section appsec
Agent [baseline] (1.17 s) : 0, 1170173
Total [baseline] (10.69 s) : 0, 10690303
Agent [candidate] (1.167 s) : 0, 1167160
Total [candidate] (10.717 s) : 0, 10716807
section iast
Agent [baseline] (1.137 s) : 0, 1136991
Total [baseline] (10.838 s) : 0, 10837696
Agent [candidate] (1.128 s) : 0, 1128012
Total [candidate] (10.797 s) : 0, 10796541
section profiling
Agent [baseline] (1.259 s) : 0, 1258634
Total [baseline] (11.154 s) : 0, 11153926
Agent [candidate] (1.251 s) : 0, 1251018
Total [candidate] (11.092 s) : 0, 11091530
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 997.236 ms -
Agent appsec 1.17 s 172.936 ms (17.3%)
Agent iast 1.137 s 139.754 ms (14.0%)
Agent profiling 1.259 s 261.397 ms (26.2%)
Total tracing 10.641 s -
Total appsec 10.69 s 48.844 ms (0.5%)
Total iast 10.838 s 196.237 ms (1.8%)
Total profiling 11.154 s 512.467 ms (4.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 994.692 ms -
Agent appsec 1.167 s 172.467 ms (17.3%)
Agent iast 1.128 s 133.32 ms (13.4%)
Agent profiling 1.251 s 256.326 ms (25.8%)
Total tracing 10.708 s -
Total appsec 10.717 s 9.256 ms (0.1%)
Total iast 10.797 s 88.991 ms (0.8%)
Total profiling 11.092 s 383.98 ms (3.6%)
gantt
    title petclinic - break down per module: candidate=1.51.0-SNAPSHOT~0e4a61656a, baseline=1.51.0-SNAPSHOT~faeb62cfb1

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (688.392 ms) : 0, 688392
BytebuddyAgent [candidate] (686.488 ms) : 0, 686488
GlobalTracer [baseline] (242.932 ms) : 0, 242932
GlobalTracer [candidate] (242.429 ms) : 0, 242429
AppSec [baseline] (30.059 ms) : 0, 30059
AppSec [candidate] (30.124 ms) : 0, 30124
Debugger [baseline] (6.072 ms) : 0, 6072
Debugger [candidate] (6.053 ms) : 0, 6053
Remote Config [baseline] (654.261 µs) : 0, 654
Remote Config [candidate] (637.899 µs) : 0, 638
Telemetry [baseline] (8.181 ms) : 0, 8181
Telemetry [candidate] (8.195 ms) : 0, 8195
section appsec
BytebuddyAgent [baseline] (709.13 ms) : 0, 709130
BytebuddyAgent [candidate] (699.632 ms) : 0, 699632
GlobalTracer [baseline] (235.631 ms) : 0, 235631
GlobalTracer [candidate] (241.421 ms) : 0, 241421
AppSec [baseline] (168.125 ms) : 0, 168125
AppSec [candidate] (168.95 ms) : 0, 168950
Debugger [baseline] (5.847 ms) : 0, 5847
Debugger [candidate] (5.84 ms) : 0, 5840
Remote Config [baseline] (599.714 µs) : 0, 600
Remote Config [candidate] (609.365 µs) : 0, 609
Telemetry [baseline] (8.136 ms) : 0, 8136
Telemetry [candidate] (8.177 ms) : 0, 8177
IAST [baseline] (21.838 ms) : 0, 21838
IAST [candidate] (21.732 ms) : 0, 21732
section iast
BytebuddyAgent [baseline] (813.209 ms) : 0, 813209
BytebuddyAgent [candidate] (799.254 ms) : 0, 799254
GlobalTracer [baseline] (233.161 ms) : 0, 233161
GlobalTracer [candidate] (238.949 ms) : 0, 238949
AppSec [baseline] (27.333 ms) : 0, 27333
AppSec [candidate] (28.912 ms) : 0, 28912
Debugger [baseline] (5.821 ms) : 0, 5821
Debugger [candidate] (5.753 ms) : 0, 5753
Remote Config [baseline] (573.431 µs) : 0, 573
Remote Config [candidate] (577.817 µs) : 0, 578
Telemetry [baseline] (7.917 ms) : 0, 7917
Telemetry [candidate] (7.836 ms) : 0, 7836
IAST [baseline] (28.05 ms) : 0, 28050
IAST [candidate] (25.235 ms) : 0, 25235
section profiling
ProfilingAgent [baseline] (103.896 ms) : 0, 103896
ProfilingAgent [candidate] (103.776 ms) : 0, 103776
BytebuddyAgent [baseline] (687.749 ms) : 0, 687749
BytebuddyAgent [candidate] (682.24 ms) : 0, 682240
GlobalTracer [baseline] (365.132 ms) : 0, 365132
GlobalTracer [candidate] (363.042 ms) : 0, 363042
AppSec [baseline] (32.9 ms) : 0, 32900
AppSec [candidate] (31.381 ms) : 0, 31381
Debugger [baseline] (9.735 ms) : 0, 9735
Debugger [candidate] (7.597 ms) : 0, 7597
Remote Config [baseline] (689.817 µs) : 0, 690
Remote Config [candidate] (2.198 ms) : 0, 2198
Telemetry [baseline] (9.536 ms) : 0, 9536
Telemetry [candidate] (11.882 ms) : 0, 11882
Profiling [baseline] (103.921 ms) : 0, 103921
Profiling [candidate] (103.8 ms) : 0, 103800
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/cleanup-instrumenter-enablement
git_commit_date 1750788935 1750792614
git_commit_sha faeb62c 0e4a616
release_version 1.51.0-SNAPSHOT~faeb62cfb1 1.51.0-SNAPSHOT~0e4a61656a
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1750794287 1750794287
ci_job_id 996768619 996768619
ci_pipeline_id 68656549 68656549
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-tte58cfm-project-304-concurrent-1-6tbk4ppu 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-tte58cfm-project-304-concurrent-1-6tbk4ppu 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 12 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:insecure-bank:iast_FULL:high_load better
[-1235.568µs; -633.653µs] or [-8.485%; -4.352%]
unstable
[-10.896op/s; +54.583op/s] or [-3.404%; +17.054%]
13.627ms 341.906op/s 14.562ms 320.062op/s
scenario:load:petclinic:appsec:high_load better
[-3.376ms; -2.477ms] or [-6.964%; -5.110%]
unstable
[-1.054op/s; +13.304op/s] or [-1.091%; +13.774%]
45.545ms 102.713op/s 48.471ms 96.588op/s
Request duration reports for petclinic
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~0e4a61656a, baseline=1.51.0-SNAPSHOT~faeb62cfb1
    dateFormat X
    axisFormat %s
section baseline
no_agent (37.318 ms) : 37021, 37615
.   : milestone, 37318,
appsec (48.471 ms) : 48039, 48902
.   : milestone, 48471,
code_origins (44.234 ms) : 43872, 44596
.   : milestone, 44234,
iast (45.007 ms) : 44604, 45411
.   : milestone, 45007,
profiling (49.902 ms) : 49463, 50341
.   : milestone, 49902,
tracing (43.144 ms) : 42796, 43492
.   : milestone, 43144,
section candidate
no_agent (36.871 ms) : 36574, 37168
.   : milestone, 36871,
appsec (45.545 ms) : 45142, 45948
.   : milestone, 45545,
code_origins (44.724 ms) : 44360, 45088
.   : milestone, 44724,
iast (45.491 ms) : 45093, 45888
.   : milestone, 45491,
profiling (51.192 ms) : 50692, 51691
.   : milestone, 51192,
tracing (41.95 ms) : 41610, 42289
.   : milestone, 41950,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 37.318 ms [37.021 ms, 37.615 ms] -
appsec 48.471 ms [48.039 ms, 48.902 ms] 11.153 ms (29.9%)
code_origins 44.234 ms [43.872 ms, 44.596 ms] 6.916 ms (18.5%)
iast 45.007 ms [44.604 ms, 45.411 ms] 7.69 ms (20.6%)
profiling 49.902 ms [49.463 ms, 50.341 ms] 12.584 ms (33.7%)
tracing 43.144 ms [42.796 ms, 43.492 ms] 5.826 ms (15.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 36.871 ms [36.574 ms, 37.168 ms] -
appsec 45.545 ms [45.142 ms, 45.948 ms] 8.674 ms (23.5%)
code_origins 44.724 ms [44.36 ms, 45.088 ms] 7.853 ms (21.3%)
iast 45.491 ms [45.093 ms, 45.888 ms] 8.62 ms (23.4%)
profiling 51.192 ms [50.692 ms, 51.691 ms] 14.321 ms (38.8%)
tracing 41.95 ms [41.61 ms, 42.289 ms] 5.079 ms (13.8%)
Request duration reports for insecure-bank
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~0e4a61656a, baseline=1.51.0-SNAPSHOT~faeb62cfb1
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.312 ms) : 4263, 4361
.   : milestone, 4312,
iast (9.177 ms) : 9022, 9332
.   : milestone, 9177,
iast_FULL (14.562 ms) : 14271, 14852
.   : milestone, 14562,
iast_GLOBAL (9.936 ms) : 9766, 10106
.   : milestone, 9936,
profiling (8.67 ms) : 8522, 8818
.   : milestone, 8670,
tracing (7.567 ms) : 7457, 7677
.   : milestone, 7567,
section candidate
no_agent (4.32 ms) : 4272, 4368
.   : milestone, 4320,
iast (9.2 ms) : 9051, 9350
.   : milestone, 9200,
iast_FULL (13.627 ms) : 13359, 13895
.   : milestone, 13627,
iast_GLOBAL (10.016 ms) : 9841, 10191
.   : milestone, 10016,
profiling (8.84 ms) : 8704, 8976
.   : milestone, 8840,
tracing (7.43 ms) : 7319, 7540
.   : milestone, 7430,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.312 ms [4.263 ms, 4.361 ms] -
iast 9.177 ms [9.022 ms, 9.332 ms] 4.865 ms (112.8%)
iast_FULL 14.562 ms [14.271 ms, 14.852 ms] 10.25 ms (237.7%)
iast_GLOBAL 9.936 ms [9.766 ms, 10.106 ms] 5.624 ms (130.4%)
profiling 8.67 ms [8.522 ms, 8.818 ms] 4.358 ms (101.1%)
tracing 7.567 ms [7.457 ms, 7.677 ms] 3.255 ms (75.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.32 ms [4.272 ms, 4.368 ms] -
iast 9.2 ms [9.051 ms, 9.35 ms] 4.88 ms (113.0%)
iast_FULL 13.627 ms [13.359 ms, 13.895 ms] 9.307 ms (215.4%)
iast_GLOBAL 10.016 ms [9.841 ms, 10.191 ms] 5.696 ms (131.9%)
profiling 8.84 ms [8.704 ms, 8.976 ms] 4.52 ms (104.6%)
tracing 7.43 ms [7.319 ms, 7.54 ms] 3.11 ms (72.0%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/cleanup-instrumenter-enablement
git_commit_date 1750788935 1750792614
git_commit_sha faeb62c 0e4a616
release_version 1.51.0-SNAPSHOT~faeb62cfb1 1.51.0-SNAPSHOT~0e4a61656a
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1750794764 1750794764
ci_job_id 996768622 996768622
ci_pipeline_id 68656549 68656549
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-tte58cfm-project-304-concurrent-2-ayz81zqo 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-tte58cfm-project-304-concurrent-2-ayz81zqo 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava
gantt
    title biojava - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~0e4a61656a, baseline=1.51.0-SNAPSHOT~faeb62cfb1
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.45 s) : 15450000, 15450000
.   : milestone, 15450000,
appsec (14.839 s) : 14839000, 14839000
.   : milestone, 14839000,
iast (18.222 s) : 18222000, 18222000
.   : milestone, 18222000,
iast_GLOBAL (18.18 s) : 18180000, 18180000
.   : milestone, 18180000,
profiling (15.576 s) : 15576000, 15576000
.   : milestone, 15576000,
tracing (14.917 s) : 14917000, 14917000
.   : milestone, 14917000,
section candidate
no_agent (15.6 s) : 15600000, 15600000
.   : milestone, 15600000,
appsec (14.853 s) : 14853000, 14853000
.   : milestone, 14853000,
iast (18.49 s) : 18490000, 18490000
.   : milestone, 18490000,
iast_GLOBAL (17.797 s) : 17797000, 17797000
.   : milestone, 17797000,
profiling (15.359 s) : 15359000, 15359000
.   : milestone, 15359000,
tracing (14.842 s) : 14842000, 14842000
.   : milestone, 14842000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.45 s [15.45 s, 15.45 s] -
appsec 14.839 s [14.839 s, 14.839 s] -611.0 ms (-4.0%)
iast 18.222 s [18.222 s, 18.222 s] 2.772 s (17.9%)
iast_GLOBAL 18.18 s [18.18 s, 18.18 s] 2.73 s (17.7%)
profiling 15.576 s [15.576 s, 15.576 s] 126.0 ms (0.8%)
tracing 14.917 s [14.917 s, 14.917 s] -533.0 ms (-3.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.6 s [15.6 s, 15.6 s] -
appsec 14.853 s [14.853 s, 14.853 s] -747.0 ms (-4.8%)
iast 18.49 s [18.49 s, 18.49 s] 2.89 s (18.5%)
iast_GLOBAL 17.797 s [17.797 s, 17.797 s] 2.197 s (14.1%)
profiling 15.359 s [15.359 s, 15.359 s] -241.0 ms (-1.5%)
tracing 14.842 s [14.842 s, 14.842 s] -758.0 ms (-4.9%)
Execution time for tomcat
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.51.0-SNAPSHOT~0e4a61656a, baseline=1.51.0-SNAPSHOT~faeb62cfb1
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.469 ms) : 1458, 1481
.   : milestone, 1469,
appsec (2.401 ms) : 2351, 2450
.   : milestone, 2401,
iast (2.179 ms) : 2117, 2241
.   : milestone, 2179,
iast_GLOBAL (2.235 ms) : 2173, 2297
.   : milestone, 2235,
profiling (2.043 ms) : 1991, 2094
.   : milestone, 2043,
tracing (2.017 ms) : 1969, 2065
.   : milestone, 2017,
section candidate
no_agent (1.471 ms) : 1460, 1483
.   : milestone, 1471,
appsec (2.395 ms) : 2346, 2444
.   : milestone, 2395,
iast (2.189 ms) : 2127, 2251
.   : milestone, 2189,
iast_GLOBAL (2.231 ms) : 2168, 2293
.   : milestone, 2231,
profiling (2.051 ms) : 2000, 2102
.   : milestone, 2051,
tracing (2.006 ms) : 1958, 2053
.   : milestone, 2006,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.469 ms [1.458 ms, 1.481 ms] -
appsec 2.401 ms [2.351 ms, 2.45 ms] 931.217 µs (63.4%)
iast 2.179 ms [2.117 ms, 2.241 ms] 709.612 µs (48.3%)
iast_GLOBAL 2.235 ms [2.173 ms, 2.297 ms] 765.423 µs (52.1%)
profiling 2.043 ms [1.991 ms, 2.094 ms] 573.497 µs (39.0%)
tracing 2.017 ms [1.969 ms, 2.065 ms] 547.295 µs (37.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.471 ms [1.46 ms, 1.483 ms] -
appsec 2.395 ms [2.346 ms, 2.444 ms] 923.543 µs (62.8%)
iast 2.189 ms [2.127 ms, 2.251 ms] 717.542 µs (48.8%)
iast_GLOBAL 2.231 ms [2.168 ms, 2.293 ms] 759.052 µs (51.6%)
profiling 2.051 ms [2.0 ms, 2.102 ms] 579.517 µs (39.4%)
tracing 2.006 ms [1.958 ms, 2.053 ms] 534.147 µs (36.3%)

@pr-commenter
Copy link

pr-commenter bot commented Jun 24, 2025

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/cleanup-instrumenter-enablement
git_commit_date 1750788935 1750792614
git_commit_sha faeb62c 0e4a616
See matching parameters
Baseline Candidate
ci_job_date 1750793847 1750793847
ci_job_id 996768627 996768627
ci_pipeline_id 68656549 68656549
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce same

mcculls added 2 commits June 24, 2025 20:16
* All instrumentations now declare a target system which they fall under
* We introduce COMMON as the system for instrumentations shared across all
  systems and SECURITY for instrumentations shared between APPSEC and IAST
* We now log when an instrumentation has been disabled
* Special enablement of security instrumentations shared between APPSEC and
  IAST has been moved to `isEnabled` which is now told the enabled systems
…fining instrumentation if you just want the package name
@mcculls mcculls force-pushed the mcculls/cleanup-instrumenter-enablement branch from ef452fa to 0e4a616 Compare June 24, 2025 19:17
@pr-commenter
Copy link

pr-commenter bot commented Jun 24, 2025

Kafka / consumer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/cleanup-instrumenter-enablement
git_commit_date 1750788935 1750792614
git_commit_sha faeb62c 0e4a616
See matching parameters
Baseline Candidate
ci_job_date 1750793883 1750793883
ci_job_id 996768629 996768629
ci_pipeline_id 68656549 68656549
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume same

@mcculls mcculls marked this pull request as ready for review June 24, 2025 21:32
@mcculls mcculls requested review from a team as code owners June 24, 2025 21:32
@mcculls mcculls requested review from Mariovido, sezen-datadog, bric3 and ygree and removed request for a team June 24, 2025 21:32
@PerfectSlayer PerfectSlayer self-requested a review June 25, 2025 06:02
Copy link
Contributor

@PerfectSlayer PerfectSlayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting change for the future! Let me know if I get it right:

We now split if an instrumentation is "applicable" from if it is "enabled".
For this, we only rely on "enabled (target) systems".
To this end, additional compound systems are introduced, like COMMON and SECURITY.

To me, it feels like "applicable" related to "product" activation, and "enabled" relates to "feature" activation due to the introduction of compound systems.
But it feels weird having CSI / security instrumentation targeting SECURITY, and then checking for another sub-system / feature like IAST at "enabled" check. It looks like there is a mix of concerns for TargetSystem.

I would rather try to have exclusive systems, like TRACING, IAST, APPSEC, etc... (no compound like COMMON nor SECURITY) and have an isApplicableFor(Set<TargetSystem> systems) {} module method instead, to allow having multiple systems applicable per module.
So every module should declare all systems which they are supposed to run with.

With this, CSI instrumentations could then be applicable for both APPSEC and IAST and have its child only enabled for specific system if needed.

I agreed this will be harder to make the instrumentation indexes, as each module won't declare the single system they apply for, so we might need to query them for all know systems and check what they answer first. But sounds like it can avoid some future issues with overlapping systems / features - like an instrumentation that will be both RUM and SECURITY for example - and avoid TargetSystem mixing product / feature concerns.

Let me know what you think of it and if I get it right?
Otherwise, I added few other comments along the changes.

@@ -132,6 +132,10 @@ class DefaultInstrumenterForkedTest extends DDSpecification {
"PERIOD_TEST" | true | "period.test" | "asdf"
}

def assertEnabled(target, enabled = true) {
target.isEnabled(EnumSet.of(TargetSystem.TRACING)) == enabled
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will it fail with assert?

Suggested change
target.isEnabled(EnumSet.of(TargetSystem.TRACING)) == enabled
assert target.isEnabled(EnumSet.of(TargetSystem.TRACING)) == enabled

protected boolean defaultEnabled() {
return InstrumenterConfig.get().isIntegrationsEnabled();
}

public boolean isEnabled() {
public boolean isEnabled(Set<TargetSystem> enabledSystems) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: I found the enabledSystems name a bit confusing for the isEnabled() method.
"Is the module enabled for those enabled systems"

What about public boolean isEnabledFor(Set<TargetSystem> systems);?
From the module POV, it does not have to know if the given systems are enabled or not, just to answer "should I be enabled for those systems". That's a nitpick but I feel it would add clarity when there is no Javadoc.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not necessarily for this PR, but I've been pondering whether annotations would be a better way to specify this type of information.
My thinking is that annotations would force you to only use static information. And we could also index the information via an annotation processor.

Comment on lines +25 to +26
public boolean isEnabled(Set<TargetSystem> enabledSystems) {
return super.isEnabled(enabledSystems);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be removed?

Comment on lines +31 to +32
public boolean isEnabled(Set<TargetSystem> enabledSystems) {
return super.isEnabled(enabledSystems);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be removed?

Comment on lines +25 to +26
public boolean isEnabled(Set<TargetSystem> enabledSystems) {
return super.isEnabled(enabledSystems)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be removed?

Comment on lines +25 to +26
public boolean isEnabled(Set<TargetSystem> enabledSystems) {
return super.isEnabled(enabledSystems);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be removed?

Copy link
Contributor

@ygree ygree left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mcculls mcculls marked this pull request as draft July 23, 2025 08:02
@mcculls mcculls added the tag: do not merge Do not merge changes label Jul 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp: core Tracer core tag: do not merge Do not merge changes tag: no release notes Changes to exclude from release notes type: refactoring
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants