DRAFT agentless #9186

jpbempel · 2025-07-16T15:01:31Z

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

pr-commenter · 2025-07-16T15:25:48Z

Debugger benchmarks

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
ci_job_date	1752687447	1752687793
end_time	2025-07-16T17:38:48	2025-07-16T17:44:33
git_branch	master	jpbempel/er-agentless
git_commit_sha	`2fe6aa2`	`005b2b2`
start_time	2025-07-16T17:37:28	2025-07-16T17:43:14

See matching parameters

	Baseline	Candidate
ci_job_id	1032264441	1032264441
ci_pipeline_id	70755933	70755933
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date	1752686938	1752686938

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 9 metrics, 6 unstable metrics.

See unchanged results

scenario	Δ mean agg_http_req_duration_min	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p75	Δ mean agg_http_req_duration_p99	Δ mean throughput
scenario:noprobe	unstable [-36.891µs; +18.612µs] or [-12.968%; +6.542%]	unstable [-53.169µs; +24.995µs] or [-16.218%; +7.624%]	unstable [-68.314µs; +35.155µs] or [-19.820%; +10.200%]	unstable [-135.831µs; +98.508µs] or [-14.349%; +10.406%]	same
scenario:basic	unsure [-7.076µs; -0.648µs] or [-2.820%; -0.258%]	same	same	unstable [-78.454µs; +87.153µs] or [-10.412%; +11.566%]	unstable [-87.279op/s; +229.527op/s] or [-3.317%; +8.722%]
scenario:loop	unsure [+1.978µs; +9.194µs] or [+0.022%; +0.104%]	same	same	same	same

Request duration reports for reports

gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (327.849 µs) : 284, 372
.   : milestone, 328,
basic (279.999 µs) : 273, 287
.   : milestone, 280,
loop (8.962 ms) : 8957, 8967
.   : milestone, 8962,
section candidate
noprobe (313.762 µs) : 285, 343
.   : milestone, 314,
basic (275.151 µs) : 270, 280
.   : milestone, 275,
loop (8.965 ms) : 8960, 8969
.   : milestone, 8965,

baseline results

Scenario	Request median duration [CI 0.99]
noprobe	327.849 µs [283.885 µs, 371.813 µs]
basic	279.999 µs [273.262 µs, 286.737 µs]
loop	8.962 ms [8.957 ms, 8.967 ms]

candidate results

Scenario	Request median duration [CI 0.99]
noprobe	313.762 µs [284.846 µs, 342.678 µs]
basic	275.151 µs [270.037 µs, 280.265 µs]
loop	8.965 ms [8.96 ms, 8.969 ms]

pr-commenter · 2025-07-16T15:41:34Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	jpbempel/er-agentless
git_commit_date	1752667115	1752678047
git_commit_sha	`2fe6aa2`	`eaef526`
release_version	1.52.0-SNAPSHOT~2fe6aa2d23	1.52.0-SNAPSHOT~eaef526026

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1752680119	1752680119
ci_job_id	1031800924	1031800924
ci_pipeline_id	70732844	70732844
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-kd63eo2q 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-kd63eo2q 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 10 performance improvements and 3 performance regressions! Performance is the same for 30 metrics, 10 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:startup:insecure-bank:iast:Agent.start	better [-43.642ms; -36.215ms] or [-3.719%; -3.086%]	1.133s	1.173s
scenario:startup:insecure-bank:iast:BytebuddyAgent	better [-43.613ms; -38.231ms] or [-5.144%; -4.509%]	806.994ms	847.916ms
scenario:startup:insecure-bank:tracing:Agent.start	better [-51.267ms; -42.967ms] or [-4.927%; -4.130%]	0.993s	1.040s
scenario:startup:insecure-bank:tracing:BytebuddyAgent	better [-49.487ms; -43.517ms] or [-6.767%; -5.950%]	684.849ms	731.351ms
scenario:startup:petclinic:appsec:BytebuddyAgent	better [-47.804ms; -25.456ms] or [-6.369%; -3.392%]	713.934ms	750.564ms
scenario:startup:petclinic:iast:Agent.start	better [-44.528ms; -36.098ms] or [-3.798%; -3.079%]	1.132s	1.172s
scenario:startup:petclinic:iast:BytebuddyAgent	better [-43.681ms; -37.854ms] or [-5.160%; -4.472%]	805.710ms	846.477ms
scenario:startup:petclinic:profiling:Agent.start	worse [+30.190ms; +64.240ms] or [+2.524%; +5.370%]	1.243s	1.196s
scenario:startup:petclinic:profiling:BytebuddyAgent	better [-100.396ms; -80.005ms] or [-13.095%; -10.435%]	676.467ms	766.668ms
scenario:startup:petclinic:profiling:GlobalTracer	worse [+135.059ms; +141.912ms] or [+60.590%; +63.664%]	361.393ms	222.908ms
scenario:startup:petclinic:tracing:Agent.start	better [-61.990ms; -21.886ms] or [-5.928%; -2.093%]	1.004s	1.046s
scenario:startup:petclinic:tracing:BytebuddyAgent	better [-57.487ms; -28.921ms] or [-7.822%; -3.935%]	691.752ms	734.956ms
scenario:startup:petclinic:tracing:Remote Config	worse [+19.242µs; +80.858µs] or [+2.963%; +12.452%]	699.417µs	649.367µs

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.52.0-SNAPSHOT~eaef526026, baseline=1.52.0-SNAPSHOT~2fe6aa2d23

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.046 s) : 0, 1045749
Total [baseline] (10.669 s) : 0, 10669160
Agent [candidate] (1.004 s) : 0, 1003811
Total [candidate] (10.769 s) : 0, 10768995
section appsec
Agent [baseline] (1.216 s) : 0, 1215642
Total [baseline] (10.777 s) : 0, 10776841
Agent [candidate] (1.182 s) : 0, 1182436
Total [candidate] (10.786 s) : 0, 10786379
section iast
Agent [baseline] (1.172 s) : 0, 1172311
Total [baseline] (10.89 s) : 0, 10890439
Agent [candidate] (1.132 s) : 0, 1131998
Total [candidate] (10.833 s) : 0, 10832874
section profiling
Agent [baseline] (1.196 s) : 0, 1196185
Total [baseline] (11.04 s) : 0, 11040037
Agent [candidate] (1.243 s) : 0, 1243400
Total [candidate] (10.93 s) : 0, 10930078

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.046 s	-
Agent	appsec	1.216 s	169.893 ms (16.2%)
Agent	iast	1.172 s	126.562 ms (12.1%)
Agent	profiling	1.196 s	150.437 ms (14.4%)
Total	tracing	10.669 s	-
Total	appsec	10.777 s	107.68 ms (1.0%)
Total	iast	10.89 s	221.279 ms (2.1%)
Total	profiling	11.04 s	370.877 ms (3.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.004 s	-
Agent	appsec	1.182 s	178.625 ms (17.8%)
Agent	iast	1.132 s	128.187 ms (12.8%)
Agent	profiling	1.243 s	239.59 ms (23.9%)
Total	tracing	10.769 s	-
Total	appsec	10.786 s	17.384 ms (0.2%)
Total	iast	10.833 s	63.879 ms (0.6%)
Total	profiling	10.93 s	161.083 ms (1.5%)

gantt
    title petclinic - break down per module: candidate=1.52.0-SNAPSHOT~eaef526026, baseline=1.52.0-SNAPSHOT~2fe6aa2d23

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (734.956 ms) : 0, 734956
BytebuddyAgent [candidate] (691.752 ms) : 0, 691752
GlobalTracer [baseline] (242.632 ms) : 0, 242632
GlobalTracer [candidate] (245.022 ms) : 0, 245022
AppSec [baseline] (30.68 ms) : 0, 30680
AppSec [candidate] (31.047 ms) : 0, 31047
Debugger [baseline] (6.083 ms) : 0, 6083
Debugger [candidate] (6.042 ms) : 0, 6042
Remote Config [baseline] (649.367 µs) : 0, 649
Remote Config [candidate] (699.417 µs) : 0, 699
Telemetry [baseline] (8.263 ms) : 0, 8263
Telemetry [candidate] (8.377 ms) : 0, 8377
crashtracking [baseline] (1.442 ms) : 0, 1442
section appsec
BytebuddyAgent [baseline] (750.564 ms) : 0, 750564
BytebuddyAgent [candidate] (713.934 ms) : 0, 713934
GlobalTracer [baseline] (234.023 ms) : 0, 234023
GlobalTracer [candidate] (236.709 ms) : 0, 236709
AppSec [baseline] (170.647 ms) : 0, 170647
AppSec [candidate] (172.527 ms) : 0, 172527
Debugger [baseline] (5.602 ms) : 0, 5602
Debugger [candidate] (5.778 ms) : 0, 5778
Remote Config [baseline] (590.796 µs) : 0, 591
Remote Config [candidate] (614.909 µs) : 0, 615
Telemetry [baseline] (8.082 ms) : 0, 8082
Telemetry [candidate] (8.075 ms) : 0, 8075
crashtracking [baseline] (1.442 ms) : 0, 1442
IAST [baseline] (23.445 ms) : 0, 23445
IAST [candidate] (23.72 ms) : 0, 23720
section iast
BytebuddyAgent [baseline] (846.477 ms) : 0, 846477
BytebuddyAgent [candidate] (805.71 ms) : 0, 805710
GlobalTracer [baseline] (231.027 ms) : 0, 231027
GlobalTracer [candidate] (233.018 ms) : 0, 233018
AppSec [baseline] (28.071 ms) : 0, 28071
AppSec [candidate] (30.778 ms) : 0, 30778
Debugger [baseline] (6.647 ms) : 0, 6647
Debugger [candidate] (5.747 ms) : 0, 5747
Remote Config [baseline] (589.048 µs) : 0, 589
Remote Config [candidate] (591.248 µs) : 0, 591
Telemetry [baseline] (8.005 ms) : 0, 8005
Telemetry [candidate] (8.01 ms) : 0, 8010
crashtracking [baseline] (1.437 ms) : 0, 1437
IAST [baseline] (29.176 ms) : 0, 29176
IAST [candidate] (27.303 ms) : 0, 27303
section profiling
ProfilingAgent [baseline] (104.127 ms) : 0, 104127
ProfilingAgent [candidate] (103.668 ms) : 0, 103668
BytebuddyAgent [baseline] (766.668 ms) : 0, 766668
BytebuddyAgent [candidate] (676.467 ms) : 0, 676467
GlobalTracer [baseline] (222.908 ms) : 0, 222908
GlobalTracer [candidate] (361.393 ms) : 0, 361393
AppSec [baseline] (30.947 ms) : 0, 30947
AppSec [candidate] (31.814 ms) : 0, 31814
Debugger [baseline] (6.338 ms) : 0, 6338
Debugger [candidate] (10.571 ms) : 0, 10571
Remote Config [baseline] (677.301 µs) : 0, 677
Remote Config [candidate] (671.304 µs) : 0, 671
Telemetry [baseline] (14.782 ms) : 0, 14782
Telemetry [candidate] (9.513 ms) : 0, 9513
Profiling [baseline] (104.149 ms) : 0, 104149
Profiling [candidate] (103.692 ms) : 0, 103692
crashtracking [baseline] (1.391 ms) : 0, 1391

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.52.0-SNAPSHOT~eaef526026, baseline=1.52.0-SNAPSHOT~2fe6aa2d23

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.04 s) : 0, 1040494
Total [baseline] (8.623 s) : 0, 8623169
Agent [candidate] (993.377 ms) : 0, 993377
Total [candidate] (8.578 s) : 0, 8578352
section iast
Agent [baseline] (1.173 s) : 0, 1173389
Total [baseline] (9.291 s) : 0, 9291117
Agent [candidate] (1.133 s) : 0, 1133460
Total [candidate] (9.299 s) : 0, 9298870

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.04 s	-
Agent	iast	1.173 s	132.895 ms (12.8%)
Total	tracing	8.623 s	-
Total	iast	9.291 s	667.948 ms (7.7%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	993.377 ms	-
Agent	iast	1.133 s	140.083 ms (14.1%)
Total	tracing	8.578 s	-
Total	iast	9.299 s	720.518 ms (8.4%)

gantt
    title insecure-bank - break down per module: candidate=1.52.0-SNAPSHOT~eaef526026, baseline=1.52.0-SNAPSHOT~2fe6aa2d23

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (731.351 ms) : 0, 731351
BytebuddyAgent [candidate] (684.849 ms) : 0, 684849
GlobalTracer [baseline] (241.275 ms) : 0, 241275
GlobalTracer [candidate] (242.275 ms) : 0, 242275
AppSec [baseline] (30.505 ms) : 0, 30505
AppSec [candidate] (30.513 ms) : 0, 30513
Debugger [baseline] (6.044 ms) : 0, 6044
Debugger [candidate] (5.996 ms) : 0, 5996
Remote Config [baseline] (652.439 µs) : 0, 652
Remote Config [candidate] (671.686 µs) : 0, 672
Telemetry [baseline] (8.282 ms) : 0, 8282
Telemetry [candidate] (8.205 ms) : 0, 8205
crashtracking [baseline] (1.44 ms) : 0, 1440
section iast
BytebuddyAgent [baseline] (847.916 ms) : 0, 847916
BytebuddyAgent [candidate] (806.994 ms) : 0, 806994
GlobalTracer [baseline] (231.74 ms) : 0, 231740
GlobalTracer [candidate] (233.042 ms) : 0, 233042
IAST [baseline] (28.919 ms) : 0, 28919
IAST [candidate] (25.711 ms) : 0, 25711
AppSec [baseline] (26.523 ms) : 0, 26523
AppSec [candidate] (31.656 ms) : 0, 31656
Debugger [baseline] (7.584 ms) : 0, 7584
Debugger [candidate] (6.617 ms) : 0, 6617
Remote Config [baseline] (590.084 µs) : 0, 590
Remote Config [candidate] (592.705 µs) : 0, 593
Telemetry [baseline] (7.871 ms) : 0, 7871
Telemetry [candidate] (7.97 ms) : 0, 7970
crashtracking [baseline] (1.436 ms) : 0, 1436

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	jpbempel/er-agentless
git_commit_date	1752667115	1752678047
git_commit_sha	`2fe6aa2`	`eaef526`
release_version	1.52.0-SNAPSHOT~2fe6aa2d23	1.52.0-SNAPSHOT~eaef526026

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1752679788	1752679788
ci_job_id	1031800925	1031800925
ci_pipeline_id	70732844	70732844
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-wgy4g6u5 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-wgy4g6u5 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 3 performance regressions! Performance is the same for 9 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:insecure-bank:tracing:high_load	worse [+176.222µs; +410.815µs] or [+2.344%; +5.465%]	unstable [-99.229op/s; +53.479op/s] or [-16.105%; +8.680%]	7.810ms	593.250op/s	7.517ms	616.125op/s
scenario:load:insecure-bank:profiling:high_load	worse [+189.543µs; +503.251µs] or [+2.199%; +5.839%]	unstable [-89.101op/s; +48.226op/s] or [-16.560%; +8.963%]	8.966ms	517.594op/s	8.619ms	538.031op/s
scenario:load:petclinic:profiling:high_load	worse [+1.902ms; +2.836ms] or [+4.041%; +6.024%]	unstable [-9.781op/s; +2.728op/s] or [-9.839%; +2.744%]	49.449ms	95.886op/s	47.080ms	99.412op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.52.0-SNAPSHOT~eaef526026, baseline=1.52.0-SNAPSHOT~2fe6aa2d23
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.261 ms) : 4206, 4315
.   : milestone, 4261,
iast (9.406 ms) : 9241, 9571
.   : milestone, 9406,
iast_FULL (14.37 ms) : 14086, 14655
.   : milestone, 14370,
iast_GLOBAL (10.448 ms) : 10265, 10631
.   : milestone, 10448,
profiling (8.619 ms) : 8484, 8755
.   : milestone, 8619,
tracing (7.517 ms) : 7409, 7625
.   : milestone, 7517,
section candidate
no_agent (4.323 ms) : 4274, 4372
.   : milestone, 4323,
iast (9.465 ms) : 9303, 9626
.   : milestone, 9465,
iast_FULL (13.835 ms) : 13562, 14109
.   : milestone, 13835,
iast_GLOBAL (10.502 ms) : 10316, 10688
.   : milestone, 10502,
profiling (8.966 ms) : 8810, 9121
.   : milestone, 8966,
tracing (7.81 ms) : 7700, 7920
.   : milestone, 7810,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.261 ms [4.206 ms, 4.315 ms]	-
iast	9.406 ms [9.241 ms, 9.571 ms]	5.145 ms (120.8%)
iast_FULL	14.37 ms [14.086 ms, 14.655 ms]	10.11 ms (237.3%)
iast_GLOBAL	10.448 ms [10.265 ms, 10.631 ms]	6.188 ms (145.2%)
profiling	8.619 ms [8.484 ms, 8.755 ms]	4.359 ms (102.3%)
tracing	7.517 ms [7.409 ms, 7.625 ms]	3.256 ms (76.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.323 ms [4.274 ms, 4.372 ms]	-
iast	9.465 ms [9.303 ms, 9.626 ms]	5.141 ms (118.9%)
iast_FULL	13.835 ms [13.562 ms, 14.109 ms]	9.512 ms (220.0%)
iast_GLOBAL	10.502 ms [10.316 ms, 10.688 ms]	6.178 ms (142.9%)
profiling	8.966 ms [8.81 ms, 9.121 ms]	4.642 ms (107.4%)
tracing	7.81 ms [7.7 ms, 7.92 ms]	3.487 ms (80.7%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.52.0-SNAPSHOT~eaef526026, baseline=1.52.0-SNAPSHOT~2fe6aa2d23
    dateFormat X
    axisFormat %s
section baseline
no_agent (37.248 ms) : 36944, 37552
.   : milestone, 37248,
appsec (49.074 ms) : 48624, 49524
.   : milestone, 49074,
code_origins (45.604 ms) : 45208, 46000
.   : milestone, 45604,
iast (43.982 ms) : 43606, 44359
.   : milestone, 43982,
profiling (47.08 ms) : 46655, 47504
.   : milestone, 47080,
tracing (44.291 ms) : 43911, 44670
.   : milestone, 44291,
section candidate
no_agent (36.942 ms) : 36646, 37239
.   : milestone, 36942,
appsec (50.025 ms) : 49576, 50475
.   : milestone, 50025,
code_origins (45.131 ms) : 44748, 45515
.   : milestone, 45131,
iast (43.065 ms) : 42678, 43452
.   : milestone, 43065,
profiling (49.449 ms) : 49006, 49892
.   : milestone, 49449,
tracing (44.002 ms) : 43636, 44369
.   : milestone, 44002,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	37.248 ms [36.944 ms, 37.552 ms]	-
appsec	49.074 ms [48.624 ms, 49.524 ms]	11.826 ms (31.8%)
code_origins	45.604 ms [45.208 ms, 46.0 ms]	8.356 ms (22.4%)
iast	43.982 ms [43.606 ms, 44.359 ms]	6.735 ms (18.1%)
profiling	47.08 ms [46.655 ms, 47.504 ms]	9.832 ms (26.4%)
tracing	44.291 ms [43.911 ms, 44.67 ms]	7.043 ms (18.9%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	36.942 ms [36.646 ms, 37.239 ms]	-
appsec	50.025 ms [49.576 ms, 50.475 ms]	13.083 ms (35.4%)
code_origins	45.131 ms [44.748 ms, 45.515 ms]	8.189 ms (22.2%)
iast	43.065 ms [42.678 ms, 43.452 ms]	6.123 ms (16.6%)
profiling	49.449 ms [49.006 ms, 49.892 ms]	12.506 ms (33.9%)
tracing	44.002 ms [43.636 ms, 44.369 ms]	7.06 ms (19.1%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	jpbempel/er-agentless
git_commit_date	1752667115	1752678047
git_commit_sha	`2fe6aa2`	`eaef526`
release_version	1.52.0-SNAPSHOT~2fe6aa2d23	1.52.0-SNAPSHOT~eaef526026

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1752680342	1752680342
ci_job_id	1031800926	1031800926
ci_pipeline_id	70732844	70732844
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-vod6w9i0 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-vod6w9i0 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 0 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:dacapo:tomcat:appsec	better [-1.401ms; -1.062ms] or [-38.245%; -29.000%]	2.432ms	3.663ms

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.52.0-SNAPSHOT~eaef526026, baseline=1.52.0-SNAPSHOT~2fe6aa2d23
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.486 ms) : 1474, 1497
.   : milestone, 1486,
appsec (3.663 ms) : 3446, 3880
.   : milestone, 3663,
iast (2.209 ms) : 2146, 2272
.   : milestone, 2209,
iast_GLOBAL (2.248 ms) : 2186, 2311
.   : milestone, 2248,
profiling (2.069 ms) : 2017, 2121
.   : milestone, 2069,
tracing (2.035 ms) : 1986, 2084
.   : milestone, 2035,
section candidate
no_agent (1.481 ms) : 1470, 1493
.   : milestone, 1481,
appsec (2.432 ms) : 2381, 2482
.   : milestone, 2432,
iast (2.21 ms) : 2147, 2272
.   : milestone, 2210,
iast_GLOBAL (2.251 ms) : 2188, 2314
.   : milestone, 2251,
profiling (2.057 ms) : 2005, 2108
.   : milestone, 2057,
tracing (2.033 ms) : 1984, 2082
.   : milestone, 2033,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.486 ms [1.474 ms, 1.497 ms]	-
appsec	3.663 ms [3.446 ms, 3.88 ms]	2.177 ms (146.6%)
iast	2.209 ms [2.146 ms, 2.272 ms]	723.264 µs (48.7%)
iast_GLOBAL	2.248 ms [2.186 ms, 2.311 ms]	762.669 µs (51.3%)
profiling	2.069 ms [2.017 ms, 2.121 ms]	582.952 µs (39.2%)
tracing	2.035 ms [1.986 ms, 2.084 ms]	549.613 µs (37.0%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.481 ms [1.47 ms, 1.493 ms]	-
appsec	2.432 ms [2.381 ms, 2.482 ms]	950.066 µs (64.1%)
iast	2.21 ms [2.147 ms, 2.272 ms]	728.186 µs (49.2%)
iast_GLOBAL	2.251 ms [2.188 ms, 2.314 ms]	769.768 µs (52.0%)
profiling	2.057 ms [2.005 ms, 2.108 ms]	575.027 µs (38.8%)
tracing	2.033 ms [1.984 ms, 2.082 ms]	551.339 µs (37.2%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.52.0-SNAPSHOT~eaef526026, baseline=1.52.0-SNAPSHOT~2fe6aa2d23
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.96 s) : 14960000, 14960000
.   : milestone, 14960000,
appsec (14.975 s) : 14975000, 14975000
.   : milestone, 14975000,
iast (18.236 s) : 18236000, 18236000
.   : milestone, 18236000,
iast_GLOBAL (18.325 s) : 18325000, 18325000
.   : milestone, 18325000,
profiling (15.183 s) : 15183000, 15183000
.   : milestone, 15183000,
tracing (15.067 s) : 15067000, 15067000
.   : milestone, 15067000,
section candidate
no_agent (15.559 s) : 15559000, 15559000
.   : milestone, 15559000,
appsec (14.854 s) : 14854000, 14854000
.   : milestone, 14854000,
iast (18.534 s) : 18534000, 18534000
.   : milestone, 18534000,
iast_GLOBAL (18.134 s) : 18134000, 18134000
.   : milestone, 18134000,
profiling (15.278 s) : 15278000, 15278000
.   : milestone, 15278000,
tracing (14.851 s) : 14851000, 14851000
.   : milestone, 14851000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.96 s [14.96 s, 14.96 s]	-
appsec	14.975 s [14.975 s, 14.975 s]	15.0 ms (0.1%)
iast	18.236 s [18.236 s, 18.236 s]	3.276 s (21.9%)
iast_GLOBAL	18.325 s [18.325 s, 18.325 s]	3.365 s (22.5%)
profiling	15.183 s [15.183 s, 15.183 s]	223.0 ms (1.5%)
tracing	15.067 s [15.067 s, 15.067 s]	107.0 ms (0.7%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.559 s [15.559 s, 15.559 s]	-
appsec	14.854 s [14.854 s, 14.854 s]	-705.0 ms (-4.5%)
iast	18.534 s [18.534 s, 18.534 s]	2.975 s (19.1%)
iast_GLOBAL	18.134 s [18.134 s, 18.134 s]	2.575 s (16.5%)
profiling	15.278 s [15.278 s, 15.278 s]	-281.0 ms (-1.8%)
tracing	14.851 s [14.851 s, 14.851 s]	-708.0 ms (-4.6%)

DRAFT agentless

eaef526

test failure

005b2b2

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

DRAFT agentless #9186

DRAFT agentless #9186

jpbempel commented Jul 16, 2025

Uh oh!

pr-commenter bot commented Jul 16, 2025 •

edited

Loading

Uh oh!

pr-commenter bot commented Jul 16, 2025

Uh oh!

Uh oh!

DRAFT agentless #9186

Are you sure you want to change the base?

DRAFT agentless #9186

Conversation

jpbempel commented Jul 16, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter bot commented Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Debugger benchmarks

Parameters

Summary

Uh oh!

pr-commenter bot commented Jul 16, 2025

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Uh oh!

pr-commenter bot commented Jul 16, 2025 •

edited

Loading