Enable performance SLO change tracking #9447

igoragoli · 2025-09-01T11:29:38Z

What Does This Do

Update check-slo-breaches to fetch a GitHub token with dd-octo-sts and export it to bp-runner.
- Here is the check-slo-breaches template being used: https://github.com/DataDog/benchmarking-platform-tools/blob/main/images/templates/gitlab/check-slo-breaches.template.yml.
Add a dd-octo-sts trust policy allowing GitLab CI jobs from all branches/tags to read GitHub contents.
- This wide scope is necessary to cover manual macrobenchmark runs, which can happen from any branch/tag and cause check-slo-breaches to run.
Remove the version pin on notify-slo-breaches to use an updated version of the template.

This is necessary for check-slo-breaches to track SLO changes using GitHub's API.

Here's a successful check-slo-breaches run: https://gitlab.ddbuild.io/DataDog/apm-reliability/dd-trace-java/-/jobs/1110816321

Motivation

https://datadoghq.atlassian.net/browse/APMSP-2198

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: APMSP-2198.

Add read-only trust policy to enable GitLab CI jobs to access GitHub API for SLO change tracking. Policy allows all branches and tags from the DataDog/apm-reliability/dd-trace-java project.

- Add id_tokens configuration for dd-octo-sts authentication - Fetch GitHub token in before_script using dd-octo-sts - Export GITHUB_TOKEN before running bp-runner for SLO tracking - Revoke token in after_script for security

datadog-official · 2025-09-01T11:43:12Z

🎯 Code Coverage
• Patch Coverage: 100.00%
• Total Coverage: 57.62% (+0.01%)

View detailed report

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: d1f8282 | Docs | Was this helpful? Give us feedback!}

pr-commenter · 2025-09-01T11:55:52Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	augusto/slo-change-detection
git_commit_date	1756829606	1756900175
git_commit_sha	`0a4e9d0`	`d1f8282`
release_version	1.53.0-SNAPSHOT~0a4e9d0069	1.53.0-SNAPSHOT~d1f8282717

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1756902028	1756902028
ci_job_id	1110816310	1110816310
ci_pipeline_id	75398046	75398046
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-6ajvkexf 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-6ajvkexf 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 47 metrics, 12 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.53.0-SNAPSHOT~d1f8282717, baseline=1.53.0-SNAPSHOT~0a4e9d0069

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.049 s) : 0, 1048838
Total [baseline] (10.753 s) : 0, 10753089
Agent [candidate] (1.065 s) : 0, 1064830
Total [candidate] (10.768 s) : 0, 10767972
section appsec
Agent [baseline] (1.229 s) : 0, 1228937
Total [baseline] (10.89 s) : 0, 10890313
Agent [candidate] (1.225 s) : 0, 1225073
Total [candidate] (10.814 s) : 0, 10813879
section iast
Agent [baseline] (1.191 s) : 0, 1190989
Total [baseline] (11.005 s) : 0, 11004829
Agent [candidate] (1.188 s) : 0, 1187674
Total [candidate] (11.143 s) : 0, 11142933
section profiling
Agent [baseline] (1.208 s) : 0, 1207579
Total [baseline] (10.928 s) : 0, 10927997
Agent [candidate] (1.198 s) : 0, 1198146
Total [candidate] (10.89 s) : 0, 10890411

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.049 s	-
Agent	appsec	1.229 s	180.099 ms (17.2%)
Agent	iast	1.191 s	142.151 ms (13.6%)
Agent	profiling	1.208 s	158.741 ms (15.1%)
Total	tracing	10.753 s	-
Total	appsec	10.89 s	137.224 ms (1.3%)
Total	iast	11.005 s	251.74 ms (2.3%)
Total	profiling	10.928 s	174.907 ms (1.6%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.065 s	-
Agent	appsec	1.225 s	160.243 ms (15.0%)
Agent	iast	1.188 s	122.844 ms (11.5%)
Agent	profiling	1.198 s	133.316 ms (12.5%)
Total	tracing	10.768 s	-
Total	appsec	10.814 s	45.907 ms (0.4%)
Total	iast	11.143 s	374.961 ms (3.5%)
Total	profiling	10.89 s	122.439 ms (1.1%)

gantt
    title petclinic - break down per module: candidate=1.53.0-SNAPSHOT~d1f8282717, baseline=1.53.0-SNAPSHOT~0a4e9d0069

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.465 ms) : 0, 1465
crashtracking [candidate] (1.477 ms) : 0, 1477
BytebuddyAgent [baseline] (733.339 ms) : 0, 733339
BytebuddyAgent [candidate] (744.096 ms) : 0, 744096
GlobalTracer [baseline] (243.22 ms) : 0, 243220
GlobalTracer [candidate] (245.874 ms) : 0, 245874
AppSec [baseline] (30.121 ms) : 0, 30121
AppSec [candidate] (30.644 ms) : 0, 30644
Debugger [baseline] (6.051 ms) : 0, 6051
Debugger [candidate] (6.116 ms) : 0, 6116
Remote Config [baseline] (671.041 µs) : 0, 671
Remote Config [candidate] (675.094 µs) : 0, 675
Telemetry [baseline] (12.906 ms) : 0, 12906
Telemetry [candidate] (14.672 ms) : 0, 14672
section appsec
crashtracking [baseline] (1.47 ms) : 0, 1470
crashtracking [candidate] (1.461 ms) : 0, 1461
BytebuddyAgent [baseline] (758.963 ms) : 0, 758963
BytebuddyAgent [candidate] (756.229 ms) : 0, 756229
GlobalTracer [baseline] (236.028 ms) : 0, 236028
GlobalTracer [candidate] (235.847 ms) : 0, 235847
IAST [baseline] (23.871 ms) : 0, 23871
IAST [candidate] (23.626 ms) : 0, 23626
AppSec [baseline] (170.944 ms) : 0, 170944
AppSec [candidate] (168.758 ms) : 0, 168758
Debugger [baseline] (6.54 ms) : 0, 6540
Debugger [candidate] (8.838 ms) : 0, 8838
Remote Config [baseline] (639.917 µs) : 0, 640
Remote Config [candidate] (638.285 µs) : 0, 638
Telemetry [baseline] (9.297 ms) : 0, 9297
Telemetry [candidate] (8.465 ms) : 0, 8465
section iast
crashtracking [baseline] (1.474 ms) : 0, 1474
crashtracking [candidate] (1.459 ms) : 0, 1459
BytebuddyAgent [baseline] (860.095 ms) : 0, 860095
BytebuddyAgent [candidate] (854.151 ms) : 0, 854151
GlobalTracer [baseline] (235.872 ms) : 0, 235872
GlobalTracer [candidate] (236.117 ms) : 0, 236117
IAST [baseline] (27.021 ms) : 0, 27021
IAST [candidate] (27.632 ms) : 0, 27632
AppSec [baseline] (27.049 ms) : 0, 27049
AppSec [candidate] (29.34 ms) : 0, 29340
Debugger [baseline] (9.299 ms) : 0, 9299
Debugger [candidate] (8.418 ms) : 0, 8418
Remote Config [baseline] (604.592 µs) : 0, 605
Remote Config [candidate] (627.011 µs) : 0, 627
Telemetry [baseline] (8.332 ms) : 0, 8332
Telemetry [candidate] (8.68 ms) : 0, 8680
section profiling
crashtracking [baseline] (1.47 ms) : 0, 1470
crashtracking [candidate] (1.444 ms) : 0, 1444
BytebuddyAgent [baseline] (768.37 ms) : 0, 768370
BytebuddyAgent [candidate] (762.468 ms) : 0, 762468
GlobalTracer [baseline] (224.501 ms) : 0, 224501
GlobalTracer [candidate] (222.703 ms) : 0, 222703
AppSec [baseline] (30.142 ms) : 0, 30142
AppSec [candidate] (30.772 ms) : 0, 30772
Debugger [baseline] (6.33 ms) : 0, 6330
Debugger [candidate] (6.312 ms) : 0, 6312
Remote Config [baseline] (715.135 µs) : 0, 715
Remote Config [candidate] (685.392 µs) : 0, 685
Telemetry [baseline] (16.812 ms) : 0, 16812
Telemetry [candidate] (15.519 ms) : 0, 15519
ProfilingAgent [baseline] (108.508 ms) : 0, 108508
ProfilingAgent [candidate] (107.602 ms) : 0, 107602
Profiling [baseline] (109.16 ms) : 0, 109160
Profiling [candidate] (108.25 ms) : 0, 108250

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.53.0-SNAPSHOT~d1f8282717, baseline=1.53.0-SNAPSHOT~0a4e9d0069

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1062882
Total [baseline] (8.682 s) : 0, 8682023
Agent [candidate] (1.05 s) : 0, 1049601
Total [candidate] (8.632 s) : 0, 8632379
section iast
Agent [baseline] (1.183 s) : 0, 1183230
Total [baseline] (9.379 s) : 0, 9379152
Agent [candidate] (1.181 s) : 0, 1180926
Total [candidate] (9.432 s) : 0, 9431679

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.063 s	-
Agent	iast	1.183 s	120.348 ms (11.3%)
Total	tracing	8.682 s	-
Total	iast	9.379 s	697.129 ms (8.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.05 s	-
Agent	iast	1.181 s	131.325 ms (12.5%)
Total	tracing	8.632 s	-
Total	iast	9.432 s	799.3 ms (9.3%)

gantt
    title insecure-bank - break down per module: candidate=1.53.0-SNAPSHOT~d1f8282717, baseline=1.53.0-SNAPSHOT~0a4e9d0069

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.476 ms) : 0, 1476
crashtracking [candidate] (1.464 ms) : 0, 1464
BytebuddyAgent [baseline] (743.304 ms) : 0, 743304
BytebuddyAgent [candidate] (733.748 ms) : 0, 733748
GlobalTracer [baseline] (246.063 ms) : 0, 246063
GlobalTracer [candidate] (243.464 ms) : 0, 243464
AppSec [baseline] (30.749 ms) : 0, 30749
AppSec [candidate] (30.144 ms) : 0, 30144
Debugger [baseline] (6.145 ms) : 0, 6145
Debugger [candidate] (6.045 ms) : 0, 6045
Remote Config [baseline] (701.232 µs) : 0, 701
Remote Config [candidate] (669.416 µs) : 0, 669
Telemetry [baseline] (13.148 ms) : 0, 13148
Telemetry [candidate] (13.019 ms) : 0, 13019
section iast
crashtracking [baseline] (1.467 ms) : 0, 1467
crashtracking [candidate] (1.454 ms) : 0, 1454
BytebuddyAgent [baseline] (853.407 ms) : 0, 853407
BytebuddyAgent [candidate] (852.009 ms) : 0, 852009
GlobalTracer [baseline] (234.452 ms) : 0, 234452
GlobalTracer [candidate] (234.411 ms) : 0, 234411
IAST [baseline] (29.391 ms) : 0, 29391
IAST [candidate] (29.595 ms) : 0, 29595
AppSec [baseline] (27.819 ms) : 0, 27819
AppSec [candidate] (27.728 ms) : 0, 27728
Debugger [baseline] (6.651 ms) : 0, 6651
Debugger [candidate] (5.776 ms) : 0, 5776
Remote Config [baseline] (613.792 µs) : 0, 614
Remote Config [candidate] (608.457 µs) : 0, 608
Telemetry [baseline] (8.278 ms) : 0, 8278
Telemetry [candidate] (8.333 ms) : 0, 8333

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	augusto/slo-change-detection
git_commit_date	1756829606	1756900175
git_commit_sha	`0a4e9d0`	`d1f8282`
release_version	1.53.0-SNAPSHOT~0a4e9d0069	1.53.0-SNAPSHOT~d1f8282717

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1756901796	1756901796
ci_job_id	1110816311	1110816311
ci_pipeline_id	75398046	75398046
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-qj8tg6bi 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-qj8tg6bi 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 1 performance regressions! Performance is the same for 10 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:insecure-bank:profiling:high_load	worse [+173.493µs; +459.783µs] or [+2.053%; +5.442%]	unstable [-91.691op/s; +52.316op/s] or [-16.707%; +9.533%]	8.766ms	529.125op/s	8.450ms	548.812op/s
scenario:load:petclinic:no_agent:high_load	better [-2.645ms; -2.023ms] or [-7.074%; -5.411%]	unstable [-0.996op/s; +17.671op/s] or [-0.796%; +14.127%]	35.054ms	133.425op/s	37.388ms	125.088op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~d1f8282717, baseline=1.53.0-SNAPSHOT~0a4e9d0069
    dateFormat X
    axisFormat %s
section baseline
no_agent (37.388 ms) : 37084, 37691
.   : milestone, 37388,
appsec (48.301 ms) : 47880, 48721
.   : milestone, 48301,
code_origins (45.522 ms) : 45126, 45918
.   : milestone, 45522,
iast (44.051 ms) : 43665, 44437
.   : milestone, 44051,
profiling (47.451 ms) : 46999, 47904
.   : milestone, 47451,
tracing (44.48 ms) : 44100, 44859
.   : milestone, 44480,
section candidate
no_agent (35.054 ms) : 34780, 35327
.   : milestone, 35054,
appsec (47.825 ms) : 47426, 48224
.   : milestone, 47825,
code_origins (45.645 ms) : 45246, 46044
.   : milestone, 45645,
iast (45.195 ms) : 44804, 45587
.   : milestone, 45195,
profiling (47.376 ms) : 46928, 47825
.   : milestone, 47376,
tracing (44.678 ms) : 44292, 45064
.   : milestone, 44678,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	37.388 ms [37.084 ms, 37.691 ms]	-
appsec	48.301 ms [47.88 ms, 48.721 ms]	10.913 ms (29.2%)
code_origins	45.522 ms [45.126 ms, 45.918 ms]	8.134 ms (21.8%)
iast	44.051 ms [43.665 ms, 44.437 ms]	6.663 ms (17.8%)
profiling	47.451 ms [46.999 ms, 47.904 ms]	10.064 ms (26.9%)
tracing	44.48 ms [44.1 ms, 44.859 ms]	7.092 ms (19.0%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	35.054 ms [34.78 ms, 35.327 ms]	-
appsec	47.825 ms [47.426 ms, 48.224 ms]	12.771 ms (36.4%)
code_origins	45.645 ms [45.246 ms, 46.044 ms]	10.591 ms (30.2%)
iast	45.195 ms [44.804 ms, 45.587 ms]	10.142 ms (28.9%)
profiling	47.376 ms [46.928 ms, 47.825 ms]	12.323 ms (35.2%)
tracing	44.678 ms [44.292 ms, 45.064 ms]	9.624 ms (27.5%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~d1f8282717, baseline=1.53.0-SNAPSHOT~0a4e9d0069
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.361 ms) : 4309, 4413
.   : milestone, 4361,
iast (9.205 ms) : 9042, 9369
.   : milestone, 9205,
iast_FULL (13.808 ms) : 13532, 14083
.   : milestone, 13808,
iast_GLOBAL (10.248 ms) : 10053, 10442
.   : milestone, 10248,
profiling (8.45 ms) : 8319, 8580
.   : milestone, 8450,
tracing (7.768 ms) : 7655, 7880
.   : milestone, 7768,
section candidate
no_agent (4.49 ms) : 4440, 4541
.   : milestone, 4490,
iast (9.2 ms) : 9042, 9359
.   : milestone, 9200,
iast_FULL (13.963 ms) : 13682, 14245
.   : milestone, 13963,
iast_GLOBAL (10.319 ms) : 10138, 10500
.   : milestone, 10319,
profiling (8.766 ms) : 8631, 8902
.   : milestone, 8766,
tracing (7.832 ms) : 7715, 7949
.   : milestone, 7832,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.361 ms [4.309 ms, 4.413 ms]	-
iast	9.205 ms [9.042 ms, 9.369 ms]	4.844 ms (111.1%)
iast_FULL	13.808 ms [13.532 ms, 14.083 ms]	9.447 ms (216.6%)
iast_GLOBAL	10.248 ms [10.053 ms, 10.442 ms]	5.887 ms (135.0%)
profiling	8.45 ms [8.319 ms, 8.58 ms]	4.089 ms (93.8%)
tracing	7.768 ms [7.655 ms, 7.88 ms]	3.407 ms (78.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.49 ms [4.44 ms, 4.541 ms]	-
iast	9.2 ms [9.042 ms, 9.359 ms]	4.71 ms (104.9%)
iast_FULL	13.963 ms [13.682 ms, 14.245 ms]	9.473 ms (211.0%)
iast_GLOBAL	10.319 ms [10.138 ms, 10.5 ms]	5.828 ms (129.8%)
profiling	8.766 ms [8.631 ms, 8.902 ms]	4.276 ms (95.2%)
tracing	7.832 ms [7.715 ms, 7.949 ms]	3.342 ms (74.4%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	augusto/slo-change-detection
git_commit_date	1756829606	1756900175
git_commit_sha	`0a4e9d0`	`d1f8282`
release_version	1.53.0-SNAPSHOT~0a4e9d0069	1.53.0-SNAPSHOT~d1f8282717

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1756902290	1756902290
ci_job_id	1110816312	1110816312
ci_pipeline_id	75398046	75398046
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-2f1fnotg 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-2f1fnotg 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~d1f8282717, baseline=1.53.0-SNAPSHOT~0a4e9d0069
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.472 ms) : 1461, 1484
.   : milestone, 1472,
appsec (3.661 ms) : 3443, 3879
.   : milestone, 3661,
iast (2.199 ms) : 2136, 2262
.   : milestone, 2199,
iast_GLOBAL (2.244 ms) : 2181, 2307
.   : milestone, 2244,
profiling (2.458 ms) : 2248, 2669
.   : milestone, 2458,
tracing (2.013 ms) : 1965, 2062
.   : milestone, 2013,
section candidate
no_agent (1.473 ms) : 1461, 1484
.   : milestone, 1473,
appsec (3.601 ms) : 3390, 3812
.   : milestone, 3601,
iast (2.201 ms) : 2138, 2264
.   : milestone, 2201,
iast_GLOBAL (2.237 ms) : 2174, 2300
.   : milestone, 2237,
profiling (2.033 ms) : 1983, 2083
.   : milestone, 2033,
tracing (2.012 ms) : 1963, 2061
.   : milestone, 2012,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.472 ms [1.461 ms, 1.484 ms]	-
appsec	3.661 ms [3.443 ms, 3.879 ms]	2.189 ms (148.7%)
iast	2.199 ms [2.136 ms, 2.262 ms]	726.965 µs (49.4%)
iast_GLOBAL	2.244 ms [2.181 ms, 2.307 ms]	771.461 µs (52.4%)
profiling	2.458 ms [2.248 ms, 2.669 ms]	985.978 µs (67.0%)
tracing	2.013 ms [1.965 ms, 2.062 ms]	540.862 µs (36.7%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.473 ms [1.461 ms, 1.484 ms]	-
appsec	3.601 ms [3.39 ms, 3.812 ms]	2.128 ms (144.5%)
iast	2.201 ms [2.138 ms, 2.264 ms]	728.109 µs (49.4%)
iast_GLOBAL	2.237 ms [2.174 ms, 2.3 ms]	764.504 µs (51.9%)
profiling	2.033 ms [1.983 ms, 2.083 ms]	559.743 µs (38.0%)
tracing	2.012 ms [1.963 ms, 2.061 ms]	539.302 µs (36.6%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~d1f8282717, baseline=1.53.0-SNAPSHOT~0a4e9d0069
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.519 s) : 15519000, 15519000
.   : milestone, 15519000,
appsec (14.984 s) : 14984000, 14984000
.   : milestone, 14984000,
iast (18.526 s) : 18526000, 18526000
.   : milestone, 18526000,
iast_GLOBAL (18.113 s) : 18113000, 18113000
.   : milestone, 18113000,
profiling (16.021 s) : 16021000, 16021000
.   : milestone, 16021000,
tracing (14.966 s) : 14966000, 14966000
.   : milestone, 14966000,
section candidate
no_agent (15.376 s) : 15376000, 15376000
.   : milestone, 15376000,
appsec (14.915 s) : 14915000, 14915000
.   : milestone, 14915000,
iast (18.684 s) : 18684000, 18684000
.   : milestone, 18684000,
iast_GLOBAL (18.225 s) : 18225000, 18225000
.   : milestone, 18225000,
profiling (15.273 s) : 15273000, 15273000
.   : milestone, 15273000,
tracing (14.972 s) : 14972000, 14972000
.   : milestone, 14972000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.519 s [15.519 s, 15.519 s]	-
appsec	14.984 s [14.984 s, 14.984 s]	-535.0 ms (-3.4%)
iast	18.526 s [18.526 s, 18.526 s]	3.007 s (19.4%)
iast_GLOBAL	18.113 s [18.113 s, 18.113 s]	2.594 s (16.7%)
profiling	16.021 s [16.021 s, 16.021 s]	502.0 ms (3.2%)
tracing	14.966 s [14.966 s, 14.966 s]	-553.0 ms (-3.6%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.376 s [15.376 s, 15.376 s]	-
appsec	14.915 s [14.915 s, 14.915 s]	-461.0 ms (-3.0%)
iast	18.684 s [18.684 s, 18.684 s]	3.308 s (21.5%)
iast_GLOBAL	18.225 s [18.225 s, 18.225 s]	2.849 s (18.5%)
profiling	15.273 s [15.273 s, 15.273 s]	-103.0 ms (-0.7%)
tracing	14.972 s [14.972 s, 14.972 s]	-404.0 ms (-2.6%)

…tions

igoragoli added 3 commits September 1, 2025 11:44

feat: add dd-octo-sts trust policy for GitLab CI GitHub access

c2fe156

Add read-only trust policy to enable GitLab CI jobs to access GitHub API for SLO change tracking. Policy allows all branches and tags from the DataDog/apm-reliability/dd-trace-java project.

add newline to trust policy

b1288a4

igoragoli added type: enhancement Enhancements and improvements tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Sep 1, 2025

igoragoli changed the title ~~chore(ci): enable performance SLO change tracking~~ feat(ci): enable performance SLO change tracking Sep 1, 2025

igoragoli added 2 commits September 1, 2025 14:04

fix: scope trust policy to master branch only

17546e6

Merge branch 'master' into augusto/slo-change-detection

7f9966f

igoragoli marked this pull request as ready for review September 1, 2025 12:08

igoragoli requested review from a team as code owners September 1, 2025 12:08

igoragoli requested review from dougqh and erikayasuda and removed request for a team September 1, 2025 12:08

igoragoli added 2 commits September 1, 2025 14:23

fix: broaden trust policy scope to support manual runs on any branch

e680185

feat: Remove notify-slo-breaches version pin to use improved notifica…

6fffc4d

…tions

igoragoli mentioned this pull request Sep 1, 2025

Unpin notify-slo-breaches template #9386

Closed

igoragoli marked this pull request as draft September 2, 2025 08:29

PerfectSlayer changed the title ~~feat(ci): enable performance SLO change tracking~~ Enable performance SLO change tracking Sep 2, 2025

igoragoli added 3 commits September 3, 2025 10:41

feat: update check-slo-breaches to use a template

36e628e

trigger pipeline

d695f7d

trigger pipeline

d1f8282

igoragoli marked this pull request as ready for review September 3, 2025 12:38

igoragoli requested a review from ddyurchenko September 3, 2025 12:38

ddyurchenko approved these changes Sep 4, 2025

View reviewed changes

igoragoli merged commit fa788ac into master Sep 5, 2025
507 checks passed

igoragoli deleted the augusto/slo-change-detection branch September 5, 2025 08:09

github-actions bot added this to the 1.54.0 milestone Sep 5, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Enable performance SLO change tracking #9447

Enable performance SLO change tracking #9447

Uh oh!

igoragoli commented Sep 1, 2025 •

edited

Loading

Uh oh!

datadog-official bot commented Sep 1, 2025 •

edited by datadog-datadog-prod-us1 bot

Loading

Uh oh!

pr-commenter bot commented Sep 1, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Enable performance SLO change tracking #9447

Enable performance SLO change tracking #9447

Uh oh!

Conversation

igoragoli commented Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

datadog-official bot commented Sep 1, 2025 • edited by datadog-datadog-prod-us1 bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter bot commented Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Uh oh!

Uh oh!

igoragoli commented Sep 1, 2025 •

edited

Loading

datadog-official bot commented Sep 1, 2025 •

edited by datadog-datadog-prod-us1 bot

Loading

pr-commenter bot commented Sep 1, 2025 •

edited

Loading