-
Notifications
You must be signed in to change notification settings - Fork 312
Improving TagMap coverage #9470
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
- adding in more variations - frequency of sets, gets, & removes is the same, but the exact method is now varied between legacy and optimized versions - also added operations from ledger
- replaced some random size selection with TagMapScenario-s - TagMapScenario is mix of TagMapType and varying number of prepopulated elements - the prepopulating helps cover the 3 different branch cases inside OptimizedTagMap: null, TagMap.Entry, BucketGroup - also added calls to checkIntegry and toInternalString, since those methods were largely uncovered - still need to work on better coverage of putAllMerge, probably need a pair of TagMapScenario-s
🎯 Code Coverage 🔗 Commit SHA: c612fba | Docs | Was this helpful? Give us feedback! |
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 47 metrics, 12 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.54.0-SNAPSHOT~c612fba1d9, baseline=1.54.0-SNAPSHOT~c4c1019d62
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.047 s) : 0, 1046536
Total [baseline] (8.649 s) : 0, 8648751
Agent [candidate] (1.046 s) : 0, 1046473
Total [candidate] (8.619 s) : 0, 8618556
section iast
Agent [baseline] (1.183 s) : 0, 1182825
Total [baseline] (9.33 s) : 0, 9330105
Agent [candidate] (1.177 s) : 0, 1177119
Total [candidate] (9.328 s) : 0, 9327645
gantt
title insecure-bank - break down per module: candidate=1.54.0-SNAPSHOT~c612fba1d9, baseline=1.54.0-SNAPSHOT~c4c1019d62
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.471 ms) : 0, 1471
crashtracking [candidate] (1.466 ms) : 0, 1466
BytebuddyAgent [baseline] (734.117 ms) : 0, 734117
BytebuddyAgent [candidate] (732.02 ms) : 0, 732020
GlobalTracer [baseline] (242.28 ms) : 0, 242280
GlobalTracer [candidate] (242.699 ms) : 0, 242699
AppSec [baseline] (30.109 ms) : 0, 30109
AppSec [candidate] (30.098 ms) : 0, 30098
Debugger [baseline] (6.09 ms) : 0, 6090
Debugger [candidate] (6.1 ms) : 0, 6100
Remote Config [baseline] (693.272 µs) : 0, 693
Remote Config [candidate] (678.893 µs) : 0, 679
Telemetry [baseline] (10.72 ms) : 0, 10720
Telemetry [candidate] (12.343 ms) : 0, 12343
section iast
crashtracking [baseline] (1.484 ms) : 0, 1484
crashtracking [candidate] (1.442 ms) : 0, 1442
BytebuddyAgent [baseline] (854.291 ms) : 0, 854291
BytebuddyAgent [candidate] (849.611 ms) : 0, 849611
GlobalTracer [baseline] (233.227 ms) : 0, 233227
GlobalTracer [candidate] (232.623 ms) : 0, 232623
IAST [baseline] (31.188 ms) : 0, 31188
IAST [candidate] (29.322 ms) : 0, 29322
AppSec [baseline] (26.067 ms) : 0, 26067
AppSec [candidate] (28.547 ms) : 0, 28547
Debugger [baseline] (6.679 ms) : 0, 6679
Debugger [candidate] (5.78 ms) : 0, 5780
Remote Config [baseline] (605.334 µs) : 0, 605
Remote Config [candidate] (603.884 µs) : 0, 604
Telemetry [baseline] (8.15 ms) : 0, 8150
Telemetry [candidate] (8.29 ms) : 0, 8290
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.54.0-SNAPSHOT~c612fba1d9, baseline=1.54.0-SNAPSHOT~c4c1019d62
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.059 s) : 0, 1058976
Total [baseline] (10.87 s) : 0, 10870280
Agent [candidate] (1.056 s) : 0, 1055503
Total [candidate] (10.726 s) : 0, 10725691
section appsec
Agent [baseline] (1.23 s) : 0, 1229624
Total [baseline] (10.835 s) : 0, 10835193
Agent [candidate] (1.223 s) : 0, 1223300
Total [candidate] (10.85 s) : 0, 10849539
section iast
Agent [baseline] (1.193 s) : 0, 1193324
Total [baseline] (11.014 s) : 0, 11014155
Agent [candidate] (1.182 s) : 0, 1181811
Total [candidate] (10.954 s) : 0, 10953980
section profiling
Agent [baseline] (1.209 s) : 0, 1209076
Total [baseline] (10.898 s) : 0, 10897973
Agent [candidate] (1.2 s) : 0, 1200201
Total [candidate] (10.894 s) : 0, 10893905
gantt
title petclinic - break down per module: candidate=1.54.0-SNAPSHOT~c612fba1d9, baseline=1.54.0-SNAPSHOT~c4c1019d62
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.473 ms) : 0, 1473
crashtracking [candidate] (1.464 ms) : 0, 1464
BytebuddyAgent [baseline] (739.88 ms) : 0, 739880
BytebuddyAgent [candidate] (737.031 ms) : 0, 737031
GlobalTracer [baseline] (246.336 ms) : 0, 246336
GlobalTracer [candidate] (244.103 ms) : 0, 244103
AppSec [baseline] (30.621 ms) : 0, 30621
AppSec [candidate] (30.371 ms) : 0, 30371
Debugger [baseline] (6.216 ms) : 0, 6216
Debugger [candidate] (6.093 ms) : 0, 6093
Remote Config [baseline] (704.115 µs) : 0, 704
Remote Config [candidate] (698.253 µs) : 0, 698
Telemetry [baseline] (12.473 ms) : 0, 12473
Telemetry [candidate] (14.562 ms) : 0, 14562
section appsec
crashtracking [baseline] (1.475 ms) : 0, 1475
crashtracking [candidate] (1.446 ms) : 0, 1446
BytebuddyAgent [baseline] (759.912 ms) : 0, 759912
BytebuddyAgent [candidate] (755.535 ms) : 0, 755535
GlobalTracer [baseline] (236.357 ms) : 0, 236357
GlobalTracer [candidate] (235.574 ms) : 0, 235574
IAST [baseline] (23.748 ms) : 0, 23748
IAST [candidate] (23.668 ms) : 0, 23668
AppSec [baseline] (167.544 ms) : 0, 167544
AppSec [candidate] (170.506 ms) : 0, 170506
Debugger [baseline] (9.509 ms) : 0, 9509
Debugger [candidate] (6.491 ms) : 0, 6491
Remote Config [baseline] (635.707 µs) : 0, 636
Remote Config [candidate] (611.769 µs) : 0, 612
Telemetry [baseline] (9.215 ms) : 0, 9215
Telemetry [candidate] (8.434 ms) : 0, 8434
section iast
crashtracking [baseline] (1.459 ms) : 0, 1459
crashtracking [candidate] (1.443 ms) : 0, 1443
BytebuddyAgent [baseline] (861.319 ms) : 0, 861319
BytebuddyAgent [candidate] (852.99 ms) : 0, 852990
GlobalTracer [baseline] (235.606 ms) : 0, 235606
GlobalTracer [candidate] (233.788 ms) : 0, 233788
IAST [baseline] (29.861 ms) : 0, 29861
IAST [candidate] (28.257 ms) : 0, 28257
AppSec [baseline] (28.723 ms) : 0, 28723
AppSec [candidate] (28.694 ms) : 0, 28694
Debugger [baseline] (5.885 ms) : 0, 5885
Debugger [candidate] (6.61 ms) : 0, 6610
Remote Config [baseline] (615.329 µs) : 0, 615
Remote Config [candidate] (608.527 µs) : 0, 609
Telemetry [baseline] (8.595 ms) : 0, 8595
Telemetry [candidate] (8.321 ms) : 0, 8321
section profiling
crashtracking [baseline] (1.438 ms) : 0, 1438
crashtracking [candidate] (1.454 ms) : 0, 1454
BytebuddyAgent [baseline] (769.311 ms) : 0, 769311
BytebuddyAgent [candidate] (763.029 ms) : 0, 763029
GlobalTracer [baseline] (225.092 ms) : 0, 225092
GlobalTracer [candidate] (223.336 ms) : 0, 223336
AppSec [baseline] (30.925 ms) : 0, 30925
AppSec [candidate] (30.558 ms) : 0, 30558
Debugger [baseline] (6.39 ms) : 0, 6390
Debugger [candidate] (6.27 ms) : 0, 6270
Remote Config [baseline] (761.042 µs) : 0, 761
Remote Config [candidate] (733.199 µs) : 0, 733
Telemetry [baseline] (15.588 ms) : 0, 15588
Telemetry [candidate] (16.389 ms) : 0, 16389
ProfilingAgent [baseline] (108.764 ms) : 0, 108764
ProfilingAgent [candidate] (108.029 ms) : 0, 108029
Profiling [baseline] (109.413 ms) : 0, 109413
Profiling [candidate] (108.705 ms) : 0, 108705
LoadParameters
See matching parameters
SummaryFound 1 performance improvements and 1 performance regressions! Performance is the same for 10 metrics, 12 unstable metrics.
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.54.0-SNAPSHOT~c612fba1d9, baseline=1.54.0-SNAPSHOT~c4c1019d62
dateFormat X
axisFormat %s
section baseline
no_agent (4.329 ms) : 4277, 4381
. : milestone, 4329,
iast (9.257 ms) : 9102, 9411
. : milestone, 9257,
iast_FULL (14.135 ms) : 13846, 14423
. : milestone, 14135,
iast_GLOBAL (10.24 ms) : 10046, 10434
. : milestone, 10240,
profiling (8.933 ms) : 8785, 9081
. : milestone, 8933,
tracing (7.405 ms) : 7302, 7507
. : milestone, 7405,
section candidate
no_agent (4.484 ms) : 4423, 4545
. : milestone, 4484,
iast (8.945 ms) : 8798, 9092
. : milestone, 8945,
iast_FULL (13.687 ms) : 13413, 13960
. : milestone, 13687,
iast_GLOBAL (10.332 ms) : 10131, 10532
. : milestone, 10332,
profiling (8.542 ms) : 8410, 8675
. : milestone, 8542,
tracing (7.312 ms) : 7210, 7415
. : milestone, 7312,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.54.0-SNAPSHOT~c612fba1d9, baseline=1.54.0-SNAPSHOT~c4c1019d62
dateFormat X
axisFormat %s
section baseline
no_agent (37.949 ms) : 37645, 38252
. : milestone, 37949,
appsec (49.104 ms) : 48670, 49538
. : milestone, 49104,
code_origins (44.974 ms) : 44587, 45361
. : milestone, 44974,
iast (44.689 ms) : 44298, 45081
. : milestone, 44689,
profiling (47.766 ms) : 47297, 48234
. : milestone, 47766,
tracing (44.808 ms) : 44419, 45197
. : milestone, 44808,
section candidate
no_agent (36.991 ms) : 36689, 37292
. : milestone, 36991,
appsec (49.733 ms) : 49280, 50186
. : milestone, 49733,
code_origins (44.816 ms) : 44413, 45218
. : milestone, 44816,
iast (45.04 ms) : 44645, 45435
. : milestone, 45040,
profiling (47.443 ms) : 47024, 47862
. : milestone, 47443,
tracing (44.459 ms) : 44077, 44840
. : milestone, 44459,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.54.0-SNAPSHOT~c612fba1d9, baseline=1.54.0-SNAPSHOT~c4c1019d62
dateFormat X
axisFormat %s
section baseline
no_agent (1.475 ms) : 1463, 1486
. : milestone, 1475,
appsec (3.637 ms) : 3420, 3853
. : milestone, 3637,
iast (2.196 ms) : 2134, 2259
. : milestone, 2196,
iast_GLOBAL (2.244 ms) : 2181, 2307
. : milestone, 2244,
profiling (2.072 ms) : 2019, 2125
. : milestone, 2072,
tracing (2.011 ms) : 1962, 2060
. : milestone, 2011,
section candidate
no_agent (1.468 ms) : 1457, 1480
. : milestone, 1468,
appsec (3.653 ms) : 3437, 3869
. : milestone, 3653,
iast (2.201 ms) : 2139, 2264
. : milestone, 2201,
iast_GLOBAL (2.24 ms) : 2177, 2303
. : milestone, 2240,
profiling (2.475 ms) : 2310, 2640
. : milestone, 2475,
tracing (2.01 ms) : 1961, 2059
. : milestone, 2010,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.54.0-SNAPSHOT~c612fba1d9, baseline=1.54.0-SNAPSHOT~c4c1019d62
dateFormat X
axisFormat %s
section baseline
no_agent (15.52 s) : 15520000, 15520000
. : milestone, 15520000,
appsec (15.04 s) : 15040000, 15040000
. : milestone, 15040000,
iast (18.779 s) : 18779000, 18779000
. : milestone, 18779000,
iast_GLOBAL (18.21 s) : 18210000, 18210000
. : milestone, 18210000,
profiling (16.005 s) : 16005000, 16005000
. : milestone, 16005000,
tracing (15.006 s) : 15006000, 15006000
. : milestone, 15006000,
section candidate
no_agent (15.329 s) : 15329000, 15329000
. : milestone, 15329000,
appsec (14.798 s) : 14798000, 14798000
. : milestone, 14798000,
iast (18.243 s) : 18243000, 18243000
. : milestone, 18243000,
iast_GLOBAL (18.063 s) : 18063000, 18063000
. : milestone, 18063000,
profiling (15.44 s) : 15440000, 15440000
. : milestone, 15440000,
tracing (14.933 s) : 14933000, 14933000
. : milestone, 14933000,
|
What Does This Do
Reduces the randomness in TagMap tests
Tests that were using a single randomly selected size -- now use TagMapScenario which is a mix of TagMapType and a pre-populated size
The pre-populating helps hit the 3 common branch paths inside OptimizedTagMap: null, TagMap.Entry, and BucketGroup
Motivation
Eliminate sporadic coverage failures in CI