-
Notifications
You must be signed in to change notification settings - Fork 40
Description
Description
The proxy_component_test fails when run concurrently with other tests:
vscode ➜ /workspaces/communication (main) $ bazel test --nocache_test_results //...
WARNING: /workspaces/communication/score/mw/com/impl/rust/BUILD:51:11: target '//score/mw/com/impl/rust:proxy_bridge_macros' is deprecated: Please change your dependency to //score/mw/com/impl/rust:bridge_macros
INFO: Analyzed 398 targets (0 packages loaded, 139 targets configured).
FAIL: //score/mw/com/impl/bindings/lola/test:proxy_component_test (see /var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/test.log)
INFO: Found 263 targets and 135 test targets...
INFO: Elapsed time: 14.689s, Critical Path: 12.30s
INFO: 134 processes: 151 processwrapper-sandbox.
INFO: Build completed, 1 test FAILED, 134 total actions
//score/mw/com/message_passing:resmgr_factory_test SKIPPED
//score/mw/com/message_passing:resmgr_traits_test SKIPPED
//bazel/tools:invalid_schema_and_valid_json_test PASSED in 0.1s
...
//score/mw/com/message_passing:unit_test PASSED in 0.4s
//score/mw/com/impl/bindings/lola/test:proxy_component_test FAILED in 2.5s
/var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/test.log
Executed 133 out of 135 tests: 132 tests pass, 1 fails locally and 2 were skipped.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
This failure does not always happen, but often enough to be annoying.
When the test itself is run in parallel with multiple instances, each one is failing:
vscode ➜ /workspaces/communication (main) $ bazel test //score/mw/com/impl/bindings/lola/test:proxy_component_test --runs_per_test=12
INFO: Analyzed target //score/mw/com/impl/bindings/lola/test:proxy_component_test (0 packages loaded, 5 targets configured).
FAIL: //score/mw/com/impl/bindings/lola/test:proxy_component_test (run 10 of 12) (see /var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_10_of_12/test.log)
FAIL: //score/mw/com/impl/bindings/lola/test:proxy_component_test (run 7 of 12) (see /var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_7_of_12/test.log)
FAIL: //score/mw/com/impl/bindings/lola/test:proxy_component_test (run 4 of 12) (see /var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_4_of_12/test.log)
FAIL: //score/mw/com/impl/bindings/lola/test:proxy_component_test (run 5 of 12) (see /var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_5_of_12/test.log)
FAIL: //score/mw/com/impl/bindings/lola/test:proxy_component_test (run 2 of 12) (see /var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_2_of_12/test.log)
FAIL: //score/mw/com/impl/bindings/lola/test:proxy_component_test (run 9 of 12) (see /var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_9_of_12/test.log)
FAIL: //score/mw/com/impl/bindings/lola/test:proxy_component_test (run 8 of 12) (see /var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_8_of_12/test.log)
FAIL: //score/mw/com/impl/bindings/lola/test:proxy_component_test (run 6 of 12) (see /var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_6_of_12/test.log)
FAIL: //score/mw/com/impl/bindings/lola/test:proxy_component_test (run 11 of 12) (see /var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_11_of_12/test.log)
FAIL: //score/mw/com/impl/bindings/lola/test:proxy_component_test (run 3 of 12) (see /var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_3_of_12/test.log)
FAIL: //score/mw/com/impl/bindings/lola/test:proxy_component_test (run 12 of 12) (see /var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_12_of_12/test.log)
FAIL: //score/mw/com/impl/bindings/lola/test:proxy_component_test (run 1 of 12) (see /var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_1_of_12/test.log)
FAILED: //score/mw/com/impl/bindings/lola/test:proxy_component_test (Summary)
/var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_10_of_12/test.log
/var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_7_of_12/test.log
/var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_4_of_12/test.log
/var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_5_of_12/test.log
/var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_2_of_12/test.log
/var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_9_of_12/test.log
/var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_8_of_12/test.log
/var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_6_of_12/test.log
/var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_11_of_12/test.log
/var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_3_of_12/test.log
/var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_12_of_12/test.log
/var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_1_of_12/test.log
INFO: Found 1 test target...
Target //score/mw/com/impl/bindings/lola/test:proxy_component_test up-to-date:
bazel-bin/score/mw/com/impl/bindings/lola/test/proxy_component_test
INFO: Elapsed time: 1.774s, Critical Path: 1.38s
INFO: 13 processes: 24 processwrapper-sandbox.
INFO: Build completed, 1 test FAILED, 13 total actions
//score/mw/com/impl/bindings/lola/test:proxy_component_test FAILED in 12 out of 12 in 1.3s
Stats over 12 runs: max = 1.3s, min = 0.7s, avg = 1.1s, dev = 0.2s
/var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_10_of_12/test.log
/var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_7_of_12/test.log
/var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_4_of_12/test.log
/var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_5_of_12/test.log
/var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_2_of_12/test.log
/var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_9_of_12/test.log
/var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_8_of_12/test.log
/var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_6_of_12/test.log
/var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_11_of_12/test.log
/var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_3_of_12/test.log
/var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_12_of_12/test.log
/var/cache/bazel/d130cfea8d610fd5cbbe7bb32e05decb/execroot/_main/bazel-out/k8-fastbuild/testlogs/score/mw/com/impl/bindings/lola/test/proxy_component_test/run_1_of_12/test.log
Executed 1 out of 1 test: 1 fails locally.
in #39 the solution was to not run any test, which uses lola (/dev/shm, /tmp/mw_com_lola) in parallel. I did that because no other test was using the service id 52719 as used by the proxy_component_test and had no other idea how to make it reliable.
Ideally the tests are isolated from each other, but that is not the case when they do not mock lola.
Error Occurrence Rate
Highly Intermittent
How to reproduce
bazel test --nocache_test_results //...
or
bazel test //... --runs_per_test=10
The last command shows that more tests than proxy_component_test are affected.
//score/mw/com/impl/bindings/lola/service_discovery/client:service_discovery_client_test FAILED in 9 out of 10 in 3.7s
...
//score/mw/com/impl/bindings/lola/test:proxy_component_test FAILED in 10 out of 10 in 1.7s
...
//score/mw/com/impl/bindings/lola/test:skeleton_component_test FAILED in 9 out of 10 in 1.0s
...
//score/mw/com/impl/bindings/lola/test:skeleton_event_component_test FAILED in 7 out of 10 in 1.1s
...
//score/mw/com/impl/plumbing:skeleton_service_element_binding_factory_test FAILED in 9 out of 10 in 1.3s
Supporting Information
No response
Classification
minor
Affected Version
0.5
Expected Closure Version
0.5
Category
- Safety Related
- Security Related
Metadata
Metadata
Assignees
Labels
No labels