Establish automated tests on hardware #353
Description
We recently established labels tested_on_hw and no_hw_test_required for PRs. However, this should evolve to automation. There are several aspects to this:
- Test environment: Physical vs. virtual hardware: Tests can be run either on real reference hardware or virtual hardware
- Real hardware would need to be integrated into PR tests
- Virtual hardware is offered by publicly available services in the internet, e.g. Corellium
- Tests definition: To make it reproducible for developers, we need to define much more specifically which exact tests (pytest, CAN, Ethernet) are supposed to be run, how, and in which environment
- https://github.com/eclipse-openbsw/openbsw/blob/main/test/pyTest/README.md proposes to use a "hardware testing board" without further description
- Retesting: We expect several PRs to be open in parallel. Whenever one is merged, the others need to be tested again. Does this need to be optimized, and how?