Script-Free Testing on OpenNMS

This project integrates the script-free GUI-testing technique introduced by Kirinuki et al. (ICSME 2021 & SANER 2022). Instead of brittle, locator-based Selenium code, each test step is expressed in plain English (e.g., click "Login"), and the framework uses NLP and heuristic search to identify the correct element at runtime.

We evaluate this approach on the OpenNMS platform.

---

Objectives

• Lower maintenance costs by eliminating explicit locators and reducing test flakiness after OpenNMS UI changes.
• Assess robustness of the script-free approach on a large, evolving open-source system.
• Provide a reproducible benchmark so others can compare against traditional locator-repair techniques.

---

Getting Started

1 · Prerequisites

Tool	Recommended Version	Notes
Java	≥ 11	Required for OpenNMS server/runtime.
Maven	≥ 3.8	Build tool for OpenNMS.
Docker	≥ 20.10	Used for selenium and postgressql
Python	≥ 3.9	Needed for the script-free test runner.

---

Running OpenNMS Tests

1 · Setup the OpenNMS Server

./setup_opennms_server.sh

Installs and starts a local OpenNMS instance. Check whether the app works.

---

2 · Run Normal Tests

./run_normal_test.sh

---

3 · Run Script-Free Tests

./run_scriptfree_test.sh

---

Example Branch Pairs for Locator-Change Experiments

Before app change (baseline)	After app change, before locator change	After app change and locator change	Impacted files/tests (examples)
before_app_change_bf7c14584db6c7be9c016b66ba3a2b973565ae40	after_app_change_before_locator_change_bf7c14584db6c7be9c016b66ba3a2b973565ae40	after_app_change_and_locator_change_bf7c14584db6c7be9c016b66ba3a2b973565ae40	smoke-test/src/test/java/org/opennms/smoketest/EventsPageIT.java

Descriptions

• Before app change
  Snapshot of OpenNMS before the UI changes that cause locator failures.
  Significance: Run script-free tests here to assess baseline robustness.

• After app change and before locator change
  Snapshot of OpenNMS with the UI changes but without locator fixes in the E2E tests.
  Significance: Confirms that failures in normal tests are due to locator changes.

• After app change and locator change
  Snapshot of OpenNMS with the UI changes and updated locators in the E2E tests.
  Significance: Both the normal E2E tests and the script-free tests should pass.

---

Example Experiment Flow

1. Verify failures on the “after app change, before locator change” branch

   git checkout after_app_change_before_locator_change_bf7c14584db6c7be9c016b66ba3a2b973565ae40
   ./setup_opennms_server.sh # stop with ctrl+C after the app runs on localhost:8980/opennms
   ./run_normal_test.sh

   Expected result: Normal E2E tests fail due to elements not being located.

2. Run script-free on the “before app change” branch

   git checkout before_app_change_bf7c14584db6c7be9c016b66ba3a2b973565ae40
   ./setup_opennms_server.sh # stop with ctrl+C after the app runs on localhost:8980/opennms
   ./run_scriptfree_test.sh

   Expected result: Script-free tests succeed (baseline).

3. Confirm success on the “after app change and locator change” branch

   git checkout after_app_change_and_locator_change_bf7c14584db6c7be9c016b66ba3a2b973565ae40
   ./setup_opennms_server.sh # stop with ctrl+C after the app runs on localhost:8980/opennms
   ./run_normal_test.sh
   ./run_scriptfree_test.sh

   Expected result: Both normal E2E tests and script-free tests succeed.

Further Reading

• Kirinuki et al. “NLP-Assisted Web Element Identification Toward Script-Free Testing” — ICSME 2021.
• Kirinuki et al. “Web Element Identification by Combining NLP and Heuristic Search for Web Testing” — SANER 2022.
• OpenNMS project: https://github.com/OpenNMS/opennms