Add plotly diagram to lobster-html-report#372
Open
TannazVhdBMWExt wants to merge 129 commits intomainfrom
Open
Add plotly diagram to lobster-html-report#372TannazVhdBMWExt wants to merge 129 commits intomainfrom
TannazVhdBMWExt wants to merge 129 commits intomainfrom
Conversation
TannazVhdBMWExt
added
the
lobster-html-report
phiwuu
force-pushed
the
feature/lobster-html-report-using-graphviz-library-nv
branch
2 times, most recently
from
968efee to
494cff2
Compare
Replacing graphviz and dot with plotly.
TannazVhdBMWExt
force-pushed
the
feature/lobster-html-report-using-graphviz-library-nv
branch
from
4c46a5c to
b88cac8
Compare
Member
|
@TannazVhdBMWExt please rebase your changes. There are merge conflicts. |
kedarnn
requested changes
Nov 3, 2025
Comment on lines
+55
to
+66
| * `lobster-html-report`: | ||
| - The tracing policy diagram is always included in the HTML report. | ||
| The `dot` tool from the [Graphviz](https://graphviz.org) project is no longer | ||
| needed. | ||
| The python package `plotly` is required instead. | ||
| If `lobster-html-report` is installed with the help of `pip`, then `plotly` will be | ||
| installed automatically. | ||
|
|
||
| (For installation instructions, see the [Installing](README.md#installing) section | ||
| in the README.) | ||
|
|
||
|
|
Contributor
There was a problem hiding this comment.
Need to be moved to 1.0.3-dev
| @@ -0,0 +1,466 @@ | |||
| # pylint: disable=invalid-name | |||
Comment on lines
+326
to
+331
| # Plotly graph | ||
| doc.style[".js-plotly-plot"] = { | ||
| "z-index": "1", | ||
| "position": "relative" | ||
| } | ||
|
|
Contributor
There was a problem hiding this comment.
Please move this to tools/core/html_report/assets/html_report.css for better segregation of CSS from Python code.
Comment on lines
+603
to
+618
|
|
||
| document.querySelectorAll('.annotation').forEach(function(ann) { | ||
| ann.style.cursor = "pointer"; | ||
| ann.addEventListener('click', function(e) { | ||
| var textElem = ann.querySelector('text.annotation-text'); | ||
| if (textElem) { | ||
| var unformatted = textElem.getAttribute('data-unformatted'); | ||
| // Extract the section anchor from the <span section="..."> | ||
| var match = unformatted && unformatted.match(/section="([^"]+)"/); | ||
| if (match && match[1].startsWith("#sec-")) { | ||
| window.location.hash = match[1]; | ||
| e.preventDefault(); | ||
| } | ||
| } | ||
| }); | ||
| }); |
Contributor
There was a problem hiding this comment.
Please check the first comments for this code part.
Comment on lines
+640
to
+654
| document.querySelectorAll('.annotation').forEach(function(ann) { | ||
| ann.style.cursor = "pointer"; | ||
| ann.addEventListener('click', function(e) { | ||
| var textElem = ann.querySelector('text.annotation-text'); | ||
| if (textElem) { | ||
| var unformatted = textElem.getAttribute('data-unformatted'); | ||
| // Extract the section anchor from the <span section="..."> | ||
| var match = unformatted && unformatted.match(/section="([^"]+)"/); | ||
| if (match && match[1].startsWith("#sec-")) { | ||
| window.location.hash = match[1]; | ||
| e.preventDefault(); | ||
| } | ||
| } | ||
| }); | ||
| }); |
Contributor
There was a problem hiding this comment.
Please check the first comments for this code part.
|
|
||
|
|
||
| class LobsterUIAsserter(Asserter): | ||
| def assertOutputFiles(self): |
Contributor
There was a problem hiding this comment.
Instead of code duplication the existing function in asserter.py file can be modified.
Contributor
|
The tracing policy graph generated is somewhat distorted. Please see |
Replaced hypens with underscores for tests-system, tests-unit and tests-integration folders and relevant changes to their references in files. In the `test_runner.py` files need to replace the `underscores` with `hyphens` for `_tool_main_path` variable as this is used to execute the tool and the tool names contains `hyphens` instead of `underscores`. **This will be reverted back if we decide to rename the lobster tools with underscores.** Reason for change - Each folder should be importable as a Python module and hence we need to replace `-` with `_` in the folder names.
Add a document that gives inspiration how to qualify the LOBSTER tools.
Due to the replacement of `-` with `_` in `tests_system` & `tests-integration` the tracing target in Makefile was failing to generate the tracing reports. **Issue** The issue was that the `unit tests` and `system tests` targets were looking for folder like `tests-unit/lobster-tool-name` & `tests-system/lobster-tool-name` but instead of hyphens it should be underscores in the folders. This PR fixes that and the generation of the reports. **NOTE** This PR also deletes the system tests for `lobster-python` which will be upgraded to new STF and is tracked in [this ticket](https://jira.cc.bmwgroup.net/browse/SWF-20309).
The Flask codebeamer mock server is started on a separate thread during class setup of the `lobster-codebeamer` system tests. A race condition can occur where the first system test sends a request to the mock server before it has fully started up, causing test failures. To prevent this, we now send test requests to verify the mock server is responsive before proceeding with system tests. This ensures reliable test execution by eliminating the startup race condition. Additionally set log level to DEBUG in `SystemTestCaseBase`.
## PR Description ### Add Semantic HTML Identifiers and Refactor Headings/Classes for Better Accessibility and Styling #### Overview This PR introduces semantic improvements and enhanced flexibility to the HTML generation in the Lobster codebase. The changes primarily focus on: - Adding meaningful `id` and `class` attributes to HTML headings, links, table rows, and issue items. - Allowing more consistent, accessible, and styleable HTML output for both documentation and reports. - Maintaining backward compatibility for existing usages. #### Key Changes ##### 1. `lobster/html/htmldoc.py` - Enhanced the `add_heading` method: - New parameter `html_identifier` allows headings to receive consistent, semantic ids and classes. - Headings now use ids like `heading-<text>` or `sec-<anchor>` and classes like `heading-<text>` for better styling and navigation. - Improved anchor link generation: - Menu items now have ids based on their names for easier reference. ##### 2. `lobster/tools/core/html_report/html_report.py` - Updated the report generation to use the improved heading and link methods: - All major sections (e.g., Overview, Filtering, Issues, Detailed Report) now have semantic ids and/or classes. - Table rows and issue list items receive dynamic classnames based on their content for improved styling and filtering. - All new usages of `add_heading` take advantage of the `html_identifier` flag for consistency. - Wrapped detailed report sections in `<div>`s with semantic classnames for easier DOM targeting. - Miscellaneous improvements: - Ensured that generated HTML is easier to style and reference from JavaScript or CSS. #### Motivation - **Accessibility**: Improved ids and classes support better navigation and assistive technologies. - **Styling/JS hooks**: Semantic selectors make it easier to apply CSS or attach event handlers. - **Maintainability**: Consolidates and standardizes how headings and sections are created and referenced. #### Backward Compatibility - Existing functionality remains unchanged unless the new `html_identifier` flag is used.
Introduced API function:
- `generate_report_file`:
This is API function for the tool `lobster-report` which takes lobster
config file as input
and generates the lobster report.
This is similar to running the tool `lobster-report`.
- Removed assertions and raised exceptions
---------
Co-authored-by: Mugdha Dhole <[email protected]>
Co-authored-by: mugdhadhole1 <[email protected]>
Add a system test that verifies the behavior of `lobster-trlc` with respect to extended TRLC types. In the test input data two TRLC record types exist: - a base type - and a type which extends the base type Depending on the configuration of `lobster-trlc` either all TRLC record objects of both types shall be converted to LOBSTER objects, or only the extended type. Converting only the base type is not supported.
Removed the function `validate_and_parse_custom_data` for the reasons given below. **The overall motivation is to clean the code base and increase the branch coverage.** - A JSON validator based on a JSON schema should be used instead of implementing a validation logic for each key individually. - The only benefit so far is that the user recieves a readable message if the LOBSTER input file contains data that does not have an `items()` function. But using `str` as key or value in that dictionary is actually not mandatory. Anything that can be converted to `str` works equally fine. So the validation is too strict. - It is okay to remove this feature from the perspective of the tool qualification. If the user provides data which is not have iterable through an `items()` function, then the tool crashes with a Python exception. This is not a convenient user experience, agreed. But removing these lines helps to increase the branch coverage, and as said above a JSON validator should be used to generate helpful messages for the user. - No unit tests or system tests existed for the function. - Tools like `lobster-report` and `lobster-online-report` do not work with the data provided in `custom_data`. They just store it, and serialize it back into their output. Only `lobster-html-report` needs data that can be converted to `str`, so any validation should happen in that tool.
add main function for gtest
The goal is to remove the 'lobster-trlc.conf' besides the yaml config
file. The tool `lobster-trlc` shall need only one configuration file,
not two. Hence removed the config parameter `trlc_config_file` holding
the file name of the `lobster-trlc.conf` config file.
Add new config parameter `trlc_config` to specify the trlc configuration
directly inside the yaml config file using the block scalar style (|)
for multi line strings.
Example entry for the yaml config file:
```yaml
trlc_config: |
package.typename {
description = field_name
tags "test" = field_name
}
```
This improvement is not perfect, because it mixes two configuraiton
styles into one file. But at least the user has to create only one file,
not two.
Note that this is a breaking change for `lobster-trlc`!
Co-authored-by: Thomas Deboben <[email protected]>
Also improve the implementation of the system test framework helper script `update_online_json_with_hashes.py`. Previously it executed `git rev-parse HEAD` for each item in the JSON file. But the hash will always be the same, because our repository does not use any submodules. So `git` can be called once, and the result can be used for all items. Co-authored-by: kedarnn <[email protected]>
Tool creates one output file, dataclass for config file has been used.
`lobster-trlc` requires at least version 2.0.1, not 1.2.2
The tests still depended on files that contain the old variant of the `lobster-trlc` configuration. Furthermore the `make integration-tests` target always deleted the `MODULE.bazel` and `MODULE.bazel.lock` files, which contain real content since the beginning of the introduction of Bazel to the repository.
Co-authored-by: Mugdha Dhole <[email protected]>
- Breakdown of the use cases into potential errors, and further down into test specifications - Tracing of these then into system requirements, software requirements to test and code - The overall tool qualification status. - Tracing relation: - system test implementations > test specifications > potential errors > use cases - system test implementation > system requirements --------- Co-authored-by: Philipp Wullstein-Kammler <[email protected]> Co-authored-by: Suraj Deore <[email protected]> Co-authored-by: Kedar Navare <[email protected]> Co-authored-by: suraj <[email protected]> Co-authored-by: mugdhadhole1 <[email protected]> Co-authored-by: Mugdha Dhole <[email protected]>
This PR updates the asserter logic to work across the entire system test framework. Changes: - Added validation to raise an error if an output is not declared in the asserter - Adjusted system test cases accordingly Issue: SWF-18239 --------- Co-authored-by: suraj <[email protected]>
- Add test_online_report_input.py for testing HTML report generation with online report data - Add test data file with git hashes. - Add update_html_expected_output.py utility to synchronize expected HTML file git commit and timestamps with actual git commit and timestamps The new utility extracts commit hashes from HTML file tags and updates timestamps to match the actual git commit time. --------- Co-authored-by: Mugdha Dhole <[email protected]>
lobster-tools: API naming convention for all tool APIs Co-authored-by: suraj <[email protected]>
- Update commit_id 'main' in online report configurations - Fix missing online report configuration in STF tracing target - Break long printf commands into multiple lines for better readability This ensures all generated traceability reports reference the correct github URLs for consistent navigation and linking. Co-authored-by: Mugdha Dhole <[email protected]>
Add potential and test specification Add system tests for Codebeamer data with traces.
The tool `lobster-online-report` printed the name "lobster-lobster-online-report" instead of "lobster-online-report" when running the tool with `--help`. The message has been fixed.
- Rename Complex_Tracing_Policy_Data to Tracing_Policy_Data - test specification descriptions (requirement -> requirements) --------- Co-authored-by: Mugdha Dhole <[email protected]>
This PR is used to fix the issues toggle based on the `Item Filters` in HTML report.
Removed feature to write output to `stdout`, making the `--out` command line argument mandatory. This decision has been made to align the behavior of `lobster-cpp` with all other LOBSTER tools, and to reduce the complexity of the tool. This simplifies testing and tool qualification in the future.
Add py_test target for test_online_report_input.py to enable Bazel-based testing of HTML report online input functionality. Co-authored-by: Mugdha Dhole <[email protected]>
Bump version after release
The use case `List_Tests_without_Requirements` applies very well to the tool `lobster-pkg`. The previously given rationale was wrong.
Add lobster-trace for UseCases.Missing_tracing_policy_violation_in_output to multiple lobster-html-report system tests This improves traceability coverage for the lobster-html-report tool Co-authored-by: Mugdha Dhole <[email protected]>
Delete the `make` target `github-release`, because we are using `.github/workflows/package.yml` to package releases. The `make` target is not needed.
- Add pkg to TOOLS list in tracing.sh - Create TRLC configuration files for pkg traceability: - pkg.potential-errors.lobster-trlc.yaml - pkg.test-specifications.lobster-trlc.yaml - pkg.system-requirements.lobster-trlc.yaml - pkg.software-requirements.lobster-trlc.yaml - Skip pkg in Makefile tracing target (handled by tracing.sh) - Enable usecase tracing for pkg tool --------- Co-authored-by: Mugdha Dhole <[email protected]> Co-authored-by: Philipp Wullstein-Kammler <[email protected]>
- Add online_report to TOOLS list in tracing.sh - Create TRLC configuration files for online report traceability: - online_report.potential-errors.lobster-trlc.yaml - online_report.test-specifications.lobster-trlc.yaml - online_report.system-requirements.lobster-trlc.yaml - online_report.software-requirements.lobster-trlc.yaml - Skip core-online-report in Makefile tracing target (handled by tracing.sh) - Enable usecase tracing for online report tool Co-authored-by: Philipp Wullstein-Kammler <[email protected]>
Add a call to `unittest-main()` in the `__main__` block of system tests. When running tests with `python -m unittest discover` the unit test framework executes all tests rgardless whether `main()` is present. But when using a `py_test` in Bazel this is no longer true. Tests are only executed through `unittest.main()`. If this function is not called, then `py_test` returns SUCCESS, but in reality no test was executed. Hence it is crucial to call `unittest.main()` to support Bazel's `py_test`. Here we update most of the system tests which do not need a modification of their Bazel call. Also include `lobster-pkg` system tests in the lint target.
- Correct the data BUILD file and add the missing file formats
Add missing ssl files to launch lobster_codebeamer system tests with bazel test --------- Co-authored-by: Philipp Wullstein-Kammler <[email protected]>
- Fix lobster-trace tag case sensitivity: change "Usecases" to "UseCases" throughout test files - Remove invalid List_Findings references from JSON potential error specifications - Add HTML_report_not_Generated to HTML_file_generation test specification verification - Remove lobster_html_report from Show_Tracing_Policy usecase affected_tools - Add missing traceability tags to various test methods for better coverage - Clean up inconsistent formatting and improve requirement descriptions --------- Co-authored-by: Mugdha Dhole <[email protected]>
Co-authored-by: BTI-00085 <[email protected]@PUN1LPT000040.bmwtechworks.local>
requirement refs other requirement, version shall be traced. - For code logic: add `version-field` in the lobster-trlc.yaml - For basic integration: delete unused `trlc_config.yaml`, and combine to the `lobster-trlc.yaml` - Update basic integration with version attribute - Add unit tests - Add system tests Co-authored-by: Melody Ma <[email protected]>
Member
|
@ThomasDebobenVispiron there are 129 commits in this PR now. Can you please clean your history? I guess something went wrong when rebasing or merging the master branch. |
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replacing graphviz and dot with plotly.