Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Development: Playwright E2E client coverage #10466

Open
wants to merge 35 commits into
base: develop
Choose a base branch
from

Conversation

muradium
Copy link
Contributor

Checklist

General

Client

Motivation and Context

It's not possible to understand how many percentage of the system is covered by E2E tests.

Description

This PR adds client code coverage for Playwright E2E tests. It's implemented using Playwright's own coverage feature and monocart-reporter for creating lcov reports.
Coverage information is generated automatically after test execution is finished. On CI, we zip and export the lcov coverage data as .zip archive.

Steps for Testing

  • Code Review: Ensure that code quality is good and does not break the existing system.
  • Review Coverage Report:
  1. Check out artifacts section of build runs: Bamboo Artifacts
  2. Run E2E tests and check if coverage report is generated correctly. Coverage reports should be generated on src/test/playwright/test-reportsfolder after test run is finished if running on headless mode or after UI is quit if running on UI mode.

Steps for running the tests:

  1. Navigate to src/test/playwright
  2. Configure Playwright using playwright.env file based on your local setup. Current configuration should work for default Artemis setup.
  3. Run npm install && npm run playwright:setup
  4. Run the tests using one of the methods:
  • Run npm run playwright:test to run all tests
  • Run npm run playwright:open to open the Playwright UI and selectively run few tests

Testserver States

Note

These badges show the state of the test servers.
Green = Currently available, Red = Currently locked







Review Progress

Code Review

  • Code Review 1
  • Code Review 2

Manual Tests

  • Test 1
  • Test 2

muradium and others added 30 commits January 6, 2025 07:56
# Conflicts:
#	src/test/playwright/package-lock.json
#	src/test/playwright/package.json
#	src/test/playwright/playwright.config.ts
…ck if the problem is on coverage data itself on CI"

This reverts commit 67c12a9.
…esent in artemis-app container"

This reverts commit 12bcab1.
# Conflicts:
#	src/test/playwright/package-lock.json
#	src/test/playwright/package.json
Comment on lines +59 to +60
// Bamboo can upload only files as an artifact, not directories
// That's why we archive the lcov coverage directory on CI to prepare it as an artifact
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a quick note: since Bamboo will soon be removed, we’re actively migrating our E2E test runs to GitHub Actions. In a couple of weeks, the tests should be fully running here. After that, Helios will parse the final test results (results.xml). So, IMO there’s not much value in adding any new Bamboo-specific functionality.

From what I can see, you’re still generating results.xml as the final artifact in JUnit format, and that logic hasn’t changed—please correct me if I’m wrong.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Work In Progress
Development

Successfully merging this pull request may close these issues.

2 participants