Fixes TealAppDriver & implement regression tests #1559

averissimo · 2025-07-14T11:19:13Z

Pull Request

Fixes #1535

Changes description

Remove testthat::skip("chromium") from e2e tests
Ensure all e2e pass
Check that "is.null()" tests are not looking for removed HTML elements from UI/UX refactor
Add regression tests:
- Sidebar visibility
- Modal popping (already covered in existing tests)
- Position of filter icons once collapsed

…null/not null

…id will be variable)

github-actions · 2025-07-17T08:08:27Z

Unit Tests Summary

1 files 26 suites 1m 58s ⏱️
271 tests 217 ✅ 54 💤 0 ❌
460 runs 405 ✅ 55 💤 0 ❌

Results for commit 513347f.

♻️ This comment has been updated with latest results.

github-actions · 2025-07-17T08:08:30Z

Unit Test Performance Difference

Test Suite	$Status$	Time on `main`	$±Time$	$±Tests$	$±Skipped$	$±Failures$	$±Errors$
module_session_info	💔	$17.03$	$+1.04$	$0$	$0$	$0$	$0$
module_teal	💚	$98.04$	$-4.11$	$0$	$0$	$0$	$0$

Additional test case details

Test Suite	$Status$	Time on `main`	$±Time$	Test Case
module_session_info	💔	$17.01$	$+1.04$	creation_process_is_invoked_for_teal.lockfile.mode_enabled_and_snapshot_is_copied_to_teal_app.lock_and_removed_after_session_ended
module_teal	💚	$3.35$	$-1.04$	appends_new_slice_and_activates_in_global_filters_when_added_in_a_module_if_module_specific
shinytest2-data_summary	👶		$+0.07$	e2e_data_summary_UI_can_be_collpased_and_expanded_bslib_regression_
shinytest2-filter_panel	👶		$+0.07$	e2e_filter_panel_UI_can_be_collpased_and_expanded_bslib_regression_
shinytest2-filter_panel	💀	$0.01$	$-0.01$	e2e_filtering_a_module_specific_filter_is_not_refected_in_other_unshared_modules
shinytest2-filter_panel	👶		$+0.10$	e2e_filtering_a_module_specific_filter_is_not_reflected_in_other_unshared_modules
shinytest2-modules	💀	$0.01$	$-0.01$	e2e_all_the_nested_teal_modules_are_initiated_as_expected
shinytest2-modules	👶		$+0.08$	e2e_nested_modules_layout_in_navigation_respect_order_and_keeps_group_names
shinytest2-show-rcode	👶		$+0.08$	e2e_Module_with_Show_R_Code_has_code
shinytest2-show-rcode	👶		$+0.08$	e2e_Module_with_Show_R_Code_has_modal_with_dismiss_and_copy_to_clipboard_buttons
shinytest2-show-rcode	👶		$+0.10$	e2e_Module_with_Show_R_Code_initializes_with_visible_button
shinytest2-show-rcode	💀	$0.02$	$-0.02$	e2e_teal_app_initializes_with_Show_R_Code_modal
shinytest2-teal_data_module	👶		$+0.06$	e2e_datasets_from_teal_data_module_show_in_filter_panel
shinytest2-teal_data_module	💀	$0.02$	$-0.02$	e2e_teal_data_module_will_have_a_delayed_load_of_datasets
shinytest2-teal_data_module	💀	$0.01$	$-0.01$	e2e_teal_data_module_will_make_other_tabs_inactive_before_successful_data_load
shinytest2-wunder_bar	👶		$+0.07$	e2e_collapsing_wunderbar_hides_filter_and_data_summary
shinytest2-wunder_bar	👶		$+0.09$	e2e_wunder_bar_srv_clicking_filter_icon_opens_filter_manager_modal
shinytest2-wunder_bar	👶		$+0.08$	e2e_wunder_bar_srv_clicking_snapshot_icon_opens_snapshot_manager_modal
shinytest2-wunder_bar	👶		$+0.07$	e2e_wunderbar_filter_and_data_summary_are_hidden_on_narrow_windows
shinytest2-wunder_bar	💀	$0.01$	$-0.01$	wunder_bar_srv_clicking_filter_icon_opens_filter_manager_modal
shinytest2-wunder_bar	💀	$0.01$	$-0.01$	wunder_bar_srv_clicking_snapshot_icon_opens_snapshot_manager_modal

Results for commit 1583ce1

♻️ This comment has been updated with latest results.

github-actions · 2025-07-17T08:09:20Z

Code Coverage Summary

Filename                          Stmts    Miss  Cover    Missing
------------------------------  -------  ------  -------  ----------------------------------------------------------------------------------------------------------------------------------------
R/checkmate.R                        24       0  100.00%
R/dummy_functions.R                  67      11  83.58%   46, 48, 90-98
R/include_css_js.R                   11      11  0.00%    12-37
R/init.R                            142      96  32.39%   137-158, 188-196, 206-231, 234-235, 242-251, 254-263, 266-275, 279-289, 291
R/landing_popup_module.R             34      34  0.00%    22-57
R/module_bookmark_manager.R         161     130  19.25%   47-68, 88-142, 147-148, 160, 207, 242-319
R/module_data_summary.R             177      10  94.35%   25-26, 40, 50, 205, 236-240
R/module_filter_data.R               64       2  96.88%   22-23
R/module_filter_manager.R           230      57  75.22%   56-62, 73-82, 90-95, 108-112, 117-118, 291-314, 340, 367, 379, 386-387
R/module_init_data.R                 74       6  91.89%   38-43
R/module_nested_tabs.R              366     215  41.26%   72-111, 138-323, 355, 473-476, 480-483, 487-490, 505, 539, 593-596
R/module_session_info.R              18       7  61.11%   35-41
R/module_snapshot_manager.R         216     146  32.41%   89-95, 104-113, 121-133, 152-153, 170-180, 184-199, 201-208, 215-230, 234-238, 240-246, 249-262, 265-273, 303-317, 320-331, 334-340, 354
R/module_teal_data.R                149      76  48.99%   43-149
R/module_teal_lockfile.R            131      69  47.33%   33-37, 45-57, 60-62, 76, 86-88, 92-96, 100-102, 110-119, 122, 124, 126-127, 142-146, 161-162, 177-186, 196-201
R/module_teal_with_splash.R          33      33  0.00%    24-61
R/module_teal.R                     163      57  65.03%   50-85, 103, 115-116, 157, 178-194, 196
R/module_transform_data.R           116       7  93.97%   20, 46, 130-134
R/modules.R                         297      53  82.15%   174-178, 233-236, 360-380, 388, 394, 571-577, 590-598, 613-628, 674, 687
R/reporter_previewer_module.R        20       2  90.00%   30, 34
R/show_rcode_modal.R                 31      31  0.00%    17-49
R/tdata.R                            14      14  0.00%    19-61
R/teal_data_module-eval_code.R       24       0  100.00%
R/teal_data_module-within.R           7       0  100.00%
R/teal_data_module.R                 20       0  100.00%
R/teal_data_utils.R                  10       0  100.00%
R/teal_modifiers.R                   71      71  0.00%    26-214
R/teal_reporter.R                    70       8  88.57%   69, 77-82, 131-132, 135, 152
R/teal_slices-store.R                29       0  100.00%
R/teal_slices.R                      63       0  100.00%
R/teal_transform_module.R            45       0  100.00%
R/TealAppDriver.R                   381     381  0.00%    57-762
R/utils.R                           250      38  84.80%   400-449
R/validate_inputs.R                  32       0  100.00%
R/validations.R                      58      37  36.21%   118-406
R/zzz.R                              15      11  26.67%   4-18
TOTAL                              3613    1613  55.36%

Diff against main

Filename                         Stmts    Miss  Cover
-----------------------------  -------  ------  --------
R/reporter_previewer_module.R       +1       0  +0.53%
R/TealAppDriver.R                   -4      -4  +100.00%
TOTAL                               -3      -4  +0.07%

Results for commit: 513347f

Minimum allowed coverage is 80%

♻️ This comment has been updated with latest results

llrs-roche

I see many changes to make code shorter and more readable. Thanks!
I have some questions about some tests changes but tests look good.

Some tests failed locally on windows. It can be related to Windows vs Linux or some update on chrome (I had to update it just today). See comments.

I see three tests skipped:

test-module_teal.R:1614:7: what happens when module$label is duplicated (when nested modules)
test-module_teal.R:1621:5: srv_teal data reload sets back the same active filters in each module
test-module_teal.R:1624:5: srv_teal data reload doesn't fail when teal_data has no datasets

Except the first that might require a PR to merge, I think the ones about srv_teal data reload could be tested already.

I haven't check if the new tests cover well any possible regression, I'll check that on Monday.

llrs-roche · 2025-07-18T13:23:17Z

tests/testthat/test-shinytest2-data_summary.R

+    teal.data::join_key("mtcars2", "mtcars1", keys = c("am"))
+  )
+
+  app <- TealAppDriver$new(data = data, modules = example_module())


Not sure what is happening, but I'm seeing a warning locally:

Warning (test-shinytest2-data_summary.R:99:3): e2e: data summary table does not list unsupported objects 'package:MultiAssayExperiment' may not be available when loading

llrs-roche · 2025-07-18T13:24:42Z

tests/testthat/test-shinytest2-init.R

  skip_if_too_deep(5)
  app <- TealAppDriver$new(
    data = simple_teal_data(),
    modules = example_module(label = "Example Module")
  )
  app$expect_no_shiny_error()
+  app$expect_screenshot(selector = "#teal-tabpanel_wrapper")


Locally I got a warning about missing windows snapshot (Not sure how important this is)

Warning (test-shinytest2-init.R:11:3): e2e: teal app initializes with no errors Adding new file snapshot: 'tests/testthat/_snaps/windows-4.5/teal-001.png'

llrs-roche · 2025-07-18T13:28:29Z

tests/testthat/test-shinytest2-filter_panel.R

+    ".teal-filter-panel [id$=\\'filters-main_filter_accordian\\']",
+    "> .accordion-item > .accordion-collapse"
+  )
+  testthat::expect_true(app$is_visible(filterpanel_accordion_selector))
+  app$click(selector = paste(
+    ".teal-filter-panel [id$='filters-main_filter_accordian']",


I fixed the typo accordian -> accordion in teal.slice regression tests but if you use an id there is no need to specify the class of the element as ids are unique. Start the selector with #filters-main_filter_accordion (if insightsengineering/teal.slice#653 gets merged before this PR)

llrs-roche · 2025-07-18T13:33:01Z

tests/testthat/test-shinytest2-module_bookmark_manager.R

@@ -56,6 +49,7 @@ testthat::test_that("bookmark_manager_button shows modal with url containing sta

  testthat::expect_match(
    rvest::html_text(app$get_html_rvest("div[id$=bookmark_modal] pre")),
-    "_state_id_"
+    "_state_id_=[a-zA-Z0-9]+",


This seems a bit weird. Is this the pattern for ids? Shouldn't then the regex include - and _? If it is for ids of elements it might accept other characters like . or space

llrs-roche · 2025-07-18T13:35:21Z

tests/testthat/test-shinytest2-reporter.R

  app$navigate_teal_tab("Report previewer")
-
  accordian_selector <- sprintf("#%s-pcards .accordion-toggle", app$active_module_ns())
  app$click(selector = accordian_selector)


Her locally it FAILS I'm using the main branch of the dependencies for testing this:

Error (test-shinytest2-reporter.R:60:3): e2e: adding a report card in a module adds it in the report previewer tab Error in `app_find_node_id(self, private, input = input, output = output, selector = selector)`: Cannot find HTML element with selector #teal-teal_modules-nav-temporary_label-module-pcards .accordion-toggle

llrs-roche · 2025-07-18T13:45:43Z

tests/testthat/test-shinytest2-teal_data_module.R

-      )
-    )
-  )
+  app$click(selector = "#teal-close_teal_data_module_modal button[data-dismiss='modal']")


Not tested but if you use the id why select with attributes properties too?

Suggested change

app$click(selector = "#teal-close_teal_data_module_modal button[data-dismiss='modal']")

app$click(selector = "#teal-close_teal_data_module_modal")

llrs-roche · 2025-07-18T13:51:47Z

tests/testthat/test-shinytest2-wunder_bar.R

+  testthat::expect_false(app$is_visible(".teal-filter-panel"))
+  testthat::expect_false(app$is_visible(".teal-active-data-summary-panel"))


I think it is best to test also that previously the panel(s) were visible. The teal filter panel could disappear and we wouldn't notice.

llrs-roche · 2025-07-18T14:18:00Z

R/reporter_previewer_module.R

@@ -43,6 +43,7 @@ reporter_previewer_module <- function(label = "Report previewer", server_args =
  # This is to prevent another module being labeled "Report previewer".
  class(module) <- c(class(module), "teal_module_previewer")
  module$label <- label
+  module$path <- label


Not sure what this change does, the commit message does not provide any clue and I find surprising that the same information is on two parts of the same object. Should label be in just one place?

llrs-roche · 2025-07-18T14:20:20Z

tests/testthat/test-module_teal.R

@@ -466,7 +466,7 @@ testthat::describe("srv_teal teal_modules", {
          `teal_modules-active_module_id` = "module_1"
        )
        out <- modules_output$module_1()
-        testthat::expect_true(!is.null(out))
+        testthat::expect_type(out, "double")


What is this magic number and how it is related to "srv_teal teal_modules are not called again when data changes"? All the previous tests expect it to be NULL?

llrs-roche · 2025-07-18T14:27:57Z

tests/testthat/helper-shinytest2.R

@@ -35,3 +35,26 @@ report_module <- function(label = "example teal module") {
    }
  )
 }
+
+example_teal_data_module <- function(with_submit = FALSE, once = TRUE) {


I only see it being used on teal_data_module did you want to reuse it in other tests too?

averissimo added 5 commits July 14, 2025 12:14

fix: namespace in TealAppDriver

78b6937

feat: shinytest2-teal_slices passes

5c2547c

feat: shinytest2-data_summary passes

fdd3839

feat: shinytest2-decorators passes

2785557

feat: shinytest2-init passes

8ba4976

averissimo force-pushed the 1535-regression branch from a4d85aa to 8ba4976 Compare July 14, 2025 11:55

averissimo added 3 commits July 14, 2025 13:56

feat: shinytest2-filter_panel passes

cce26d9

e2e: landing_popup passes

9d180ad

e2e: bookmark_manager passes

e6c40a9

averissimo mentioned this pull request Jul 15, 2025

Redesign and change position of reporter options #1562

Open

averissimo added 9 commits July 15, 2025 10:47

e2e: reporter passes

e7690e8

e2e: show-rcode passes

37fac62

e2e: teal_data_module passes

e3fc4ba

e2e: wunder_bar passes

f76a74f

e2e: decorators passes

b9c5f33

e2e: decorators passes (2)

0080615

feat: initial bslib regression tests on visibility of sidebar

90b41bd

chore: fix lintr

195962f

feat: take screenshot of initial run

b696e1e

averissimo changed the title ~~WIP: Fixes TealAppDriver & implement regression tests~~ Fixes TealAppDriver & implement regression tests Jul 16, 2025

averissimo added 5 commits July 16, 2025 12:25

feat: be more explicit when testing for not null

77e4ccf

chore: standardize with app() and use same selector when looking for …

d2f020c

…null/not null

feat: expect screenshot for ui (with the exception of footer as the p…

beba32b

…id will be variable)

feat: include snaps

a300b4f

Merge branch 'main' into 1535-regression

d702108

averissimo marked this pull request as ready for review July 17, 2025 07:56

github-actions bot and others added 2 commits July 17, 2025 07:58

[skip style] [skip vbump] Restyle files

ffc5327

empty: trigger ci

513347f

averissimo added the core label Jul 17, 2025

llrs-roche reviewed Jul 18, 2025

View reviewed changes

llrs-roche self-assigned this Jul 18, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Fixes TealAppDriver & implement regression tests #1559

Fixes TealAppDriver & implement regression tests #1559

Uh oh!

averissimo commented Jul 14, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Jul 17, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Jul 17, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Jul 17, 2025

Uh oh!

llrs-roche left a comment

Uh oh!

llrs-roche Jul 18, 2025

Uh oh!

llrs-roche Jul 18, 2025

Uh oh!

llrs-roche Jul 18, 2025

Uh oh!

llrs-roche Jul 18, 2025

Uh oh!

llrs-roche Jul 18, 2025

Uh oh!

llrs-roche Jul 18, 2025

Uh oh!

llrs-roche Jul 18, 2025

Uh oh!

llrs-roche Jul 18, 2025

Uh oh!

llrs-roche Jul 18, 2025

Uh oh!

llrs-roche Jul 18, 2025

Uh oh!

Uh oh!

	app$click(selector = "#teal-close_teal_data_module_modal button[data-dismiss='modal']")
	app$click(selector = "#teal-close_teal_data_module_modal")

		testthat::expect_false(app$is_visible(".teal-filter-panel"))
		testthat::expect_false(app$is_visible(".teal-active-data-summary-panel"))

Uh oh!

Fixes TealAppDriver & implement regression tests #1559

Are you sure you want to change the base?

Fixes TealAppDriver & implement regression tests #1559

Uh oh!

Conversation

averissimo commented Jul 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request

Changes description

Uh oh!

github-actions bot commented Jul 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Unit Tests Summary

Uh oh!

github-actions bot commented Jul 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Unit Test Performance Difference

Uh oh!

github-actions bot commented Jul 17, 2025

Code Coverage Summary

Diff against main

Uh oh!

llrs-roche left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

averissimo commented Jul 14, 2025 •

edited

Loading

github-actions bot commented Jul 17, 2025 •

edited

Loading

github-actions bot commented Jul 17, 2025 •

edited

Loading