8319555: [TestBug] Utility for creating instruction window for manual tests

[andy-goryachev-oracle]

Provides the base class for manual tests which displays the test instructions, the UI under test, and the Pass/Fail buttons.

Uses EmojiTest to illustrate the use of the new class.

Example:

public class ManualTestExample extends ManualTestWindow {
    public ManualTestExample() {
        super(
            "Manual Test Example",
            """
            Instructions:
            1. you will see a button named "Test"
            2. press the button
            3. verify that the button can be pressed""",
            400, 250
        );
     }

     public static void main(String[] args) throws Exception {
         launch(args);
     }

     @Override
     protected Node createContent() {
         return new Button("Test");
     }
}

---

Progress

• Change must not contain extraneous whitespace
• Commit message must refer to an issue
• Change must be properly reviewed (2 reviews required, with at least 1 Reviewer, 1 Author)

Warning

 ⚠️ Patch contains a binary file (tests/manual/util/doc/ManualTestWindow.png)

Issue

• JDK-8319555: [TestBug] Utility for creating instruction window for manual tests (Enhancement - P4)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jfx.git pull/1747/head:pull/1747
$ git checkout pull/1747

Update a local copy of the PR:
$ git checkout pull/1747
$ git pull https://git.openjdk.org/jfx.git pull/1747/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 1747

View PR using the GUI difftool:
$ git pr show -t 1747

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jfx/pull/1747.diff

Using Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back angorya! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

❗ This change is not yet ready to be integrated.
See the Progress checklist in the description for automated requirements.

[openjdk]

⚠️ @andy-goryachev-oracle This pull request contains merges that bring in commits not present in the target repository. Since this is not a "merge style" pull request, these changes will be squashed when this pull request in integrated. If this is your intention, then please ignore this message. If you want to preserve the commit structure, you must change the title of this pull request to Merge <project>:<branch> where <project> is the name of another project in the OpenJDK organization (for example Merge jdk:master).

[mlbridge]

Webrevs

• 00: Full (8f23a763)

[kevinrushforth]

Without using an IDE (which is the default use case for manual tests), how would someone compile and run these?

/reviewers 2

[openjdk]

@kevinrushforth
The total number of required reviews for this PR (including the jcheck configuration and the last /reviewers command) is now set to 2 (with at least 1 Reviewer, 1 Author).

[andy-goryachev-oracle]

Without using an IDE (which is the default use case for manual tests), how would someone compile and run these?

Frankly, I have no idea. We might create a special module which contains common test code and utilities, I suppose. Any suggestions will be greatly appreciated!

The following command line expectedly fails because it can't find com.oracle.util.testing.ManualTestWindow:

java --module-path=build/sdk/lib --add-modules=javafx.controls ./tests/manual/text/EmojiTest.java

[andy-goryachev-oracle]

On a related note, where is it documented how to run manual tests?

https://wiki.openjdk.org/display/OpenJFX/Building+OpenJFX#BuildingOpenJFX-Testing
says nothing, still references JDK9 and JDK10 (though the content might still be true), and a link called "several items to work around module export during build and testing" under "Adding new packages in a modular world" section leads to the beginning of the document:

https://wiki.openjdk.org/display/OpenJFX/Building+OpenJFX#BuildingOpenJFX-UnderstandingaJDK9Modularworldmixedinourdeveloperbuild

[kevinrushforth]

On a related note, where is it documented how to run manual tests?

It isn't documented anywhere. Nor are the tests wired up to the build. We have the following test sprint bug filed:

JDK-8296441: Build and run manual tests from gradle

We might want to do that one first before implementing this one. Without some way from gradle, ant, or a shell script, it will be difficult to run any of the tests that start using this utility.

[beldenfox]

The following command line expectedly fails because it can't find com.oracle.util.testing.ManualTestWindow:

java --module-path=build/sdk/lib --add-modules=javafx.controls ./tests/manual/text/EmojiTest.java

When I was working on my manual tests @tsayao provided a useful tip. If you change thelaunch(args) call in EmojiTest.java to launch(EmojiTest.class, args) it's easy to run it from the command line like so:

java @build/run.args tests/manual/text/EmojiTest.java

It would be nice if all the manual tests did this, it makes them a lot easier to interact with.

[andy-goryachev-oracle]

It would be nice if all the manual tests did this

Good idea. I might need some help with this though - this command line

java @build/testrun.args ./tests/manual/text/EmojiTest.java fails because it cannot find

./tests/manual/text/EmojiTest.java:32: error: package com.oracle.util.testing does not exist
import com.oracle.util.testing.ManualTestWindow;
                              ^
./tests/manual/text/EmojiTest.java:34: error: cannot find symbol
public class EmojiTest extends ManualTestWindow {
                               ^
  symbol: class ManualTestWindow
./tests/manual/text/EmojiTest.java:36: error: cannot find symbol
        launch(args);
        ^
  symbol:   method launch(String[])
  location: class EmojiTest
./tests/manual/text/EmojiTest.java:62: error: method does not override or implement a method from a supertype
    @Override
    ^
4 errors

I do have the ManualTestWindow.class in tests/manual/util/bin/ but I think it's the Eclipse build directory.

[beldenfox]

Good idea. I might need some help with this though - this command line

java @build/testrun.args ./tests/manual/text/EmojiTest.java fails because it cannot find

Sorry, I don't think this can be made to work. I've been able to get Java's single-file source code mode to compile classes from more than one file but I now realize the setup is too restrictive to work here and isn't actually documented.

[andy-goryachev-oracle]

I don't think this can be made to work.

The only other though I have is to build a special test-common module and include it in the --module-path in build/testrun.args.

[kevinrushforth]

I like the idea of a standard test utility, as long as there is some sort of build script or easy way to get at it. Probably not build/testrun.args, though, since is meant for unit tests and includes the shims classes, so isn't suitable for most manual tests where we typically want to stick to the public API.

We might want to wait until we implement at least part of JDK-8296441 is implemented.

[andy-goryachev-oracle]

ok, postponing until JDK-8296441 is done.