Practice 11 ‐ Code Coverage

Code Coverage

Initialization

• Download the project https://github.com/ftsrg-edu/ase-labs.git.

• Switch to branch practice-2b-end.

• initiate Gradle build. Wait for indexing to finish.

Running tests

• Select a simple test ```logic/src/test/java/hu/bme/mit/ase/shingler/logic/TokenizerTests.java''' and select Right Click | Run TokenizerTest.

• Observe the result of the unit tests.

• Now select **More Run/Debug | Run TokenizerTest with Coverage. You will notice, that a new window will be available called "Coverage". Here, we can observe different coverage metrics.

• Check the code annotated with different coverages.

• Add a new test to improve the Branch Coverage of the Tokenizer class .

• You can run all the tests by Right click on the subproject (e.g., diversity or similarity) | More Run/Debug | Run 'All Tests' with Coverage. If a new coverage measurement would replace the results of the previous one, a new dialog window will appear asking for approval: Select Replace Active Suites.

Randoop

• Read the introduction and stages of test generation sections of Randoop's manual to get a quick overview about the tool.

• Download the latest version of randoop-all jar from the releases. As of now, it is randoop-all-4.3.3.jar.

• Set an environment variable pointing to the downloaded jar file.

  export RANDOOP_JAR=/home/.../randoop-all-4.3.3.jar
  

• Run the following command to print the help message for generating tests in order to check that everything is working so far.

  java -cp $RANDOOP_JAR randoop.main.Main help
  

Exercises

Generating tests

• Use the practice-2b-end branch of the ase-labs repository.

• Run the existing manually created tests:

 ./gradlew test

• Generate tests for the MyInteger class. Randoop needs the compiled class files, and you need to set the Java class path correctly, otherwise Randoop will not find it.
  • Navigate to the root folder of the project. The MyInteger.class is located inside the folder build/classes/java/main/math, and its fully qualified name is math.MyInteger.
  • Call Randoop to generate tests for MyInteger:
    • The : character is the separator in the class path (-cp). Note that the test class is given with its fully qualified name, and the .class extension is not needed.
    • The --testclass specifies the class to generate tests for.
    • The --junit-output-dir sets the folder where the generated tests are placed.
    • The --output-limit parameter sets the limit for the number of generated tests.

  java -cp build/classes/java/main:$RANDOOP_JAR randoop.main.Main gentests --testclass=math.MyInteger --junit-output-dir=src/test/java --output-limit=20

• Execute the generated tests (./gradlew test)

• Examine the generated tests! Randoop generated error-revealing tests (tests violating some general contract or some implicit test oracles) and regression tests (tests capturing the current behavior for some generated inputs).

• Investigate the error-revealing tests!

  • What are the test checking?
  • Find the problem in the implementation!

• See the fix in the source of the implementation and run the tests again to validate the fixes.

Changing the default parameters (1p)

Explore the functionality of Randoop by changing the parameters of the test generation. More information can be found in the detailed manual.

Some initial ideas:

• Change the limits (time, output). Is Randoop able to increase the coverage?
• Change the values used in tests (nulls, literals...).
• Change the classification of tests (e.g. whether exceptions are considered errors).
• Try to specify expected code behavior with pre- and post-conditions.