jUnit: Dynamic Tests Generation

Dynamic tests generation is useful when you need to run the same set of tests on many different input values or configurations. It can be achieved either using parametrized tests or using theories.

Theories are valuable when you have a bunch of data to be used as parameters and want to run tests on all their combinations. You get less control, but you do not have to write combining and iterating code by yourself. Basics about how theories work are explained on java code geeks (original at java advent calendar), so this post focus on parametrized tests.

Parametrized tests are better when you need to have good control over the input values, e.g. directory with files that are served as an input or a list of meaningful parameters combinations.

Parametrized Tests

Parametrized test is a test case able to accept parameters and a list of all parameter combinations you want it to run at. JUnit goes through the list of parameters, initializes the test case with each of them and then runs all its test methods.

Both GUI and Maven runners then interpret each parametrized test run as a separate test. If some of them fails, it is immediately clear which did failed and how many of them failed.

Example Use Case

Less4j is less to css compiler, so each of its tests is defined by an input less file and an expected css file. The compiler is run on input file and its output is compared to the expected css. If they match, test is passed.

All .less files are stored in a directory. Parametrized test case reads that directory and creates one jUnit test for each file. Therefore we can add new tests just by creating new .less and .css, run tests via "run all" button and see new test in all reports.

How to Use It

Parametrized test case must have following things:

• a @RunWith(Parameterized.class) class annotation,
• a constructor that accepts test case parameters,
• a static method annotated with @Parameters to generate parameters,
• test methods that runs on parameters supplied in constructor.

Constructor

Parametrized constructor must have at least one parameter. For example, the compiler test case can take input less as a first argument and expected compiled css as second argument. The third argument name is ignored and will be explained later:

@RunWith(Parameterized.class)
public class ParametrizedTest {

  public ParametrizedTest(String less, String expectedCss, String name) {
    this.less = less;
    this.expectedCss = expectedCss;
  }

}

Parameters

The static method generating parameters must return an implementation of the Iterable interface. The iterator returns arrays containing sets of parameters. Each array is used to create one test case instance and objects in it are used as constructor parameters.

For example, following method returns two arrays and thus leads to two test case instances:

@Parameters(name="Name: {2}")
public static Iterable<Object[]> generateParameters() {
  List<Object[]> result = new ArrayList<Object[]>();
  result.add(new Object[] {"less", "css", "pass"});
  result.add(new Object[] {"less", "error", "fail"});
  return result;
}

The name annotation parameter is optional. Its value will be shown in GUI or maven report as the test case name. The {n} is placeholder for n-th array value. They are indexed from 0, so the first test case will be named Name: pass and second test case will be named Name: fail.

Test Methods

Parametrized test case can have any number of tests and they must be annotated with @Test annotation:

@Test
public void testCss() { //dummy test method
  String actualCss = compile(less);
  assertEquals(expectedCss, actualCss);
}

@Test
public void testSourceMap() { //another test method
  String actualCss = compile(less);
  assertEquals(expectedCss, actualCss);
}

private String compile(String less) { //dummy compile method
  return "css"; 
}  

Output

If you run the above test class, the JUnit view will show following structure:

[F] com.github.sommeri.jUnit4Examples.ParametrizedTest
[ ] |-- [Name: pass]
[ ] |---------------- testCss[Name: pass] 
[ ] |---------------- testSourceMap[Name: pass] 
[F] |-- [Name: fail]
[F] |---------------- testCss[Name: fail] 
[F] |---------------- testSourceMap[Name: fail] 

Full Test Case

@RunWith(Parameterized.class)
public class ParametrizedTest {
  
  private String less;
  private String expectedCss;

  public ParametrizedTest(String less, String expectedCss, String name) {
    this.less = less;
    this.expectedCss = expectedCss;
  }
  
  @Parameters(name="Name: {2}")
  public static Iterable<Object[]> generateParameters() {
    List<Object[]> result = new ArrayList<Object[]>();
    result.add(new Object[] {"less", "css", "pass"});
    result.add(new Object[] {"less", "error", "fail"});
    return result;
  }
  
  @Test
  public void testCss() {
    String actualCss = compile(less);
    assertEquals(expectedCss, actualCss);
  }

  @Test
  public void testSourceMap() {
    String actualCss = compile(less);
    assertEquals(expectedCss, actualCss);
  }

  //dummy compile method
  private String compile(String less) {
    return "css"; 
  }  
}

More About jUnit

Following post about jUnit 4 features:

• jUnit: Rules