Saturday, August 30, 2014

jUnit: Rules

Rules add special handling around tests, test cases or test suites. They can do additional validations common for all tests in the class, concurrently run multiple test instances, set up resources before each test or test case and tear them down afterwards.

The rule gets complete control over what will done with the test method, test case or test suite it is applied to. Complete control means that the rule decides what to do before and after running it and how to deal with thrown exceptions.

First chapter shows how to use rules and second shows what build-in rules can do. The third chapter describes third party rules libraries I found and the last one explains how to create new rules.

Table Of Contents


Using Rules

This chapter shows how to declare and use rules inside a test case. Most rules can be applied to each test method separately, once to the whole test case or once to the whole test suite. Rules run separately for each test are called test rules and rules applied to the whole test case or suite are called class rules.

We will use temporary folder rule as an example, so first subchapter explains what it does. Second subchapter declares it as test rule and third one as class rule. Last subchapter shows how to access the folder from inside the tests.

Example Rule - Temporary Folder
Temporary folder rule creates new empty folder, runs test or test case and then deletes the folder. You can either specify where to create the new folder, or let it be created in system temporary file directory.

Temporary folder can be used as both test rule and class rule.

Declaring Test Rules
Test rules e.g., rules that run for each test method separately, have to be declared in public field annotated with @Rule annotation.

Declare test rule:
public class SomeTestCase {
  @Rule
  public TemporaryFolder folder = new TemporaryFolder();
}

The above folder rule creates new folder before every test method and destroys it afterwards. All tests are able to use that directory, but they are not able to share files through it. Since we used constructor with no parameters, the folder will be created in system temporary file directory.

Test rule does its work before methods annotated with @Before and after those annotated with @After. Therefore, they will have access to temporary folder too.

Declaring Class Rules
Class rules e.g., rules that run once for the whole test case or test suite, have to be declared in public static field and annotated with @ClassRule annotation.

Declare test case rule:
public class SomeTestCase {
  @ClassRule
  public static TemporaryFolder folder = new TemporaryFolder();
}

The above folder rule creates new folder before running the first test method and destroys it after the last one. All tests are able to use that directory and they are able to see files created be previously running tests.

Class rules are run before anything inside that class. E.g. methods annotated with @BeforeClass or @AfterClass will have access to temporary folder too. The rule runs before and after them.

Using Rules Inside Tests
Rules are classes as any other and tests are free to call their public methods and use their public fields. Those calls are used to add test specific configuration to the rule or read data out of it.

For example, temporary folder can be accessed using newFile, newFolder or getRoot methods. First two create new file or folder inside the temporary folder and the getRoot method returns temporary folder itself.

Create temporary file and folder:
@Test
public void test1() {
  // Create new folder inside temporary directory. Depending on how you 
  // declared the folder rule, the directory will be deleted either 
  // right after this test or when the last test in test case finishes.
  File file = folder.newFolder("folder");
}

@Test
public void test2() {
  // Create new file inside temporary folder. Depending on how you 
  // declared the folder rule, the file will be deleted either 
  // right after this test or when the last test in test case finishes.
  File file = folder.newFile("file.png");
}

Default Rules

JUnit comes with five directly useable rules: temporary folder, expected exception, timeout, error collector and test name. Temporary folder have been explained in previous chapter, so we will briefly explain only remaining four rules.

Expected Exception
Expected exception runs the test and catches any exception it throws. The rule is able to check whether the exception contains the right message, the right cause and whether it was thrown by the right line.

Expected exception has private constructor and must be initialized using static none method. Each exception throwing test has to configure expected exception parameters and then call the expect method of the rule. The rule fails if:
  • the test throws any exception before the expect method call,
  • the test does not throw an exception after the expect method call,
  • thrown exception does not have the right message, class or cause.

The last test line throws an exception. Expected exception rule is configured right before causing the exception:
@Rule
public ExpectedException thrown= ExpectedException.none();

@Test
public void testException() {
  // Any exception thrown here causes failure
  doTheStuff();
  // From now on, the rule expects NullPointerException exception
  // to be thrown. If the test finishes without exception or if it 
  // throws wrong one, the rule will fail.
  thrown.expect(NullPointerException.class);
  // We well check also message
  thrown.expectMessage("Expected Message.");

  // this line is supposed to throw exception
  theCodeThatThrowsTheException();
}

Bonus: the expected message method accepts also hamcrest matcher argument. That allows you to test the message prefix, suffix, whether it matches some regular expressions or anything else.

Timeout
The timeout rule can be used as both test rule and class rule. If it is declared as test rule, it applies the same timeout limit to each test in the class. If it is declared as class rule, it applies the timeout limit to the whole test case or test suite.

Error Collector
Error collector allows you to run multiple checks inside the test and then report all their failures at once after the test ends.

Expected-vs-actual value assertions are evaluated using the checkThat method exposed by the rule. It accepts hamcrest matcher as an argument and thus can be used to check anything.

Unexpected exceptions can be reported directly using addError(Throwable error) method. Alternatively, if you have an instance of Callable to be run, you can call it through checkSucceeds method which adds any thrown exception into errors list.

Test Name
Test name rule exposes test name inside the test. It might be useful when you need to create custom error reporting.

Third Party Rules Libraries

Rules are decoupled from the test class, so it is easy to write libraries of general purpose rules and share them between projects. This chapter describes three such libraries.

System rules is rules collection for testing code that uses java.lang.System. It is well documented, available in maven and released under Common Public License 1.0 (the same as jUnit). System rules allows you to easily:
  • test content of System.err and System.out,
  • simulate input in System.in,
  • configure system properties and revert their values back,
  • test System.exit() calls - whether it was called and what return value was,
  • customize java SecurityManager and revert it back.

A big set of useful rules is available on aisrael account on github. Its documentation is somewhat limited, but you can always look at the code. All rules are released under MIT license:

Another undocumented set of rules on github. I will not list them here, because their names are self-explanatory and they do not have specified license. Look at the rules directory to see their list.

Custom Rule

This chapter shows how to create new rules. They can be implemented from scratch by implementing the TestRule interface or by extending one of two convenience classes ExternalResource and Verifier available in jUnit.

We will create a new rule from scratch and then rewrite it using ExternalResource class.

New Rule
New rule ensures that all files created by tests are properly deleted after each test finishes its work. The tests themselves have only one responsibility: report all new files using the ensureRemoval(file) method exposed by the rule.

How to declare and use the DeleteFilesRule rule:
@Rule
public DeleteFilesRule toDelete = new DeleteFilesRule();

@Test
public void example() throws IOException {
  // output.css will be deleted whether the test passes, fails or throws an exception
  toDelete.ensureRemoval("output.css");
  // the compiler is configured to create output.css file
  compileFile("input.less");
  checkCorrectess("output.css");
}

From Scratch
Each rule, including class rules, must implement the @TestRule interface. The interface has exactly one method:
public interface TestRule {
  Statement apply(Statement base, Description description);
}

Our job is to take statement supplied in the base parameter and turn it into another statement. The statement represents a set of actions e.g., test, test case or test suite to be run. It might have already been modified by other declared rules and includes before and after test or class methods.

The second description parameter describes the input statement. It can tell test class name, test name, annotations placed on it, it knows whether we are dealing with test or test suite etc. We will not need it.

We need to create a new statement which will do three things:
  • Empty the list of files to be deleted.
  • Run underlying test, test case or test suite represented by the base parameter.
  • Delete all files reported by tests inside previously run statement.

The statement is a class with one abstract method:
public abstract class Statement {
  public abstract void evaluate() throws Throwable;
}

Since underlying statement can throw an exception, the code to delete all files must run from finally block:
public class DeleteFilesRule implements TestRule  {
  
  public Statement apply(final Statement base, final Description description) {
    return new Statement() {
      
      @Override
      public void evaluate() throws Throwable {
        emptyFilesList(); // clean the list of files
        try {
          base.evaluate(); // run underlying statement
        } finally {
          removeAll(); // delete all new files
        }
      }
    };
  }
}

Both referenced methods emptyFilesList and removeAll are declared outside of new statement, directly inside the DeleteFilesRule class:
public class DeleteFilesRule implements TestRule  {

  private List<File> toDelete;
  
  private void emptyFilesList() {
    toDelete = new ArrayList<File>();
  }

  private void removeAll() {
    for (File file : toDelete) {
      if (file.exists())
        file.delete();
    }
  }

  /* ... the apply method ... */
}

The last thing we need is a public method able to add files to be deleted:
public void ensureRemoval(String... filenames) {
  for (String filename : filenames) {
    toDelete.add(new File(filename));
  }
}

Full Class
Click to expand

public class DeleteFilesRule implements TestRule  {

  private List<File> toDelete;
  
  public void ensureRemoval(String... filenames) {
    for (String filename : filenames) {
      toDelete.add(new File(filename));
    }
  }
  private void emptyFilesList() {
    toDelete = new ArrayList<File>();
  }

  private void removeAll() {
    for (File file : toDelete) {
      if (file.exists())
        file.delete();
    }
  }

  public Statement apply(final Statement base, final Description description) {
    return new Statement() {
      
      @Override
      public void evaluate() throws Throwable {
        emptyFilesList(); // clean the list of files
        try {
          base.evaluate(); // run underlying statement
        } finally {
          removeAll(); // delete all new files
        }
      }
    };
  }
}

Extending Build-in Classes
JUnit contains two convenience classes ExternalResource and Verifier meant to simplify the above process even more.

External Resource
The ExternalResource helps when you need to do some kind of preprocessing and postprocessing around the underlying test statement. If you need preprocessing, override the before method. If you need postprocessing, override the after method. The after is called from finally block, so it will be run no matter what.

Our DeleteFilesRule could be rewritten like this:
public class DeleteFilesRule2 extends ExternalResource  {
  
  /* ... list, ensureRemoval and removeAll methods ... */

  @Override
  protected void before() throws Throwable {
    toDelete = new ArrayList<File>();
  }

  @Override
  protected void after() {
    removeAll();
  }

}

Verifier
The Verifier has only one method verify to override. That method runs after the wrapped test finished its work and only if it did not thrown an exception. As the name suggests, the verifier is good if you want to run additional checks after the test.

More About jUnit

Previous post about jUnit 4 features:

33 comments:

Anonymous said...

Great Post :) it would be very helpful if you provide steps to use Custom Rules in our tests

SEO BACKLINKS said...

SEE MORE ARTICLE IN HERE. CLICK IN HERE

Anonymous said...

Complete and excellent source for @Rule and @ClassRule. Thanks

harry said...


Thanks for sharing the information. It is very useful for my future. keep sharing


visit our web

prash p said...

Awesome information.
such an useful article.
thanks for posting.keep sharing.

Mika Farron said...

Banyak tempat judi yang bisa ditemukan, seperti yang terdapat di Macau, Las Vegas, Texas, Hongkong, dan berbagai negara lainnya. Pada jenis permainan judi slot mesin ini dalam taruhan darat tersebut berbentuk kotak, besar dan tinggi, bahkan juga terdapat sistem dalam perhitungan unik.
asikqq
http://dewaqqq.club/
http://sumoqq.today/
interqq
pionpoker
bandar ceme terbaik
betgratis
paito warna terlengkap
forum prediksi

Machine Learning Training in Bangalore | Machine Learning course - ICSS said...

This is amazing blog.Thank you for sharing. Python is very demanding course in this days. Indian Cyber Security Solutions is a vary large institute in bangalore.
https://indiancybersecuritysolutions.com/python-training-in-bangalore-advance-level/

jayapriya said...
This comment has been removed by the author.
gowsalya said...

This is such a good post. One of the best posts that I\'ve read in my whole life. I am so happy that you chose this day to give me this. Please, continue to give me such valuable posts. Cheers!
java training in Bangalore

writepaperfor.me discount code said...

I truly appreciated perusing your article. I discovered this as an instructive and intriguing post, so I think it is exceptionally valuable and learned. I might want to thank you for the exertion you have made in composing this article.

Anonymous said...

This is an awesome post.Really very informative and creative contents. These concept is a good way to enhance the knowledge.I like it and help me to development very well.Thank you for this brief explanation and very nice information.Well, got a good knowledge.
hadoop admin ceritification training

Chandra Sekhar Reddy said...

nice Post
"Pressure Vessel Design Course is one of the courses offered by Sanjary Academy in Hyderabad. We have offer professional
Engineering Course like Piping Design Course,QA / QC Course,document Controller course,pressure Vessel Design Course,
Welding Inspector Course, Quality Management Course, #Safety officer course."
Piping Design Course in India­
Piping Design Course in Hyderabad
QA / QC Course
QA / QC Course in india
QA / QC Course in Hyderabad
Document Controller course
Pressure Vessel Design Course
Welding Inspector Course
Quality Management Course
Quality Management Course in india
Safety officer course

Digital_seo said...

Thank you for sharing useful information. Keep sharing more post
Selenium Training in Bangalore |
Software Testing Training in Bangalore|
Java Selenium Training in Bangalore |
Selenium Training Institute in Bangalore |
Automation Testing Training in Bangalore

wood couter said...

These ways are very simple and very much useful, as a beginner level these helped me a lot thanks fore sharing these kinds of useful and knowledgeable information.outdoor game

venkateshj said...

Thanks for providing valuable information
Cucumber Training in Marathahalli
Cucumber Training in Bangalore
Java Selenium Automation Training in Bangalore
Selenium Training in Marathahalli
Manual testing training in bangalore
Software Testing Training in Bangalore
Selenium Software Training in Bangalore
Selenium Automation Training in Bangalore
Selenium Training Institutes in Bangalore
Selenium Training in Marathahalli
Best Selenium Automation Training in Bangalore
Selenium Training in Marathahalli
Best Selenium Training in Bangalore
Selenium Software Training in Bangalore
Selenium Training Institutes in Bangalore
Top 10 selenium training institutes in bangalore
Selenium Training in Bangalore
Software Testing Training in Bangalore
Java Selenium Training in Bangalor
Best Selenium Training in Bangalore
Best Selenium Training Institute in Bangalore

Gaius Guild said...

bentuk kasino online mungkin lebih disukai daripada karakter dibandingkan dengan dua lainnya. Sebagai contoh, pertandingan berbasis langsung telah menjadi modern yang mudah diperoleh 98toto

Unknown said...

As stated by Stanford Medical, It is indeed the SINGLE reason women in this country get to live 10 years more and weigh on average 19 KG lighter than we do.

(Just so you know, it is not related to genetics or some hard exercise and really, EVERYTHING to "how" they are eating.)

P.S, What I said is "HOW", not "what"...

Tap this link to discover if this easy quiz can help you decipher your true weight loss possibility

Shivaraj said...

I am glad to read that you come up with outstanding information that definitely allows me to share with others. Thank you for sharing this with us.

digital marketing coaching in hubli

Quickbooks Expert said...

Nice Blog !
One such issue is QuickBooks Payroll Error PS036. Due to this error, you'll not be able to work on your software. Thus, to fix these issues, call us at 1-855-977-7463 and get the best ways to troubleshoot QuickBooks queries.

QuickBooks Error said...

Nice & Informative Blog !
In case you are searching for the best technical services for QuickBooks, call us at QuickBooks Error 102 1-855-977-7463 and get impeccable technical services for QuickBooks. We make use of the best knowledge for solving your QuickBooks issues.

shakunthala said...

thanks for sharing this blog with us
https://be-practical.com/Advanced-full-stack-developers-course-in-bangalore.html
job guaranteed courses in bangalore

Creativeline said...

Nice sharing information. if you needs social media marketing, logo design, packaging design then visit my site show my portfolio. you watch latest design then visit my social media platform.

anjali said...

PHP String Contains
PHP array length
Parsing JSON in Javascript
jQuery Ajax serialize form data example
PHP random quote generator
PHP sanitize input for MySQL
Python program to sort words in alphabetical order
ASCII value in Python

alexa jone said...

This is amazing blog its really helpful for me .if you have any problem with quickbooks software then our team available 24 hours at

quickbooks customer service

welcome to softcrayons said...

Thanks for this amazing blog , it is very useful content for us
keep sharing this type of informtion if anyone is looking for the best training institute in nodia visit us.

Python Training Institute
data science training in noida
machine learning institute in noida
java training institute in noida
data science training in noida

casinositeone.JDS said...


Thanks for such a fantastic blog.

casinosite777top.JDS said...

Where else could anyone get that kind of info written in such a perfect way?

casinositeguidecom.JDS said...

I have a presentation that I am presently writhing on, and I have been on the look out for such great information.

safetotositepro.JDS said...

It’s really a nice and useful piece of information.

casinositerank said...

I want to to thank you for this good read!! I definitely enjoyed every little bit of it.

bacarasite said...


this is the best way to share the great article with everyone one, so that everyone can able to utilize this information

gwolf said...


I was impressed by your writing. Your writing is impressive. I want to write like you

totosafesite said...

Hi! this is often nice article you shared with great information.

Post a Comment