Monday, January 26, 2015

Testing Grunt Plugin From Grunt

Writing tests for grunt plugin turned out to be less straightforward then expected. I needed to run multiple task configurations and wanted to invoke them all by typing grunt test in main directory.

Grunt normally exits after first task failure. That makes it impossible to store multiple failure scenarios inside the main project gruntfile. Running them from there would require the --force option, but grunt then ignores all warnings which is not optimal.

Cleaner solution is to have a bunch of gruntfiles in separate directory and invoke them all from the main project gruntfile. This post explains how to do that.

Table of Contents

Demo Project

Demo project is small grunt plugin with one grunt task. The task either fails with warning or prints success message into the console depending on the value of action options property.

The task:
grunt.registerMultiTask('plugin_tester', 'Demo grunt task.', function() {
  //merge supplied options with default options
  var options = this.options({ action: 'pass', message: 'unknown error'});

  //pass or fail - depending on configured options
  if (options.action==='pass') {
    grunt.log.writeln('Plugin worked correctly passed.');
  } else {
    grunt.warn('Plugin failed: ' + options.message);
There are three different ways how to write grunt plugin unit tests. Each solution has its own nodeunit file in test directory and is explained in this post:
All three demo tests consist of three different task configurations:
// Success scenario
options: { action: 'pass' }
// Fail with "complete failure" message
options: { action: 'fail', message: 'complete failure' }
//Fail with "partial failure" message
options: { action: 'fail', message: 'partial failure' }

Each configuration is stored in separate gruntfile inside test directory. For example, success scenario stored inside gruntfile-pass.js file looks like this:
  // prove that npm plugin works too
  jshint: { 
    all: [ 'gruntfile-pass.js' ] 
  // Configuration to be run (and then tested).
  plugin_tester: { 
    pass: { options: { action: 'pass' } } 

// Load this plugin's task(s).
// next line does not work - grunt requires locally installed plugins

grunt.registerTask('default', ['plugin_tester', 'jshint']);
All three test gruntfiles look almost the same, only the options object of plugin_tester target changes.

Running Gruntfile From Subdirectory

Our test gruntfiles are stored in test subdirectory and grunt does not handle such situation well. This chapter explains what the problem is and shows two ways how to solve it.

The Problem
To see the problem, go to the demo project directory and run following command:
grunt --gruntfile test/gruntfile-problem.js
Grunt responds with following error:
Local Npm module "grunt-contrib-jshint" not found. Is it installed?
Warning: Task "jshint" not found. Use --force to continue.

Aborted due to warnings.
Grunt assumes that grunfile and node_modules repository are stored in the same directory. While node.js require function searches all parent directories for required module, grunts loadNpmTasks does not.

This problem has two possible solutions, simple and fancy one:
  • create local npm repository in tests directory (simple),
  • make grunt load tasks from parent directories (fancy).

Although the first "simple" solution is somewhat cleaner, demo project uses second "fancy" solution.

Solution 1: Duplicate Npm Repository
The main idea is simple, just create another local npm repository inside the tests directory:
  • Copy package.json file into tests directory.
  • Add test only dependencies into it.
  • Run npm install command every time you run tests.

This is the cleaner solution. It has only two downsides:
  • test dependencies have to be maintained separately,
  • all plugin dependencies have to be installed in two places.

Solution 2: Load Grunt Tasks From Parent Directory
The other solution is to force grunt to load tasks from npm repository stored inside another directory.

Grunt Plugin Loading
Grunt has two methods able to load plugins:
  • loadTasks('directory-name') - loads all tasks inside a directory,
  • loadNpmTasks('plugin-name') - loads all tasks defined by a plugin.

The loadNpmTasks function assumes fixed directory structure of both grunt plugin and modules repository. It guess name of directory where tasks should be stored and then calls loadTasks('directory-name') function.

Local npm repository has separate subdirectory for each npm package. All grunt plugins are supposed to have tasks subdirectory and .js files inside it are assumed to contain tasks. For example, loadNpmTasks('grunt-contrib-jshint') call loads tasks from node_mudules/grunt-contrib-jshint/tasks directory and is equivalent to:
Therefore, if we want to load all tasks of grunt-contrib-jshint plugin from parent directory, we can do following:
Loop Parent Directories
More flexible solution is to climb through all parent directories until we find closest node_modules repository or reach root directory. This is implemented inside grunt-hacks.js module.

The loadParentNpmTasks function loops parent directories :
module.exports = new function() {

  this.loadParentNpmTasks = function(grunt, pluginName) {
    var oldDirectory='', climb='', directory, content;

    // search for the right directory
    directory = climb+'node_modules/'+ pluginName;
    while (continueClimbing(grunt, oldDirectory, directory)) {
      climb += '../';
      oldDirectory = directory;
      directory = climb+'node_modules/'+ pluginName;

    // load tasks or return an error
    if (grunt.file.exists(directory)) {
    } else {'Tasks plugin ' + pluginName + ' was not found.');

  function continueClimbing(grunt, oldDirectory, directory) {
    return !grunt.file.exists(directory) &&
      !grunt.file.arePathsEquivalent(oldDirectory, directory);

Modified Gruntfile
Finally, we need to replace the usual grunt.loadNpmTasks('grunt-contrib-jshint') call in the gruntfile by following:
var loader = require("./grunt-hacks.js");
loader.loadParentNpmTasks(grunt, 'grunt-contrib-jshint');

Shortened gruntfile:
module.exports = function(grunt) {
  var loader = require("./grunt-hacks.js");

    jshint: { /* ... */  },
    plugin_tester: { /* ... */ }

  loader.loadParentNpmTasks(grunt, 'grunt-contrib-jshint');

This solution has two disadvantages:
  • It does not deal with collection plugins.
  • If grunt ever changes expected structure of grunt plugins, you will have to modify the solution.

If you need collection plugins too, have a look at grunts task.js to see how to support them.

Calling Gruntfile From Javascript

Second thing we need to do is to invoke the gruntfile from javascript. The only complication is that grunt exits whole process on task failure. Therefore, we need to call it from child process.

Node module child process has three different functions able to run command inside child process:
  • exec - executes command on command line,
  • spawn - differently executes command on command line,
  • fork - runs node module in child process.

The first one, exec, is easiest to use and is explained in the first subchapter. Second subchapter shows how to use fork and why it is less optimal then exec. Third subchapter is about spawn.

Exec runs command line command inside a child process. You can specify in which directory to run it, set up environment variables, set timeout after which the command will be killed and so on. When the command finishes its run, exec calls callback and passes it stdout stream, stderr streams and error if the command crashed.

Unless configured otherwise, command is run in current directory. We want it to run inside tests subdirectory, so we have to specify cwd property of options object: {cwd: 'tests/'}.

Both stdout and stderr streams content are stored inside a buffer. Each buffer has maximum size set to 204800 and if the command produces more output, exec call will crash. That amount is enough for our small task. If you need more you have to set maxBuffer options property.

Call Exec
Following code snippet shows how to run the gruntfile from exec. The function is asynchronous and calls whenDoneCallback after all is done:
var cp = require("child_process");

function callGruntfile(filename, whenDoneCallback) {
  var command, options;
  command = "grunt --gruntfile "+filename+" --no-color";
  options = {cwd: 'test/'};
  cp.exec(command, options, whenDoneCallback);
Note: if you installed npm into tests directory (simple solution), then you need to use callNpmInstallAndGruntfile function instead of callGruntfile:
function callNpmInstallAndGruntfile(filename, whenDoneCallback) {
  var command, options;
  command = "npm install";
  options = {cwd: 'test/'};
  cp.exec(command, {}, function(error, stdout, stderr) {
    callGruntfile(filename, whenDoneCallback);
Unit Tests
First node unit test runs success scenario and then checks whether process finished without failure, whether standard output contains expected message and whether standard error is empty.

Success scenario unit test:
pass: function(test) {
  callGruntfile('gruntfile-pass.js', function (error, stdout, stderr) {
    test.equal(error, null, "Command should not fail.");
    test.equal(stderr, '', "Standard error stream should be empty.");

    var stdoutOk = contains(stdout, 'Plugin worked correctly.');
    test.ok(stdoutOk, "Missing stdout message.");
Second node unit test runs "complete failure" scenario and then checks whether process failed as expected. Note that standard error stream is empty and warnings are printed into standard output.

Failing scenario unit test:
fail_1: function(test) {
  var gFile = 'gruntfile-fail-complete.js';
  callGruntfile(gFile, function (error, stdout, stderr) {
    test.equal(error, null, "Command should have failed.");
    test.equal(error.message, 'Command failed: ', "Wrong error message.");
    test.equal(stderr, '', "Non empty stderr.");

    var stdoutOk = containsWarning(stdout, 'complete failure');
    test.ok(stdoutOk, "Missing stdout message.");
Third "partial failure" node unit test is almost the same as the previous one. Whole tests file is available on github.

  • Maximum buffer size must be set in advance.

Fork runs node.js module inside child process and is equivalent to calling node <module-name> on command line. Fork uses callbacks to send standard output and standard error to caller. Both callbacks can be called many times and caller obtains child process outputs in pieces.

Using fork makes sense only if you need to handle arbitrary sized stdout and stderr or if you need to customize grunt functionality. If you do not, exec is easier to use.

This chapter is split into four sub-chapters:
Call Grunt
Grunt was not meant to be called programatically. It does not expose "public" API and does not document it.

Our solution mimics what grunt-cli does, so it is relatively future safe. Grunt-cli is distributed separately from grunt core and therefore is less likely to change. However, if it does change, this solution will have to change too.

Running grunt from javascript requires us to:
  • separate gruntfile name from its path,
  • change active directory,
  • call grunts tasks function.

Call grunt from javascript:
this.runGruntfile = function(filename) {
  var grunt = require('grunt'), path = require('path'), directory, filename;
  // split filename into directory and file
  directory = path.dirname(filename);
  filename = path.basename(filename);

  //change directory

  //call grunt
  grunt.tasks(['default'], {gruntfile:filename, color:false}, function() {
Module Arguments
The module will be called from command line. Node keeps command line arguments inside process.argv array:
module.exports = new function() {
  var filename, directory;

  this.runGruntfile = function(filename) {
    /* ... */

  //get first command line argument
  filename = process.argv[2];

Call Fork
Fork has three arguments: path to module, array with command line arguments and options object. Call module.js with tests/Gruntfile-1.js parameter:
child = cp.fork('./module.js', ['tests/Gruntfile-1.js'], {silent: true})
The silent: true option makes stdout and stderr of the returned child process available inside the parent. If it is set to true, returned object provides access to stdout and stderr streams of the caller.

Call on('data', callback) on each stream. Passed callback will be called each time the child process sends something to the stream:
child.stdout.on('data', function (data) {
  console.log('stdout: ' + data); // handle piece of stdout
child.stderr.on('data', function (data) {
  console.log('stderr: ' + data); // handle piece of stderr

Child process can either crash or end its work correctly:
child.on('error', function(error){
  // handle child crash
  console.log('error: ' + error); 
child.on('exit', function (code, signal) {
  // this is called after child process ended
  console.log('child process exited with code ' + code); 

Demo project uses following function to calls fork and to bind callbacks:
 * callbacks: onProcessError(error), onProcessExit(code, signal), onStdout(data), onStderr(data)
function callGruntfile(filename, callbacks) {
  var comArg, options, child;
  callbacks = callbacks || {};

  child = cp.fork('./test/call-grunt.js', [filename], {silent: true});

  if (callbacks.onProcessError) {
    child.on("error", callbacks.onProcessError);
  if (callbacks.onProcessExit) {
    child.on("exit", callbacks.onProcessExit);
  if (callbacks.onStdout) {
    child.stdout.on('data', callbacks.onStdout);
  if (callbacks.onStderr) {
    child.stderr.on('data', callbacks.onStderr);

Write Tests
Each unit test calls the callGruntfile function. Callbacks search for expected content inside the standard output stream, check whether exit code was correct, fail when something shows up on error stream or fail if fork call returns an error.

Success scenario unit test:
pass: function(test) {
  var wasPassMessage = false, callbacks;
  callbacks = {
    onProcessError: function(error) {
      test.ok(false, "Unexpected error: " + error);
    onProcessExit: function(code, signal) {
      test.equal(code, 0, "Exit code should have been 0");
      test.ok(wasPassMessage, "Pass message was never sent ");
    onStdout: function(data) {
      if (contains(data, 'Plugin worked correctly.')) {
        wasPassMessage = true;
    onStderr: function(data) {
      test.ok(false, "Stderr should have been empty: " + data);
  callGruntfile('test/gruntfile-pass.js', callbacks);
Tests corresponding to failure scenario are pretty much the same and can be found on github.

  • Used grunt function does not belong to official API.
  • Child process output streams are available in chunks instead of one big block.

Spawn is a cross between fork and exec. Similarly to exec, spawn is able to run an executable file and pass it command line arguments. Child process output streams are treated the same way as in fork. They are send to parent in pieces via callbacks. Therefore, exactly as with fork, using spawn makes sense only if you need arbitrary sized stdout or stderr.

The Problem
The main problem with spawn happens on windows. The name of command to be run must be specified exactly. If you call spawn with an argument grunt, spawn expects executable filename without suffix. Real grunt executable grunt.cmd will not be found. Otherwise said, spawn ignores windows environment variable PATHEXT.

Looping Suffixes
If you want to call grunt from spawn, you will need to do one of the following things:
  • use different code for windows and for linux or
  • read PATHEXT from environment and loop through it until you find the right suffix.

Following function loops through PATHEXT and passes the right filename to the callback:
function findGruntFilename(callback) {
  var command = "grunt", options, extensionsStr, extensions, i, child, onErrorFnc, hasRightExtension = false;

  onErrorFnc = function(data) {
    if (data.message!=="spawn ENOENT"){
      grunt.warn("Unexpected error on spawn " +extensions[i]+ " error: " + data);

  function tryExtension(extension) {
    var child = cp.spawn(command + extension, ['--version']);
    child.on("error", onErrorFnc);
    child.on("exit", function(code, signal) {
      hasRightExtension = true;
      callback(command + extension);

  extensionsStr = process.env.PATHEXT || '';
  extensions = [''].concat(extensionsStr.split(';'));
  for (i=0; !hasRightExtension && i<extensions.length;i++) {
Write Tests
Once you have grunt command name, you are ready to call spawn. Spawn fires exactly the same events as fork, so callGruntfile accepts exactly the same callbacks object and binds its properties to child process events:
function callGruntfile(command, filename, callbacks) {
  var comArg, options, child;
  callbacks = callbacks || {};

  comArg = ["--gruntfile", filename, "--no-color"];
  options = {cwd: 'test/'};
  child = cp.spawn(command, comArg, options);
  if (callbacks.onProcessError) {
    child.on("error", callbacks.onProcessError);
  /* ... callbacks binding exactly as in fork ...*/
Tests are also almost the same as those in the previous chapter. Only difference is that you have to find the grunt executable filename before doing everything else. Success scenario test looks like this:
pass: function(test) {
  var wasPassMessage = false;
    var callbacks = {
      /* ... callbacks look exactly the same way as in fork ... */
    callGruntfile(gruntCommand, 'gruntfile-pass.js', callbacks);
Full success scenario test along with both failure scenarios tests are available on github.

  • Spawn ignores PATHEXT suffixes, custom code to handle it is needed.
  • Child process output streams are available in chunks instead of one big block.


There are three ways how to test grunt plugin from inside gruntfile. Unless you have very strong reason not to, use exec.


ganga pragya said...

Your good knowledge and kindness in playing with all the pieces were very useful. I don’t know what I would have done if I had not encountered such a step like this.
Block Chain Training in chennai

Block Chain Training in annanagar

Block Chain Training in pune

Block Chain Training in velachery

Unknown said...
This comment has been removed by the author.
Unknown said...

Good Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging.
Data Science training in kalyan nagar | Data Science training in OMR
Data Science training in chennai | Data science training in velachery
Data science training in tambaram | Data science training in jaya nagar

Unknown said...

I appreciate your efforts because it conveys the message of what you are trying to say. It's a great skill to make even the person who doesn't know about the subject could able to understand the subject . Your blogs are understandable and also elaborately described. I hope to read more and more interesting articles from your blog. All the best.
java training in chennai | java training in bangalore

java online training | java training in pune

sai said...

Good Post, I am a big believer in posting comments on sites to let the blog writers know that they ve added something advantageous to the world wide web.
online Python certification course
python training in OMR
python training course in chennai

cynthiawilliams said...

Thanks for sharing this pretty post, it was good and helpful. Share more like this.
Blockchain Training in Chennai
Blockchain course in Chennai
Angularjs Training in Chennai
AWS Training in Chennai
DevOps Training in Chennai
Python Training in Chennai

Anbarasan14 said...

I believe that your blog will surely help the readers who are really in need of this vital piece of information. Waiting for your updates.

French Class in Mulund
French Coaching in Mulund
French Classes in Mulund East
French Language Classes in Mulund
French Training in Mulund
French Coaching Classes in Mulund
French Classes in Mulund West

Sadhana Rathore said...

Useful content, I have bookmarked this page for my future reference.
Appium Training in Chennai
Best Appium Training institute in Chennai
Appium Certification in Chennai
Mobile Appium Training in Chennai
Mobile Appium course in Chennai

Aruna Ram said...

Very creativity blog!!! I learned a lot of new things from your post. It is really a good work and your post is the knowledgeable. Waiting for your more updates...
Blue Prism Training Institute in Bangalore
Blue Prism Course in Bangalore
Blue Prism Training Bangalore
Blue Prism Classes in Bangalore
Blue Prism Course in Adyar
Blue Prism Training in Mogappair

sathyaramesh said...
This comment has been removed by the author.
punitha said...

Nice article I was really impressed by seeing this blog, it was very interesting and it is very useful for me.
Javascript Training in Bangalore
Java script Training in Bangalore
Javascript Training Institutes in Bangalore
Advanced Java Training Institute in Bangalore
Best Institute For Java Course in Bangalore

pavithra dass said...

I am obliged to you for sharing this piece of information here and updating us with your resourceful guidance. Hope this might benefit many learners. Keep sharing this gainful articles and continue updating us.
Angularjs Training in Chennai
Angularjs Training
Angularjs Training near me
DevOps Training in Chennai
DevOps certification Chennai
DevOps certification

mercyroy said...

Great!it is really nice blog information.after a long time i have grow through such kind of ideas.thanks for share your thoughts with us.
Selenium Courses in T nagar
Selenium Training Institutes in T nagar
Selenium Certification Training in OMR
Selenium Training in Perungudi

Aruna Ram said...

Well post, very useful content and I really impressed. I need more info to your blog. Keep Posting.
SEO Course in Nungambakkam
SEO Training in Saidapet
SEO Course in Tnagar
SEO Course in Omr
SEO Training in Sholinganallur
SEO Course in Navalur

jefrin said...

Thanks for the post very impressive

Best Blockchain training in chennai

Clipping Path said...

Thank you so much for the detailed article

lekha mathan said...

Such an excellent and interesting blog, do post like this more with more information, this was very useful, Thank you.
Aviation Courses in Chennai
cabin crew training chennai
airline academy in chennai
airport ground staff training courses in chennai
medical coding course in chennai
fashion designing institute in chennai
best interior design courses in chennai

Earth Fare said...

I couldn’t resist commenting. Exceptionally well written!

Boots Opticians
CiCi's Pizza
Panda Express Feedback from these surveys you can win coupons, Free Fresh Frozen Custard from culvers & Free Donuts within minutes. give your feedback and grab these offers

manisha said...

Thanks for sharing this valuable information and we collected some information from this blog.
Javascript Training in Delhi

Mika Farron said...

Apabila anda tidak memperhatikan nilai kartu yang anda miliki tersebut, maka tentukan taruhan dengan memperhatikan nilai kartu terlebih dahulu. Dengan begitu, anda tidak akan mendapatkan kerugian yang besar.
bandar ceme terbaik
paito warna terlengkap
forum prediksi

Anonymous said...

Come on... Keep posting, Lady... I was a while since your last post. Don't make me sad (:

Chris Hemsworth said...

I have been reading for the past two days about your blogs and topics, still on fetching! Wondering about your words on each line was massively effective. Techno-based information has been fetched in each of your topics. Sure it will enhance and fill the queries of the public needs. Feeling so glad about your article. Thanks…!
magento training course in chennai
magento training institute in chennai
magento 2 training in chennai
magento development training
magento 2 course
magento developer training

steve said...

Top Chauffeur service in Melbourne
Whether you need a last minute chauffeur car or a planned vehicle for your outing, book with us and get served on time. With well-mannered chauffeurs and finest vehicles, we arrange to pick and drop our customers with great punctuality. A hassle-free traveling experience waits at Silver Executive Cab for every customer

subha said...

The post you wrote which is full of informative content. I Like it very much. Keep on posting!!thanks a lot
Ai & Artificial Intelligence Course in Chennai
PHP Training in Chennai
Ethical Hacking Course in Chennai Blue Prism Training in Chennai
UiPath Training in Chennai

QuickBooks Error said...

Nice Blog !
Our experts at QuickBooks Phone Number are ready to give you immediate support for all QuickBooks errors in this difficult time.

vé máy bay từ canada về Việt Nam said...

Đặt vé máy bay tại Aivivu, tham khảo

vé máy bay đi Mỹ hạng thương gia

vé máy bay hà nội sài gòn vietjet

vé máy bay đà nẵng hà nội hôm nay

vé máy bay đi nha trang tháng 7

vé máy bay đi quy nhơn tháng 2

dịch vụ taxi sân bay

combo phú quốc 4 ngày 3 đêm vinpearl

Vé máy bay từ Canada về việt nam said...

Mua vé máy bay tại Aivivu, tham khảo

vé máy bay đi Mỹ bao nhiêu tiền

mua vé máy bay về vn từ mỹ

mua ve may bay gia re tu duc ve viet nam

dịch vụ vé máy bay tại nga

giá vé máy bay từ anh về hà nội

vé máy bay từ pháp về việt nam

khách sạn cách ly ở đà nẵng

anjali said...

Django store image in database
Python MySQL Tutorial
Node js Data types
PHP multiple choice questions and answers
PHP html to pdf
Javascript add rows to table dynamically
Insert in database without page refresh php
Django insert data into database
Python convert xml to csv
Send email to multiple recipients Python

Quickbooks Customer Service said...

Awesome bolg.if you looking for a best quickbook customer service you can contact us on phone call.+1 888-272-4881

QuickBooks support Service said...

awesome bolg.if you need best quickbook support service. contact to quickbook service team.+1 888-471-2380

Quickbooks customer service said...

Nice content!!
if you are looking for a best Quickbook support serviceyou can reach us at.+1 855-444-2233

Quickbooks Customer Service said...

QuickBooks also offers live chat support as well as email support which allows businesses who are out of the office or at home to access QuickB's qualified help desk staff. The QuickBooks Service Number is +1 888-471-2380 and is always available 24 hours a day to serve you with all your QuickBooks questions.

VBSPU BA 1st Year Exam Result said...

Thanks for posting a very sweet article. So much needed data I'll get and keep in my file. Thank u so much.

VBSPU BA 2nd Year Exam Result | VPSPU BA 3rd Year Exam Result.

casinositeone.JDS said...

Thank you so much for sharing this blog with us.

casinositeguidecom.JDS said...

I really appreciate your work.

safetotositepro.JDS said...

Good article! Keep sharing this type of information to expand user knowledge.

gostopsite said...

Wow, amazing blog format! How long have you been blogging for? you make blogging look easy.

sportstoto365 said...

“You are so entrancing! I figure I haven’t actually examined anything indistinct at this point.

bacarasite said...

I have read it, but the information is still less than mine, i am wanting to learn more

totosafesite said...

this type of article that enlighted me all thoughout and thanks for this.

Post a Comment