Using Grunts Config API

What was the problem again?

If you look at my previous post you'll see the Sass architecture/structure we're currently using on the BBC News responsive website means that Sass is unable to watch changes in files outside of the specific language folders we've asked Sass to watch (e.g. we tell Sass to watch the Arabic Sass files for changes, which it does that fine, but the Arabic files themselves import Sass files from a 'partials' directory outside of the language directory and those files Sass cannot detect changes for).

How were we solving this originally?

We use Grunt to handle our front-end tooling tasks (such as running Sass, linting CSS/JS, running JS unit-tests, running our AMD build script and a host of other things) and so we initially were just hard coding in the relevant watch and sass Grunt task commands to work around the issue.

Next (as per my previous Grunt post) we then tried dynamically updating the Grunt file by inserting hooks into the file which at runtime we would use to insert our dynamically generated stack of sub tasks. Although this works fine and didn't cause us any problems, it just felt… wrong?

So what now?

Since publishing that Grunt post, the creator of Grunt (Ben Alman) suggested there was a better way using the Grunt API itself.

How are we solving this now then?

Exactly as Ben suggested, by using the Grunt API. Specifically: grunt.config.set (there are other Grunt API's involved but this one is the fundamental solution).

Let's review the solution

We have three files…

The first file you should all know (if you don't then have a read back over my original Grunt Boilerplate post).

The second file is a generic file we're using to store our custom Grunt tasks (and ultimately is what does most of the work for this particular problem).

The third file is again another generic file but this time we use it to store any additional utility helper methods.

Here's how they're used…

Our Gruntfile.js pulls in the grunt-customtasks.js file like so: require('./grunt-customtasks.js')(grunt);. You'll notice that the module we're pulling in is actually a function which we execute immediately and pass it a reference to Grunt (which means our custom tasks module can access Grunt and use its API).

The grunt-customtasks.js module itself pulls in the grunt-helpers.js file like so: var util = require('./grunt-helpers.js')(grunt);. Again you'll notice that the module we're pulling in is actually a function, we execute it and pass in a reference to Grunt, but this module (although a function) once executed then returns an object which we assign to the variable util. The returned object has some useful methods we can use to implement our solution.

Looking again at grunt-customtasks.js we have the following custom Grunt tasks…

…now normally when using a pre-built Grunt task such as a the Sass task, the user would execute something like sass:my_sub_task, but our users won't ever directly interact with the Sass task. This is because there are no Sass sub tasks in our Grunt file to use! Instead they'll use our facade sass_compile which at run time will dynamically generate the Sass sub task they have requested.

Same for the pre-built Watch task. The user will use our watch_service custom task which is a facade that dynamically generates the requested watch task at run time.

sass_compile

This task dynamically generates our Sass sub tasks (e.g. sass:arabic). It uses the Grunt Config API to dynamically set this up (we'll look at this in more detail later on).

In this task we have to carry out some checks on the sub task that we are going to create. The reason being: we want to let the user know whether they've requested a service that doesn't exist.

If the user executes sass:something_that_does_not_exist we don't want to just create that Sass sub task because ultimately it'll not do anything and that would be very confusing to the user who expects it to work (or maybe they entered a typo and haven't realised) - so we need to give the user feedback to let them know the command they entered isn't recognised.

So we use our own utility/helper method util.serviceExists to check whether the requested service actually exists or not. If it does then we go ahead and dynamically generate the sub task, otherwise we display a message to the user to let them know the service doesn't exist.

We also have slightly different settings for the sub task depending on what service was requested. If the user enters a language as the service sass_compile:arabic then we'll use development settings such as enabling debugInfo and turning on lineNumbers and making sure the code is expanded and not compressed, but if they enter sass_compile:dist we'll make sure the compiled code is ready for distribution.

watch_service

This custom task is much like sass_compile in that it first checks that the service being requested is valid and if so will continue to dynamically generate a watch sub task.

But there is additional work required in this task. Let's just take a moment to remember the purpose of the watch task which is to tell Grunt to execute another Grunt task (pre-built or custom) when the files being watched are changed.

We hit a problem when first implementing this solution which was that when the Watch sub task was generated we had set it to execute a Sass sub task matching the service requested.

So for example if the user ran watch_service:arabic we would tell Grunt that when those files changed then it should execute the Sass sub task sass:arabic. But that Sass sub task didn't exist at that point because the user hadn't previously run sass_compile:arabic (remember, our Sass task inside our Gruntfile.js is empty).

To work around this we told the Watch sub task to execute two tasks when the watched files changed. The first task to execute was config and that would be passed the service name that was passed to the watch_service task. For example, config:arabic and then after that we would execute sass:arabic (or whatever service was requested).

Let's have a look now at the config task to see what that does…

config

So as explained above, we needed to call this task before calling the Sass sub task (because at that point the sub task that needed to be executed didn't exist).

In this custom task all we do is literally call our sass_compile custom task, pass through the requested service but also pass through an additional parameter which lets the sass_compile task know that it should just create the Sass sub task and not execute it.

To explain, if the user runs sass_compile:arabic that task will not only dynamically generate a Sass sub task (e.g. sass:arabic), but once created will actually execute that task now it exists. But if the user has run the watch_service task, we call this config task which delegates onto sass_compile, but we don't want the Sass sub task we've just generated to be executed because the files being watched haven't necessarily been changed! So we wrote sass_compile in a way that said "if this particular parameter is passed through, then just create the Sass sub task but don't execute it".

The Grunt API

Before we get to see our finished code let's first take a look at the Grunt API we're using to implement this solution (sorry, I know there has been a lot of talk up until this point, but we're almost done explaining how everything has been put together, I promise!)

So the API's we're using are…

grunt.config.get

If you call grunt.config.get() then you'll get a copy of the entire grunt.initConfig object back. If you pass in a property like grunt.config.get('sass') you get the value of that specific property.

grunt.config.set

If you call grunt.config.set(property, value) then you'll set that property and its corresponding value on the grunt.initConfig object.

Be aware you can only set a single value onto a property, so if you wanted to create multiple sub tasks for a property (such as creating multiple sub tasks for the Sass task) then you would have to first construct an object containing the sub tasks and then assign that whole object as the property value.

grunt.task.run

This method simply lets you run any task (custom, pre-built, sub task) from within your code.

So if I had a variable called service and it had a value of 'arabic' then running grunt.task.run('watch:' + service) would run watch:arabic.

We use this API method inside our watch_service, config and sass_compile custom tasks.

grunt.file.expand

This API method is used indirectly via our util helper object. We have a method called serviceExists which takes in a service and checks whether it exists or not. That method checks a service property on our util object which is an Array holding the services we have available.

The way that service Array is populated is via the grunt.file.expand API call. We pass the method a file glob and it returns an Array of all directories/files that match the glob provided.

So for example, grunt.file.expand('sass/services/*') returns an Array of all folders within our Sass/Services directory. As you'll see in the example code below we actually filter out some folders that we know aren't valid.

Example code

OK, finally we get to see some code! Hopefully by now it should be much clearer as to what it's doing and why…

grunt-customtasks.js

module.exports = function (grunt) {

    var util = require('./grunt-helpers.js')(grunt);

    grunt.registerTask('sass_compile', 'Dynamically generate Sass sub task', function(service, calledFromConfigTask) {
        if (!util.serviceExists(service)) {
            console.log('Sorry, that service does not exist');
            return;
        }

        var _          = require('lodash'),
            src        = ['**/*.scss', '!**/_*.scss', '!locator/*.scss'],
            debug      = { debugInfo: true, lineNumbers: true },
            style      = 'expanded',
            requested  = service,
            cwd = dest = '',
            service    = {},
            config;

        service[requested] = {};

        switch (requested) {
            case 'dist':
                style = 'compressed';
                debug = {};
                break;
            case 'dev':
                // Want to prevent setting cwd/dest to the service name
                break;
            default:
                src = ['*.scss', '!_*.scss'];
                cwd = dest = 'services/' + requested;
        }

        config = {
            options: {
                style: style,
                require: ['<%= dir.static_sass %>partials/helpers/url64.rb']
            },
            expand: true,
            cwd: '<%= dir.static_sass %>' + cwd,
            src: src,
            dest: '<%= dir.static_css %>' + dest,
            ext: '.css'
        }

        _.merge(config.options, debug);
        _.merge(service[requested], config);

        grunt.config.set('sass', service);

        if (!calledFromConfigTask) {
            grunt.task.run('sass:' + requested);
        }
    });

    grunt.registerTask('config', 'Add dynamically generated Sass sub task into config object but does not run the sub task', function(service) {
        grunt.task.run('sass_compile:' + service + ':watch'); // ensure sub task exists before running the dynamically created watch sub task
    });

    grunt.registerTask('watch_service', 'Dynamically generate Watch (and Sass) sub tasks and then run the relevant watch task', function(service) {
        if (!util.serviceExists(service)) {
            console.log('Sorry, that service does not exist');
            return;
        }

        var content = {};

        if (service === 'all') {
            content.all = {
                files: ['<%= jshint.files %>', '<%= dir.static_sass %>**/*.scss'],
                tasks: ['config:dev', 'default']
            }
        } else {
            content[service] = {
                files: ['<%= dir.static_sass %>partials/**/*.scss',
                        '<%= dir.static_sass %>services/' + service + '/*.scss'],
                tasks: ['config:' + service, 'sass:' + service]
            }
        }

        grunt.config.set('watch', content);
        grunt.task.run('watch:' + service);
    });

};

grunt-helpers.js

module.exports = function (grunt) {
    function removePath(element) {
        return element.split('/').pop();
    }

    function removeBlacklistedDirectories(element) {
        if (element.indexOf('journalism') !== -1) {
            return false
        }

        return true;
    }

    return {
        services: grunt.file.expand('tabloid/webapp/static/sass/services/*').map(removePath).filter(removeBlacklistedDirectories),
        serviceExists: function(service) {
            if (this.services.indexOf(service) !== -1) {
                return true;
            }

            return false;
        },
        isBlacklistedTask: function(task) {
            var tasks = ['pkg', 'dir', 'noop', 'config'];

            if (tasks.indexOf(task) !== -1) {
                return true
            }

            return false;
        }
    }
};

Conclusion

So there you have it. A real-world look at using the Grunt API to do something a little bit more involved with Grunt. Hopefully you found it useful and will give you a better idea of how you can integrate Grunt with your own work flow and solve your own specific domain problems.

Update

A quick update to say we found a much simpler route to handle the dynamic generation of Sass content and this came from recognising Grunt's ability to access arguments passed through the command line: <%= grunt.task.current.args[0] %>.

With this knowledge we then call the task like so: 'sass:service:<%= grunt.task.current.args[0] %>'.

Goes to show it's always worth your time reading through all of the API documentation for the tools you use to discover these little gems :-)

So here is a more realistic example...

sass: {
    /*
        Example usage...

        grunt sass:service:afrique
        grunt sass:service:news
     */
    service: {
        expand: true,
        cwd: '/sass/',
        src: ['services/<%= grunt.task.current.args[0] %>/*.scss', '!**_*.scss'],
        dest: '/stylesheets/',
        ext: '.css'
    },
    dist: {
        expand: true,
        cwd: '/sass/',
        src: ['**/*.scss', '!**/_*.scss', '!locator/*.scss'],
        dest: '/stylesheets/',
        ext: '.css',
        options: {
            style: "compressed"
        }
    },
    dev: {
        expand: true,
        cwd: '/sass/',
        src: ['**/*.scss', '!**/_*.scss', '!locator/*.scss'],
        dest: '/stylesheets/',
        ext: '.css',
        options: {
            debugInfo: true,
            lineNumbers: true
        }
    },
    options: {
        style: 'expanded',
        require: ['/sass/partials/helpers/url64.rb']
    }
}

Links