Roll Your Own Asset Pipeline with Gulp

By Jeff Dickey

I’ve found myself using Gulp for just about everything involving HTML/CSS/JS these days. It’s super fast, quick to write scripts for and flexible. I’m at a point now where I have a ton of projects I can just cd into, run gulp and be up and running. It’s the best solution I’ve found for delivering static assets whether for local development or production.

If you haven’t used a tool like this before, you might be thinking that your cobbled together scripts (or Rails’ Asset Pipeline) work just fine. The value of using Gulp is not within its ability to concatenate or minify files or any of that. Gulp’s true value lies in its ability to compose these into a workflow well. Its easy to concatenate JavaScript, but mixing that in with something like Rev for static asset revisioning will immediately compound the complexity of managing your home-grown scripts. Gulp’s value compared to Asset Pipeline is that it is still flexible.

If you’ve been using the Asset Pipeline, or on a totally different stack, Gulp should offer you a great solution for static asset compilation. This article will explain how.

Rails Asset Pipeline Woes

One of my favorite parts of Rails has been the Asset Pipeline. The ability to just start throwing assets into the application and have Rails spit out CDN-ready assets has been super helpful. After using it for a few years now though, I’m feeling the pain that many of you are. Asset pre-compilation errors, slow deploys, inflexibility, tying yourself to the Rails stack. It’s rough.

Enter Gulp

Gulp is a build system. It’s like Grunt, Make, Rake, and the like. It’s easy to use for the person running it. While it does have a slight learning curve, you’ll find it a super useful tool for all kinds of tasks. It’ll be the fastest weapon in your toolbox for asset compilation (both in terms of time to develop and run time).

CSS Compilation

I imagine just about everyone these days is using a CSS preprocessor. So let’s see how to do this with Gulp.

First, create a new app folder and add an empty package.json to store your Node.js dependencies (I’m assuming you have Node installed here). Then we install Gulp and its LESS compiler:

mkdir mynewapp
echo "{}" > package.json
 
npm install --global gulp
npm install --save-dev gulp
npm install --save-dev gulp-less

Alright, now create your gulpfile.js:

var gulp = require('gulp');
var less = require('gulp-less');
 
gulp.task('less', function() {
  return gulp.src('css/app.less')
    .pipe(less())
    .pipe(gulp.dest('dist'));
});

This file just defines a single task, less, that will compile the css/app.less file into dist/

Here’s a basic LESS file we may have at css/app.less:

@color-base: #2d5e8b;
 
.class1 {
    background-color: @color-base;
}
.class2 {
    background-color: #fff;
    color: @color-base;
}
.class3 {
    border: 1px solid @color-base;
}

Now if we run gulp less we’ll see our compiled css in dist/app.css. Great!

.class1 {
  background-color: #2d5e8b;
}
.class2 {
  background-color: #fff;
  color: #2d5e8b;
}
.class3 {
  border: 1px solid #2d5e8b;
}

Watching for Changes

The prior example, where we manually called the compilation step, is great for building the assets once (perhaps in a deploy), but for local development it would be a pain to have to type gulp less every time we wanted new assets. gulp.watch() will be our helper here. To use it, rewrite your gulpfile like so:

var gulp = require('gulp');
var less = require('gulp-less');
 
gulp.task('less', function() {
  return gulp.src('css/app.less')
    .pipe(less())
    .pipe(gulp.dest('dist'));
});
 
gulp.task('watch', ['less'], function() {
  gulp.watch('css/**/*.less', ['less']);
});

Note a couple things about this example:

• We watch for any less file but only compile app.less. This is because with LESS you would likely @import whatever other files you need. This strategy will not work for JavaScript (but we’ll cover that later).
• The watch task depends on less. This ensures that the first run will have the assets there already.

Now, run a gulp watch, make an edit to your file and see gulp recompiling the stylesheets each time:

✗✗✗ myapp:(gh-pages) ✗ gulp watch
[gulp] Using gulpfile ~/src/myapp/gulpfile.js
[gulp] Starting 'watch'...
[gulp] Finished 'watch' after 6.17 ms
[gulp] Starting 'less'...
[gulp] Finished 'less' after 24 ms
[gulp] Starting 'less'...
[gulp] Finished 'less' after 2.73 ms

Now we have Gulp compiling our assets every time we touch the file – great! On my projects, I like to include livereload to refresh the browser automatically on updated files. (It’s not as hard to setup as you may think – really!)

Concatenating JavaScript

Now that we’ve mostly reimplemented the Asset Pipeline for CSS, it’s time to look at JavaScript. JavaScript, unlike LESS does not have the convenient @import to specify what files to import, so we will just concatenate all of them into one file. (If you want something like the import, look into Browserify, but that’s out of scope for this guide)

First install gulp-concat:

npm install --save-dev gulp-concat

Then add this script task to your gulpfile.js:

var gulp = require('gulp');
var less = require('gulp-less');
var concat = require('gulp-concat');
 
gulp.task('less', function() {
  return gulp.src('css/app.less')
    .pipe(less())
    .pipe(gulp.dest('dist'));
});
 
gulp.task('scripts', function() {
  return gulp.src('src/**/*.js')
    .pipe(concat('app.js'))
    .pipe(gulp.dest('dist'));
});
 
gulp.task('watch', ['less', 'scripts'], function() {
  gulp.watch('css/**/*.less', ['less']);
  gulp.watch('src/**/*.js', ['scripts']);
});

For our example, let’s add a JavaScript file called src/users.js with the following code (not the most interesting code in the world):

console.log('user.js is called');

Now a gulp scripts or a gulp watch will concatenate your JavaScript as well!

Minification

Minification is easy. Just pipe the JavaScript code stream through UglifyJS (make sure you run npm install with the new gulpfile.js below) and set the compress flag on less:

var gulp = require('gulp');
var less = require('gulp-less');
var concat = require('gulp-concat');
var uglify = require('gulp-uglify');
 
gulp.task('less', function() {
  return gulp.src('css/app.less')
    .pipe(less({compress: true}))
    .pipe(gulp.dest('dist'));
});
 
gulp.task('scripts', function() {
  return gulp.src('src/**/*.js')
    .pipe(concat('app.js'))
    .pipe(uglify())
    .pipe(gulp.dest('dist'));
});
 
gulp.task('watch', ['less', 'scripts'], function() {
  gulp.watch('css/**/*.less', ['less']);
  gulp.watch('src/**/*.js', ['scripts']);
});

Rev

This is my favorite part of using Gulp. Rev will give you that friendly app-ef62e7.js filename output that Asset Pipeline is famous for. The reason for it is you can cache it forever. New requests will just point to new files. CDNs love this. Getting the files to have the hash is pretty easy with Rev.

var rev = require('gulp-rev');
 
gulp.task('rev', ['less', 'scripts'], function() {
  return gulp.src(['dist/**/*.css', 'dist/**/*.js'])
    .pipe(rev())
    .pipe(gulp.dest('dist'))
    .pipe(rev.manifest())
    .pipe(gulp.dest('dist'));
});

Now the filename has the hash appended to it! Note the digest is generated as well; it looks like the following:

{
  "app.css": "app-1c1d3237.css",
  "app.js": "app-26ad0c3f.js"
}

Here is where a bit of custom code would come into play. When you generate your index.html (or wherever else you reference the CSS/JS) you will have to swap out the URL for the one in the digest file. This should only be a matter of parsing this JSON file in your framework of choice, or having Gulp rewrite your index.html to replace the CSS/JS include with the correct filename.

A Note for Angular.js Users

Most of my front-end apps are built with Angular. Two issues crop up when using Angular with the methods described here: the ordering of the module getters/setters and minification, both of which can break Angular’s dependency injection.

If you’ve used Angular, you know there is a way to ‘get’ the module, and a way to ‘set’ the module. angular.module('appname', []); vs angular.module('appname');. You must call the setter once before the getter, then can use the getter as much as you want.

To make this work with my concat strategy, I first create a module.js for each of the modules, then edit the gulpfile’s script task like so:

var gulp = require('gulp');
var uglify = require('gulp-uglify');
var concat = require('gulp-concat');
var ngmin = require('gulp-ngmin');
 
gulp.task('scripts', function() {
  return gulp.src(['src/**/module.js', 'src/**/*.js'])
    .pipe(concat('app.js'))
    .pipe(ngmin())
    .pipe(uglify())
    .pipe(gulp.dest('dist'));
});

Also note that the ngmin Gulp module is thrown in there to solve the minification/DI issue.

Conclusion

There are a lot of places you can go with this. It becomes trivial to deploy to Google Pages, S3, or the public folder of a web app. You could put all of your static assets into its own Git repo, or make this part of your deploy automation. The point here is not to dictate how your app should be built, but that Gulp can offer you flexibility to compose these tools into a workflow that works for you.

So next time you need asset pre-compilation, take a look at Gulp and let me know what you think.

This article was originally published at https://blog.carbonfive.com/2014/05/05/roll-your-own-asset-pipeline-with-gulp/