Towards a More Modular Future for JavaScript Libraries

by Brian Rinaldi on November 11, 2013

The modern web is always changing, and this article is more than two years old.

By Toby Ho

Until recently, JavaScript did not have much of a module story. In other words, there was no good way of sharing code with others. I would argue that it still doesn’t have a great one, even today. First, let’s look at the ways we share JavaScript code today. I’ll discuss why I think we are framework-heavy as a community. Finally, I’ll look at potential solutions that exist, and some of the issues they face.

Copy/Paste

Let’s imagine that you’re searching for how to throttle in JavaScript, and you land on a blog post that contains 11 lines of magic code you can copy/paste. Copy/pasting code into your project may feel wrong, you think- perhaps it’d better to fetch a file to clearly delineate your own code versus 3rd party code. Plus, if the author were to fix a bug or add enhancements in the future, it would be more straight-forward to update the code. But is 11 lines really worth it? Probably not. It’s almost faster to copy/paste those 11 lines than to download a .js file. You paste in the code and leave the blog url in a comment to let others and future you know where it’s from.

The .js File

Another possibility is you find the functionality you need from a more inclusive library. For example, Underscore.js includes the throttle function out of the box. To use Underscore.js in your project, you would:

  1. download the file underscore.js.
  2. copy it into a subdirectory in your project.
  3. add a <script> tag in your page(s).

This workflow is good compared to what it used to be. There was a time when using a JavaScript library involved these steps

  1. downloading a zip file.
  2. unpacking it.
  3. figuring out which files you need.
  4. move them to the appropriate subdirectories in your project.
  5. adding <script> tags to your page(s).

It was a hassle. Most libraries have gone away from that, opting for the all-you-have-to-download-is-the-one-js-file approach, because it’s much easier for the user. Now the user only has to add one <script> tag, and there is no guessing as to which .js file it is.

Depend on No One

An unspoken rule in the world of .js files is “depend on no one” – that is, unless it’s jQuery.

Today, you seldomly encounter a library that requires you to download other dependencies. People are impatient, and the more stuff you ask them to do before actually seeing things happen, the less likely they are to use the library in the first place.

If a library must have dependencies, what usually happens is: either they ask you to download a .zip file that contains all of the dependencies, or they concatenate the dependencies into a monolithic file along with the library itself. You’ll see this with some libraries that depend on JSON parsing or base64 encoding, for example.

For the most part, we, the library authors, have shied away from depending on other libraries. This is a limitation. It makes it hard to stand on other’s shoulders, but it also makes it hard to reuse our own code in multiple contexts, which can be an effective way to remove code duplication. In this way, we have been handicapped for many years.

We Make an Exception for jQuery

Depending on jQuery though, is OK, because

  1. you’re already using jQuery, right?
  2. jQuery is rock solid, battled-tested, has a stable API, and it’s only the most useful script in the world.
  3. you are already using jQuery, right?

Although, even for jQuery plugins, the “depend on no one” rule holds – you will seldom find a jQuery plugin that depends on another jQuery plugin or a standalone library. jQuery is what I refer to as a “god framework.”

God Frameworks

I call jQuery a “god framework” because without it, the universe ceases to function as so much depends on it. A dependency graph of a typical jQuery-based project probably has most, if not all, of it’s modules pointing to jQuery. The only hope of getting by without it is to replace it with something that works exactly like jQuery – a sort of clone. Zepto is exactly that, and it exists probably because someone went to great pains to rid themselves of jQuery.

jQuery is not the only god framework though – not by far. Most frameworks, more or less, want to be god-like. One tell is if it has plugins as a plugin ecosystem is obviously completely dependent on its parent fraework.

Framework-Heavy

The JavaScript world is dominated by frameworks, and I believe it is because of our reluctance to having dependencies in our libraries. If you are going to write a JavaScript library, you have two options:

  1. Depend on a proven framework.
  2. Depend on no one.

If you choose option #1, you contribute to the worship of the framework.

If you choose option #2, you are faced with a problem. If you make it too small, it may not be useful enough to warrant a download (are you “download-worthy”?). What happens sometimes is a set of smaller functionalities get packaged up into a larger library to give users more value for the download. Or established libraries incorporate the new features and techniques for existing users. Examples of this are:

  1. the addition of promises into jQuery.
  2. version 0.1.0 of underscore had 42 methods. Version 1.5.2 of underscore has 109 methods.

This is why the JavaScript world is dominated by frameworks. There is good reason why jQuery is a “Swiss Army Knife”: it has tons of features in a single download which makes it convenient.

The Dream

As someone who likes to write small libraries, the dream for me, is this: I want easy dependencies. Regardless of how many dependencies and/or nested dependencies my project has, I want to be able tell people “using my library is as easy as 1-2-3.”

What I want is something like npm.

The Beauty of npm

The rise of Node.js and with it, npm has been an important development. npm is a joy to use, and has made publishing server-side JavaScript modules really easy. It has made dependencies easy, both for module authors and users.

If you want to add the express module to your project, just install it with the command:

npm install express 

…and then in your code you’d just require it by the same name:

var express = require('express')

If the module depends on other modules, they are automatically fetched recursively for you. You may not even notice them if you don’t pay attention to npm’s console output. Node developers take full advantage of dependencies. This is the dependency graph of Express, the server-side web framework:

express_dependencies

Visualization generated by https://david-dm.org/visionmedia/express#info=dependencies&view=tree

npm has re-taught us that modules do, in fact, work, and that it works in JavaScript. It has flourished beyond most people’s expectations. In just a few years, npm surpassed CSPAN and PyPI in number of packages, and that number grows at a consistently higher rate than the module registries for any other programming language. (source)

Small is Beautiful

The npm community of modules embrace smallness. Small modules have these characteristics

  1. they are small (duh), a good rule of thumb is ~200 LOC or less.
  2. they have a single-purpose. This is actually good engineering practice in general – it aligns with the Single Responsibility Principle.
  3. they have a simple API and short learning curve, given #2.
  4. they are easy to make, given #1.
  5. they are easy to get rid of – you can either rewrite the parts of your code that depend on it, or you can re-implement the API of the module, either way is easy given #1 and #2.

NPM for Browser Code?

But npm is for server-side JavaScript. What about client-side JavaScript?

The short answer is: It’s complicated.

The slightly longer answer is: it’s a race to the top, and the front runners at the time of writing are NPM with Browserify and Bower.

Browserify is a tool that allows the CommonJS module system to be used in the browser. It does require a compilation step, but this step can be automated away. Browserify works great with NPM, and there is an increasing number of client-side modules on NPM.

Bower is a newer module registry for browser code. Bower does not have an opinion about module formats. To install a library, Bower simply fetches the contents of a git repository. It is up to the users to decide what module system they want to use and how they want to consume the fetched library, whether it’s CommonJS, AMD with Require.js or simply <script> tags.

Component is the contender in third place, but nevertheless has accumulated many useful modules. Like Node, it is based on CommonJS.

Small Modules in the Wild

Despite the fragmentation in the frontend JavaScript package manager space, their availability is giving rise to reusable modules in smaller chunks. Here are some examples of this at work:

  1. hyperscript – a tiny DOM builder that supports automatic two-way data binding via another tiny module observable. It depends on even tinier modules such as
    • class-list – add and remove class names on an element
    • data-set – set data attributes on an element
    • browser-split – simply a cross-browser String#split implementation
  2. level.js – an implementation the [leveldb] api in the browser. It uses IDBWrapper, which provides a usable cross-browser interface for IndexedDB.
  3. superagent – an elegant ajax library. It’s dependencies
    • emitter – a browser version of Node’s event emitter
    • reduce-component – a cross-browser version of Array#reduce
  4. page.js is a small client-side router inspired by Express designed to work with pushState.
  5. masonry – Masonry is a cascading grid layout library. It works with or without jQuery, but what is interesting is that its author made it’s internals very modular. The bulk of the work is of Masonry is actually done within another library outlayer – which in turn has more dependencies like
  6. lodash – lodash started out as an Underscore clone, but now has features all its own. Interestingly, Lodash has made each of its methods available as a separate module on npm. For example, if all you needed were the _.throttle function you can install it via npm install lodash.throttle and you’d get the code needed for that function alone.
  7. jQuery – the maintainers have focused on making jQuery more modular internally. The jQuery source is now authored in AMD, and it is possible to create custom builds of jQuery yourself which excludes the modules you don’t need. It shouldn’t be too much of a stretch from this to be able to include just the modules you need. Although, if you want small portions of jQuery’s functionality, there already are modules for that

But the FUD

There are still many questions in the air, not to mention fear, uncertainty, and doubt. Questions like

  1. Which is the one true package manager?
  2. Module formats: CommonJS or AMD? (or ES6 Harmony?) Or nothing at all?
  3. How to consume these modules? Require.js, Browserify or something else?

Some of the modules mentioned above use CommonJS while others use AMD, some support both, some neither. Without everyone agreeing on the one true way to write modules, how can we move forward?

Go Forth and Publish

My answer to that question is: do it anyway. Pick the format/tool you like and go for it. The important thing is that you can:

  1. Start writing modular code right away, in production.
  2. Publish the reusable parts of your app as modules, in small chunks.

The landscape of JavaScript libraries is changing – it’s becoming more modular. Once there is a lot of value in this newer space, consolidation will probably happen, but until then, let’s put more value in there!

Related Resources

This article was originally published at http://tobyho.com/2013/10/20/smalljs/

Header image courtesy of http://upload.wikimedia.org/wikipedia/commons/1/19/Lego_bricks.jpg