Taming Asynchronous JavaScript Programming with ECMAScript6

By Vittorio Zaccaria

Asynchronous programming is at the core of making a complex web application feel responsive to the user. However, building code that handles asynchronous method calls can be inherently complex, especially when one method is dependent on the result of another. This is part of what the promises pattern tries to solve, and it has been discussed quite a lot in the JavaScript community lately. Oftentimes, methods that return promises are chained using a then() method construct. However, what if you are dealing a combination of methods, not all of which return promises?

In this article, I will introduce a new library I wrote called Chained that allows you to chain asynchronous method calls whether they return promises or not and without the need for using a then() method construct.

What is Chained?

Chained is an experimental prototype library for Javascript that I developed and licensed under the MIT license. It is meant to show a concept and to provide some limited but useful functionality. All the examples below are written in CoffeeScript (which I guess makes me a hippie, right?).

Chained allows you to chain functions explicitly – both those that return promises and those who don’t – without using then-based constructs.

For example, the following code creates a chained computation that:

• downloads a user’s npm page using jQuery’s get method
• filters the links to the user’s projects using Underscore
• prints the links.

jQuery       = require('jQuery')
underscore   = require('underscore')
string       = require('underscore.string')
linklib      = require('linklib')
{ using, _ } = require('./chain').chain.init()

using(jQuery)
using(underscore)
using(string)
using(console)
using(linklib)

getUser = (user) ->
    _("https://npmjs.org/~#{user}")
      .get()
      .extractLinks()
      .filter( -> /package/.test(arguments[0]) )
      .map( -> "https://npmjs.org#{arguments[0]}" )
      .log()

getUser("vzaccaria")

In this example we have chained together promise-based functions (like get from jQuery) and normal functions with additional parameters (like map and filter from Underscore). All function invocations receive an implicit argument that is the value computed by the previous link in the chain.

The function defined in getUser implements a processing pipeline (à la streaming). It starts with the link to be retrieved by specifying it with _():

_("https://npmjs.org/~#{user}")

This value is then passed to the get() method of jQuery. Chained knows that jQuery has a get method since we introduced its scope in the following line at the beginning of the code. The using clause extends the scope within which Chained will look for functions to be chained.

using(jQuery)

The result of get() – i.e. the webpage – is sent to extractLinks (from linklib) when the associated promise is resolved. The rest of the computation proceeds intuitively, as you can see.

If an exception is thrown in any of the links of the processing chain then the following links are not executed, just as you would expect when using the promise pattern.

We can add additional arguments, beside the implicit one, to method invocations. For example, the filter function from Underscore is invoked as follows:

...
.filter( -> /package/.test(arguments[0]) )
...

The lambda function is passed as the second parameter to Underscore .filter() function, with the first being the implicit parameter.

So we covered a few things here:

• We can chain promise-based functions and normal functions without distinction using the natural syntax of Javascript.
• We can directly use their method names instead of using then.
• We can chain any method from an arbitrary number of modules, provided that the module is bound to Chained with the using clause.
• The imported modules are not dynamically modified. No prototypes are changed.
• The chain stops executing when one of the links throws an exception or a promise is rejected.
• We are not using any CoffeeScript mumbo-jumbo for chaining; we can write with the same expressiveness in pure Javascript.

What’s the catch?

We are exploiting ECMA6/harmony implementation of introspection. You need an ECMA6/harmony implementation, either in node (node --harmony) or in your browser (Firefox is ok at the moment) to make it work.

Does it work in the browser?

Yes, check out this example Domain Specific Language for form validation directly built with Chained. Warning: it needs a recent build of Firefox.

Installing Chained

Assuming you have Node installed, you can install Chained using npm:

npm install chained

In the browser, download it from Github and include these files:

• reflect.js
• dsl.js,
• chain.js

You should be good to go.

Implementation

The idea for Chained came while tinkering with domain specific languages. DSLs provide the developer with the vocabulary of a problem domain to express a programmatic solution that could be understood by a problem domain expert. Make, SQL, CSS are all famous examples of DSLs widely used by developers.

I will not cover all the general concepts of DSLs here. For an in depth overview, I’d suggest to look at these resources:

• Debasish Ghosh, “DSLs in Action” — Manning Publications Co.
• Martin Fowler Website (and book)
• Writing Domain Specific Languages — Groovy website.
• Validation DSL for Client-Server Applications — Vitalii Fedorenko Master Thesis
• WebDSL

Internal DSLs are built using the syntactic and semantic features of another language. They are used in the context of a broader application for which they address a narrow but important problem.

My quest was to find a way to implement a DSL in Javascript. Let’s see what features are needed by a general purpose language like Javascript to support an internal DSL:

• Support for named arguments (or hashes) in function invocation (aka Smart API), e.g.:

  move(the: pen, on: theTable)

• Support for a property/method missing exception to implement method chaining:

  take('spaghetti').and().cookIt(at: 100C)

  In this example, and is a non-existing method of the return value V of take('spaghetti'). The missing exception handler will return the same value V so that cookIt could be called on it.

• The following are optional, but highly desirable:
  • • Reduced boilerplate code.
    • Synthetic expression of closures (w/ implicit arguments).
    • Limited presence of parenthesis {} and ():

  withObject chair, ->
      move it, in: theLivingRoom

At some point, it became clear to me that DSLs’ method chaining can be used to implicitly structure promise based chains. In fact, you could build a promise-chain by handling a sequence of method missing exceptions. As a byproduct, it is straightforward to use the same technique to chain normal functions without requiring a construct like then.

The roadblocks

Javascript and its relatives (such as CoffeeScript and LiveScript) have almost all the characteristics to build a sophisticated DSL. The only problem is that they lack of a method missing exception, at least until ES6 Harmony comes out.

Dylan Barrell already demonstrated that with the new reflection API, it is possible to manage missing methods. His technique exploits the concept of Proxies. Think about a Proxy like a wrapper around your original object that is able to intercept calls to methods that are not defined. Chained is based on Dylan’s work.

The basic idea

The basic idea goes as follows:

1. We create an object that can handle missing methods through an appropriate mechanism. Let’s call it o.
2. o contains a promise property. Whenever a method missing is invoked on o…

   o.m()

   …we catch it and handle it with the following handler:

   handleMethodMissing = (m) -> 
        this.promise = this.promise.then( (it) -> scope[m](it) )
        return this

   Note that:

   • We need a scope in which to look for m. In Chained, we use using(module) to specify/extend this scope.
   • m is called with the final value of the previous promise.
   • Even if m returns a new promise, this is perfectly fine since the then method is going to resolve the original o.promise accordingly.
   • We return o, so that we can repeat the same process by seemingly chaining methods that are effectively missing.
3. We need a way to fire the very first promise off. In Chained, we use the function _() both for creating o and to resolve the first promise of the chain with the value passed to it:

   _(v) = -> 
        o = new methodMissingObject()
        o.deferred = defer()
        o.promise = o.deferred.promise
        o.deferred.resolve(v)
        return o

API

The API is pretty simple at the moment, however I think it offers a good level of flexibility that can be leveraged.

chain.using(module)
Extends the scope within which all missing methods are searched. You can include multiple modules:

using(jQuery)
using(underscore)

o = chain._(value)
Returns a deferred object whose promise property will eventually be resolved to value. The object handles missing methods by chaining them using Q.

Deferred object methods and properties

Given the deferred object o, the following methods apply, keeping in mind that they return a deferred object:

o.foo(...)
If o eventually resolves to v, this is essentially equivalent to building a deferred foo(v, ...).

o._(foo)
Specify a one shot function foo to be chained with the rest, without extending the scope with using.

o._forEach(bar)
If o will eventually resolve to an array, apply bar to each element; the combined promise is created with Q.all.

o.promise
This is a Q promise corresponding to the deferred o.

This article was originally published at https://www.vittoriozaccaria.net/blog/2013/11/05/taming-asynchronous-programming-with-harmony.html