Replacing callbacks with ES6 Generators

By Matt Baker

There are a lot of articles out there discussing the use of ES6 generators to remove the “callback pyramid” we so often encounter in JavaScript. Unfortunately, most of them rely on libraries, and few tell the whole story.

In this article we’re going to go step-by-step. We’ll progressively modify a toy callback-based example towards a generator-based solution. The goal is for you to understand how the entire process works, from end to end.

Generators are a new concept in JavaScript, but not in programming. You might have used them in other languages like Python. If not, have no fear, we’re going to do a gentle intro.

How to run the examples

Before we get started, you will need Node 0.11.* to run the examples. When you run the examples, you must tell Node to run with ES6 (a.k.a. Harmony) support: node --harmony example.js.

What is a generator?

Before we dive in to how to use generators in place of callbacks, let’s talk about generators are.

Generators are a lot like functions, except you can pause their execution. You can ask them for a value and they’ll provide one, but the rest of the function won’t execute until you ask it for a value again.

 Take-a-number ticketing machines are actually a good metaphor for generators. You can ask the ticketing machine for a value by pulling a ticket. You receive your number, but it doesn’t hand you the next one. In other words, the ticket machine “pauses” until someone requests another number, at which point is advances to the next one.

Generators in ES6

Generators in ES6 are declared like functions, except they have an asterisk:

function* ticketGenerator() {}

When you want a generator to provide a value and then pause you use the yield keyword. yield is like return in that it hands back a value, except the function pauses after yield is called.

function* ticketGenerator() {
  yield 1;
  yield 2;
  yield 3;
}

In our example we’ve defined an iterator called ticketGenerator. If we ask it for a value, it will return 1 and then pause. If we ask it again, we’ll get two, and finally 3.

When you call a generator (e.g. ticketGenerator()), it returns a new Iterator object. This iterator object has a method called next which allows you to unpause your generator function and get the next value.

next returns more than just the value, it returns an object with two properties: done and value. value is what you’ve yielded, done says whether or not your generator is done providing values.

Let’s pull a few numbers from our ticket machine:

var takeANumber = ticketGenerator();
takeANumber.next(); 
// > { value: 1, done: false }
takeANumber.next(); 
// > { value: 2, done: false }
takeANumber.next(); 
// > { value: 3, done: false }
takeANumber.next(); 
// > { value: undefined, done: true }

Right now our ticketing system only goes up to 3, which isn’t very useful. We’d like it to increment indefinitely, so let’s create a loop.

function* ticketGenerator() {
  for(var i=0; true; i++) {
    yield i;
  }
}

Now, if this were a function and we were using return we’d get back 0 every time. Not so with a generator:

var takeANumber = ticketGenerator();
console.log(takeANumber.next().value); //0
console.log(takeANumber.next().value); //1
console.log(takeANumber.next().value); //2
console.log(takeANumber.next().value); //3
console.log(takeANumber.next().value); //4

Each time we call next() our generator executes another iteration of our loop and then pauses. This means you can have generators that, like ours, run infinitely. Because the generator pauses, you won’t freeze your program. In fact, generators are a perfect way of representing infinite sequences.

Affecting the generator’s state

In addition to advancing the iterator-generator object, next() actually has a secondary use. If you pass a value to next, it will be treated as the result of a yield statement inside the generator.

So next is sort of a way of passing information in to the generator in the middle of its execution. We’re going to use that fact to enhance our ticket generator so that it can be reset back to zero. We want to be able to tell it to reset itself at any point in time.

function* ticketGenerator() {
  for(var i=0; true; i++) {
    var reset = yield i;
    if(reset) { i = -1; }
  }
}

As you can see, if yield returns true then we set i to -1. The for loop will increment i at the end of the loop, so the next time it is yielded, i will be zero.

Don’t get confused. yield i sends i out of the generator, but the result of yield i in our generator is the value provided by next.

Let’s look at it in action:

var takeANumber = ticketGenerator();
console.log(takeANumber.next().value); //0
console.log(takeANumber.next().value); //1
console.log(takeANumber.next().value); //2
console.log(takeANumber.next(true).value); //0
console.log(takeANumber.next().value); //1

Replacing Callbacks with Generators

Now that we have some knowledge of generators under our belts, let’s talk about about generators and callbacks. As you know, we typically use callbacks when we’re calling asynchronous code like AJAX requests. For simplicity’s sake I’d like to define a delay function for use in our examples.

Our delay function will be asynchronous – after a certain number of milliseconds the callback you provide to delay will be executed, and delay will pass your callback a string telling it how long it slept.

In the mean time your other code will keep executing. This is just like making an AJAX request – you make the request, your code keeps executing, and your callback runs when the server comes back with a result.

So, let’s define delay:

function delay(time, callback) {
  setTimeout(function () {
    callback("Slept for "+time);
  }, time);
}

Nothing special there. Now let’s use it to delay twice. First we’ll delay for 1000ms, and when that completes we’ll delay for another 1200ms.

delay(1000, function(msg) {
  console.log(msg);
  delay(1200, function (msg) {
      console.log(msg);
    }
})
//...waits 1000ms
// > "Slept for 1000"
//...waits another 1200ms
// > "Slept for 1200"

The only way to ensure that our two delay calls are executed one after the other is to make our second delay call in the callback of the first.

If we needed to delay 12 times, one after the other, we would need to call delay in 12 nested callbacks. This is when you end up with the “callback pyramid”, and things start to get ugly.

Enter Generators

Generators are one approach to eliminating “callback hell”. Asynchronous calls are tough because our functions don’t wait for the async call to complete, hence the need for callbacks.

With generators, we can make our code wait. Instead of nested callbacks, we can use generators to pause their execution while each asynchronous call finishes before we ask the generator function to advance to the next line of code.

So, if we can pause execution while an asynchronous call completes, that means we can make calls to delay one after another – as if delay is actually synchronous.

So how do we do it?

First, we know that our code that makes asynchronous calls needs to happen in a generator instead of a typical function, so let’s define one.

function* myDelayedMessages() {
    /* delay 1000 ms and print the result */
    /* delay 1200 ms and print the result */
}

Next we need to call delay in our generator. Remember, delay takes a callback. That callback will need to advance our generator, but we don’t have one yet so we’re going to put in an empty function.

function* myDelayedMessages() {
    console.log(delay(1000, function(){}));
    console.log(delay(1200, function(){}));
}

Our code is still asynchronous. That’s because we haven’t put any yield statements in. Generators only pause when they see a yield statement!

function* myDelayedMessages() {
    console.log(yield delay(1000, function(){}));
    console.log(yield delay(1200, function(){}));
}

We’re getting closer now. However, if we run our generator it won’t actually do anything. Nothing is telling it to advance.

The crucial concept for you to understand is right here: the generator is supposed to advance when the callback provided to delay completes, that’s how it knows to unpause.

This means that whatever’s in that callback needs to know how to push the generator forward. Let’s pass in a function called resume that does this for us. Keep in mind that we haven’t defined resume yet.

function* myDelayedMessages(resume) {
    console.log(yield delay(1000, resume));
    console.log(yield delay(1200, resume));
}

Ok, our generator will take a resume function that will advance it, and we’re passing that to delay.

Now the tricky part, how do we write resume, and how does it know about our generator?

If you look at other examples of code that use generators to replace callbacks, you’ll see that the generator functions are always wrapped by another function – often something called “run” or “execute.” The purpose of these run functions is to do the following:

• Accept a generator as an argument
• Use the generator to create a new generator-iterator object that we can call next() on.
• Create a resume function that uses the generator-iterator object to advance the generator.
• Pass resume to the generator so that it has access to it.
• Kicks things off by calling next() once at the very beginning, so that our code can start executing before hitting the first yield.

Let’s build up run.

function run(generatorFunction) {
    var generatorItr = generatorFunction(resume);
    function resume(callbackValue) {
        generatorItr.next(callbackValue);
    }
    generatorItr.next()
}

Now we have a function that can take a generator function and pass it a function that knows how to advance the generator-iterator object it creates.

Note that we’re using the second ability of next in our resume function. resume is the callback passed to delay, so it will receive the value delay provides. resume passes that value on to next, so that the result of the yield statement is actually the result of our asynchronous function!

All we have to do now is wrap our own generator function with run and we should be set:

run(function* myDelayedMessages(resume) {
    console.log(yield delay(1000, resume));
    console.log(yield delay(1200, resume));
})
//...waits 1000ms
// > "Slept for 1000"
//...waits 1200ms
// > "Slept for 1200"

There we go. You can see that we call delay twice without any callback nesting. If you’re still confused, let’s explain what’s happening at a high level step-by-step.

• run takes our generator and creates a resume function.
• run creates a generator-iterator object (the thing you call next on), providing resume. Then it advances the generator-iterator one step to kick everything off.
• Our generator encounters the first yield statement and calls delay. Then the generator pauses.
• delay completes 1000ms later and calls resume.
• resume tells our generator to advance a single step. It passes the result of delay on so that the console can log it out.
• Our generator encounters the second call to yield, calls delay and pauses again.
• delay waits 1200ms and ultimately calls the resume callback.
• resume advances the generator again.
• There are no more calls to yield, the generator finishes executing.

Conclusion

We’ve successfully replaced nested callbacks with a generator-based solution. Just to recap, replacing callbacks with generators consists of the following:

• Creating a run function that takes a generator, and provides it a resume function.
• Creating a resume function that advances that generator by a single step, and passes the generator whatever value resume is called with by the asynchronous function
• Passing resume as the callback to all of our asynchronous calls. Those async functions execute resume when they complete, allowing our generator to only advance after each async call completes.

Whether generators are a good approach to handling “callback hell” is debateable, but it’s a great exercise to expand your understanding of generators and iterators in ES6. If you’re looking for a practical solution for dealing with your asynchronous code that doesn’t require ES6, consider promises. I’ve written about jQuery’s promise implementation (aka “deferred”) here.