Browserify and dependency mocking

Update (2019-04-11) - After exploring various alternatives, I wrote a new Babel plugin to solve the problems outlined in this blog post.

Browserify is a tool that enables a collection of Node.js-style modules (aka. CommonJS) to be bundled together into a script file that can be loaded in the browser.

This post explores the structure of the JavaScript bundles that Browserify generates and how the proxyquireify package modifies the bundle to support mocking of require calls in tests. It assumes that you are familiar with creating modules in Node. Towards the end I explain some of the caveats with Proxyquireify’s approach and briefly explore alternatives.

Although many projects use other bundling tools instead, such as Webpack, I think it is useful to learn how Browserify works since it inspired many later projects.

Usage of browserify

First, a quick recap of Browserify usage:

npm install -g browserify
browserify src/app.js > bundle.js

These commands will read a Node.js-style module, src/app.js, discover all of its transitive dependencies, and package them into a single JS file, bundle.js, which can be loaded in the browser using a <script> tag.

Anatomy of a Browserify bundle

The high-level structure of a Browserify bundle looks like this:

(prelude)()(module-map, cache, entry-points)

prelude - This is a function that creates the machinery needed to evaluate modules. It creates the require function that enables modules to load dependencies and a cache to store evaluated modules. The result is a function which is then passed the module-map, initial cache and entry-points to begin the process of executing the code from the entry modules. The prelude is defined in the browser-pack package. Note that it appears in a minified form at the top of Browserify bundles, even if the bundle’s contents are not minified.

module-map - A map of module-id to [module-function, dependencies] tuples.

module-id - A numeric identifier for the module, unique within the bundle.

dependencies - A map of module path to module-id, for require calls that appear within the module’s code.

module-function - The original module code, wrapped in a function which isolates the module’s code from the rest of the bundle and exposes the require function and module and exports objects to the module’s code. It is conceptually similar to Node’s module wrapper:

function (require, module, exports) {
  // Original module code here
}

The wrapper function prevents variables in the module code from being exposed to other modules; only the module’s exports are available to them. It also enables modules to be evaluated only on-demand when required. Some alternative packaging tools avoid this wrapper function to reduce the size of the generated code.

cache - A map of module-id to module-object for modules that have been evaluated. When require is called for the first time with a particular module-id, an empty module object is created for that module and the module’s code is evaluated, which populates module.exports. The resulting module object is then stored in the cache. This ensures that the module is only evaluated once, and that if different modules require a particular dependency, they will get the same object back.

module-object - The object that is exposed as module when the module is evaluated. The module’s exported functions/constants are stored on the exports property of this object. When other modules require a dependency, they get the value of the exports property back.

entry-points - An array of module-id s, listing the modules to evaluate immediately when the bundle is loaded. The final part of the prelude function calls require(module_id) for each ID in this list.

Mocking modules with proxyquireify

In tests it is useful to be able to load a module under test and replace certain require’d dependencies with replacements. For example, we might want to replace a module that makes API requests with a fake that returns dummy responses.

Proxyquire is a tool that does this. It comes in three flavours: proxyquire is a package for Node. proxyquireify is a Browserify plugin for use in the browser. There are slight syntactic differences in how they are used. proxyquire-universal is a Browserify plugin which makes it possible to use proxyquireify (in the browser) in the same way that you would use Proxyquire (in Node). This is useful if you want to be able to run the same test suite for a set of Node modules in the browser or node itself.

How proxyquireify works

proxyquireify operates by replacing the default prelude function in the bundle with one that defines a different require function. This replacement require function can return mocked versions of modules if called in the context of a proxyquire call. Outside of a proxyquire call it behaves the same as the standard require.

It can be used as follows:

var proxyquire = require("proxyquireify")(require);
var moduleUnderTest = proxyquire("./a-module", stubs);

When the proxyquire call in this code is evaluated:

• A temporary empty module cache is created in the proxyquireify module. When this cache exists, it is used instead of the default module cache by require calls. The temporary cache prevents a require call from returning a real dependency if a mocked module was already loaded before proxyquire was called.
• The module under test is evaluated and the result is stored in the temporary cache.
• When any modules are required during the proxyquire call, they are either fulfilled by the stubs passed to the proxyquire function, or loaded from the original bundle if no stub is registered. The resulting module is stored in the temporary cache.
• Just before the proxyquire call returns, the temporary cache is set to null, which means that subsequent require calls behave normally and use the standard module cache.

An important caveat here is that because a temporary cache is used during the proxyquire call, all required modules, whether they are stubbed or not, will be re-evaluated. This can cause surprising issues with code that depends on the identity of exported objects, or if a dependency has side-effects when evaluated.

As a simple demonstration of problems this can cause, here is a set of modules which define a custom date class and a function we want to test, currentDate, that should return an instance of that class:

// custom-date.js
class CustomDate {}
module.exports = CustomDate;

// date-util.js
const CustomDate = require("./custom-date");

function currentDate() {
  return new CustomDate();
}
module.exports = { currentDate };

// date-util-test.js
const { assert } = require("chai");
const CustomDate = require("./custom-date");
const proxyquire = require("proxyquireify")(require);

const { currentDate } = proxyquire("./date-util", {});
assert.instanceOf(currentDate(), CustomDate);

Let’s bundle the test as if we were going to run it in a browser, but we’ll run it in Node for convenience:

browserify -p proxyquireify/plugin date-util-test.js | node

The assert check surprisingly fails:

AssertionError: expected {} to be an instance of CustomDate

The reason is that the custom-date.js module gets evaluated twice, once in date-util-test.js and once during the proxyquire call. Each time it returns a different CustomDate object, so the instanceof check fails. In the normal application, the custom-date.js module would only be evaluated once and so every require('./custom-date') call would return the same object.

A workaround for this is to pass through the “real” dependency when we call proxyquire:

const { currentDate } = proxyquire("./date-util", {
  "./custom-date": CustomDate
});

This ensures that when require('./custom-date') is evaluated during the evaluation of the module being proxyquire’d, it returns the same object that it would outside of the proxyquire call.

Aside from confusion caused by modules being evaluated multiple times, there is also a performance overhead to this. Re-evaluating dependency trees with a lot of code many times during the execution of a test suite will slow it down.

Alternative approaches to dependency mocking

Given the caveats mentioned above, what are the alternatives? Here are a few that I am aware of:

1. A simple solution is to require the module to mock in the test, which will return its module.exports object (or an equivalent object if using ES6 exports), and temporarily re-assign properties on this object during the test. See this Stack Overflow answer.

2. Use a compiler plugin to automatically rewrite the code of the module being tested to provide hooks that can be used to temporarily replace imports. This is the approach taken by babel-plugin-rewire. A caveat here is that the rewriting may add quite a lot of code to your bundle, thus slowing down the test runner. You may also run into issues where the rewriting has unexpected interactions with certain code or other compiler plugins being used.

3. Use a compiler plugin to automatically rewrite the code of the module you want to mock, to provide hooks that can be used to temporarily replace exports during tests. This is the approach taken by babel-plugin-rewire-exports. A disadvantage of this approach is that it only works for modules which are processed by the JavaScript compiler (eg. Babel) that you are using.

4. Modify the code being tested to support injecting dependencies, eg. via function arguments, constructor arguments or an object somewhere with properties that you can modify during test execution. This has the advantage that it works in any environment and does not require any particular tooling. The downside is that it has to be done manually for each function being tested. In the context of a team of many engineers maintaining a codebase, you’ll need to come up with conventions to ensure consistency.

5. A variation on (4) is to use a dependency injection library such as InversifyJS. This makes it easier to do “manual” injection on a larger scale and more systematically across a large codebase.

6. Use a test runner such as Jest which provides built-in support for mocking. The downside is that your mocking approach is tied to the test runner. In the case of Jest, it runs in Node and not in the browser. This may or may not be a blocker for your use case.

Looking at the above options, there is a trade-off between the amount of change (or design for testability) required from the code to be tested, and the portability of the solution to work in different JS environments. The automated approaches can be very convenient to use, especially if you are dealing with an existing codebase not written with testing in mind, but they may also have significant caveats which you need to be aware of.