AB testing bundles with webpack

In this post I want to show you a quick way to get some basic AB testing functionality in place via a webpack loader.

All code can be found at my github repo.

The basic page

Lets say we have the following page running in production:

The App component looks like this

import React from 'react';
import Button from './Button.jsx';

export default () => (
<div>
  <ul>
    <li>Item 1</li>
    <li>Item 2</li>
  </ul>
  <Button />
</div>
);

Now our designer decides that it might be best for conversion to have a blue button that alerts onClick instead.
This change is easy enough to implement, but will it really help?

To test it we should roll out two seperate versions of our app. BUT the only thing that will differ is this component.
So we can just tell webpack to bundle everything in the same way, except this one component.

The different components

Here is the code for the two different buttons:

Old Button

JS at Button.jsx

import React from 'react';
import styles from './Button.css';

export default () => <button className={styles.button}>Button</button>;

CSS at Button.css

.button {
  color: green;
}

New Button

JS at Button.b.jsx

import React from 'react';
import styles from './Button.css';

const onClick = () => alert('clicked');

export default () => (
  <button onClick={onClick} className={styles.button}>
    Button
  </button>
);

CSS at Button.b.css

.button {
background-color: blue;
  color: green;
}

Notice that the .b. is added between the filenames and extensions for the new component and its styles.
Now we can use a loader that will, given that a variable was set, exchange all *.jsx and *.css files with *.b.jsx and *.b.css whenever they are imported.

This means that our component tree will stay exactly the same, except when we want to exchange a component to test it out.

Webpack Loader

I have written a loader to handle the file switching during build time. You can find it at npm or github

The documentation on writing webpack loaders can be found here.

In order to get webpack to do what we want we need to install the loader and add it into our configuration like so:

npm i -D file-switch-loader


  rules: [
    {
      test: /\.jsx?$/,
      use: [
        'babel-loader',
        {
          loader: 'file-switch-loader',
          options: {
            version: 'b',
          },
        },
      ],
    },
    {
      test: /\.css$/,
      use: [
        'style-loader',
        {loader: 'css-loader', options: {modules: true, importLoaders: 1}},
        'postcss-loader',
        {
          loader: 'file-switch-loader',
          options: {
            version: 'b',
          },
        },
      ],
    },
  ],

Note that the loader has to be the last in the array. Otherwise the functionality of the other loaders will be skipped on the switched out file.
You can also see, that the version we pass in is b, which corresponds to the addition we have made to the filenames.

If we restart our webpack-dev-server or rebuild we can now see that the correct new button is implemented.

What is missing?

One thing that is still missing is the ability to track the impact of the change.
If there is some demand I will write an article about that as well.
But if youd like to hack around yourself try to create an exchangeable config file for your tracking, that will send data to different endpoints, depending on which version is running.

You can also use ENV variables in your webpack config to align deployment and building, so that all these things can be automated and you can run as many different versions as you like.