Optimizely is a tool that lets you perform AB testing. Like all or most other client-side AB testing tools, it uses the following mechanism:
- A script tag is included on your page which alters it on load for a variation group of your users
- The variation is compared to the unchanged control group to get insights on how well it performed
The main sell of this model for our team is simple: no code changes are needed to run the tests. This lets product and marketing teams launch tests in production faster and without involvement of engineers.
There is one problem however: Optimizely usage is inherently problematic with React apps. React re-renders components every time state or store are changed. So, Optimizely changes don’t stick — they are removed by React diffing mechanism on re-render.
What’s already out there?
There has lately been a number of articles that introduced different solutions for React+Optimizely usage.
[UPDATE: Tilt blog seems to have since been taken down] Tilt engineering team designed a pretty intricate system using Store to hold experiment information. Looks like a well structured approach for client side rendering where big changes are necessary (like removing/adding components).
Shesh’s idea is less complex, easier to implement, and is querying Optimizely’s globally available Data Object directly to get active experiments. Read about it here.
Milan talks about a different angle to the problem — server-side rendered apps and using Optimizely REST API to fetch experiment information on the server. Not just React, but any server side AB testing will pretty much always require a deploy. To me though, this approach seems to under-utilize Optimizely — I would rather use its GUI for client-side AB testing or not use it at all.
A lot of good ideas here. However, it bothers me quite a lot that all of the above solutions require a deploy for every new AB test.
React/Redux + Optimizely without deploy or code changes
Here is a lightweight approach. Things to note:
- This is for small to medium UX tweaks. If your AB test is massive, then a code change might be a better path.
- The success of this method in your codebase depends on your specific React implementation. This works well with our React/Redux app, because it re-renders the top level component on every change to props and actions are used predominantly (rather than local states).
Step 1
React re-renders components when state or store are changed. If you use React, you are definitely familiar with the lifecycle methods. You can use several of these methods to call a global function which acts like a hook to trigger the JavaScript for desired DOM alteration.
What this looks like:
window.optimizelyHook would be a global function that gets called immediately after the component mounts on the page. Utilize your top level component’s componentDidMount and componentDidUpdate lifecycle methods, like so:
Step 2
In Optimizely’s variation JavaScript, define theoptimizelyHook() function. Then, perform all DOM alteration inside of this function.
componentDidMount is called once on the initial mountcomponentDidUpdate is called on every consecutive update of props
The result
optimizelyHook() will run on every props update, resulting in experiment that persists through re-renders triggered by Redux actions.
Deploy once, and use many times across multiple AB tests. 🎉