I used to average about 10 sign-ups per day. But, during a 12-day period, I averaged 2,083 sign-ups per day — a 20,730% growth in sign-ups.
My email list for Design for Hackers went from 5,000 to 30,000 during that 12-day period.
I did it by NOT A/B testing. If I had concentrated on A/B testing, maybe I could have bumped my respectable 9% conversion rate up to 10%. I would have gotten 11 sign-ups per day instead of 10.
To get the same results I got by NOT A/B testing, I would have had to increase my conversion rate by 20,944%. In other words, I would need to get 19 sign-ups for each visitor.
Even if I had doubled or tripled my conversion rate through A/B testing, it would have taken me years to reach 30,000 subscribers.
What did I do instead of A/B testing? Instead of spending my limited resources coming up with variations, designing and implementing tests, then constantly checking the results of those tests — while running the risk of being completely mislead by those results — I instead used my mind to come up with something appealing, original, and powerful.
The thing that grew my subscriber base was an email course formula that used a “viral loop” to drive sign-ups. Since then, I’ve advised numerous other people on using this formula, with similar success. noah kagan added 10,000 sign-ups with his Summer of Marketing, Militza Maury added 20,000 sign-ups and landed a book deal with How to Become a Morning Person, and Josh Doody turned his Fearless Salary Negotiation into a book that debuted at the top of his Amazon category_._
I used my knowledge of human behavior to design something that could have never been produced through A/B testing.
If you’re asking What does that have to do with NOT A/B testing? The fact that you think that is really the point. You’re tempted to think of your brainpower as an infinite resource: Why not do the email course AND A/B test? But your brain doesn’t work that way.
Each thing you choose to concentrate on prevents you from concentrating on some other thing. It’s called Opportunity Costs. Worse yet, you’re hardwired to be totally unaware of it. Daniel Kahneman calls it the What You See is All There Is bias.
I can’t deny that A/B testing, when used correctly, in an organization that truly has the resources, can be a profitable endeavor. More often, it’s individuals or small startups that really can’t afford to be A/B testing, and are prone to being mislead by the results.
Instead of wasting your precious mental resources on A/B testing, learn to use your point-of-view, your understanding of the world, and your creativity to make something that brings you not just incremental growth — but explosive growth.
I don’t A/B test a thing for my podcast, Love Your Work. It’s just real conversations and lessons about carving out success by your own definition. Start with the interview with Jason Fried, or Subscribe on iTunes.