I recently published The 3x Conversion Playbook to help you level-up your conversion work and get more customers. In it, I reference learning tools to better understand your customer behavior and customer information.
Learning tools are important as they answer key questions: the ‘what’ and ‘how’ of user behavior. ‘What’ customers are doing can be studied using quantitative tools like Mixpanel and Google Analytics. ‘How’ they act can be determined using qualitative tools such as FullStory and AppSee.
The answers to these questions will help you identify ways to unlock new growth from your current products.
In this article, I’ll lay out my seven favorite tools and how to properly use them, complete with specific examples of how they helped improve our business.
Quantitative Tools: Understand ‘what’ your users are doing
Analytics tools help you define how users interact with your website and product. This is a great starting point for identifying patterns and behaviors.
Mixpanel**: Granular tracking**Your funnel is the process of people moving from first touch point on your site/product to purchase (or other key action). Start by defining your business goal, i.e. getting a customer to purchase a pair of shoes, or in our case booking a tutor. Create a funnel inclusive of all steps it takes to get someone to buy the shoes.
By tracking each micro-step, you will be able to size the opportunity for making changes at each step of the process and get a better understanding of your customer base, leading to more informed data-driven hypotheses.
Quickly identify steps with massive user drop off — this is that low hanging fruit everyone talks about.
_Mixpanel Example_MathElf (Rev’s on-demand math tutoring business) had a high drop-off on signup. Our goal on this step was to increase the number of people who signed up after downloading the app.
We added tracking in the app for every screen users had to navigate to before they successfully registered. By studying the funnel at each step, we realized there was a significant drop-off on the password-setting step, which had an optional field for referral code.
Actual Funnel Steps for Registration (pre-Growth work) & Screenshot of Step 9
We took this information and then directed the qualitative tool AppSee (see below) at this problem. By watching user sessions, we realized that people were entering random referral codes or exiting the app to search the internet for referral codes.
This data informed our user testing with Validately (see below) and speaking with customers. We realized that our core customer base (students) believe that “referral code” meant “discount code.”
The original intent for the referral codes was for tutors to refer offline students and bring their tutoring sessions online through our app, not offering discounts. The feature had decent adoption, but we hypothesized that removing the screen would help more than it hurt, and we killed it.
It worked. By removing the option, the number of people completing this step increased 20%.
Google Analytics (GA): Website usage dataImplement GA on your main landing pages to see tons of data:
- Where are people coming from?
- What pages are they going to?
- What device are they using?
- Is this content relevant to them?
Look for outliers in bad performance that drive people away from your site, including having high bounce rates or exit rates. These pages need to have a deeper look.
GA Example: Rev’s mobile site had a high drop-off on our transcription landing page. Our goal was to increase the number of people who bought after visiting the site.
We already had GA on the site, so we dug-in to the metrics and realized that there were two big problems: bounce rate and page load time. I knew these were problems because they were higher than other pages on our site and industry averages for similar pages, which I learned from asking peers at similar companies. While metrics are specific to the company, you should be aware of outliers from industry averages.
Bottom line: We were taking too long to give customers the relevant information.
To remedy the problem, we simplified our desktop site for mobile devices and focused our energy on improving the page performance. If it didn’t help performance, we took it out. I’d go into specifics, but our engineering team did the heavy lifting here.
It worked. By making the page more lean, we improved this step by 20%.
Google Optimize: A/B TestingLet me start by saying A/B Testing is not for every change and not required to see improvements in conversion.
- Test design, copy, and layout changes on high-volume pages.
- Low volume pages will never get the amount of data you need to be successful (topic of future post).
- No need to test obvious improvements, such as faster page load speed.
Once you have decided to test, I suggest using a tool like Google Optimize to manage and monitor the test. If you use Google’s other products on your site (AdWords, Google Analytics), it is incredibly simple to set up.
The beauty of a tool like Google Optimize is that it manages everything in one place and tells you when tests are successful or not by doing the math on statistical significance for you.
Optimize Example: While we’ve never A/B tested low-volume pages, we have done A/B tests that have fallen flat.
As Growth people, we know the Above-the-Fold (portion of a webpage you see when you visit it before scrolling) is sacred ground. I hypothesized that removing the sub-navigation would remove wasted space and free up valuable real estate. This was an early test that I was extremely confident on. I was eager to make the change without testing. In the end I’m glad we didn’t, because I was very wrong.
Subnavigation on Rev.com’s Transcription Landing Page
By removing the sub-navigation, conversion went down by 10%. We did not make these changes to the site and moved on to the next test.
Note: Click ‘Follow’ for future examples of successful A/B tests
**In-house data**As great as all 3rd party tools will tell you they are, never swear your life by their numbers. I have, and it really hurt when they were wrong.
- Look to immutable numbers, such as revenue, to substantiate your other tracking.
- Sanity check the data.
- Be paranoid about your data.
This is most important when starting a new project that you are not entirely familiar with. It will be tempting to trust all the data sources. Don’t.
Example: Too many to count. Instead, I’ll tell you why the tools aren’t perfect. Mixpanel can’t get past website ad blockers. GA can’t tell you granular data on speed performance. Optimize is only useful as a tracker, not for understanding problems.
Qualitative Tools: Understand ‘how’ your users are behaving
After identifying where the opportunity is, use these tools to learn more about specific steps in the funnel.
FullStory**: Watch real people use your website**Conversion funnel in mind, watch how people navigate your product. Understand what they do and don’t read, what they do and don’t click on, what may be confusing and cause them to scroll back.
Although large-scale data collection is invaluable, there’s no replacement for seeing how individual users navigate through your site or app.
I’ve easily watched 50+ hours of user sessions on FullStory, and been all the better for it.
FullStory Example: We wanted to improve Temi’s landing page. Our goal was to increase the number of people who placed a trial order.
After watching hundreds of FullStory sessions, I learned that many people were confused by the navigation of the page and thought that the information they were looking for lived in other parts of the site. The biggest question they had was about the quality of the automated service for their audio file, and we were not doing a great job of informing them of our high-quality offering. Because of this, visitors were searching for FAQs and examples.
_To remedy, we simplified the navigation and put the key information where customers needed it most, above the fold on the landing page. We did this through copy ‘_Higher quality audio improves results’ and through a pop-up interaction once they clicked the main button to upload files.
Old Temi Page: Overwhelming Navigation and no mention of Transcript Quality
New Temi Page: Simplified Header and mentioning Transcript Quality
It worked. We increased the number of visitors submitting trials by 33%.
AppSee**: FullStory for mobile apps**FullStory is an incredible product, but they do not work for mobile apps. We use AppSee for apps because it offers high caliber qualitative app analytics..
AppSee also has touch heatmaps and crash reporting, which are vital for improving app usability and retention.
Intercom**: Talk with people in real-time**Chat tools on websites and products are changing the internet for the better. Intercom is our tool of choice (besides Drift, Zendesk, LiveChat) due to their incredible focus on the customer and being able to tie in with support.
Intercom greatly reduces the barrier to speaking directly with your customers and gain insights from them. Additionally, you get to speak with them while they are engaged and on your website.
- Put Intercom everywhere you can, especially on your main pages to interact with the greatest number of customers.
- Handle the requests yourself, do not immediately offload to support
- Look for ways to address common questions and requests in the product
Intercom Example: We wanted to improve Rev’s landing page. Our goal was to increase the number of people who bought from us.
We added Intercom to the page and I proceeded to chat with thousands of potential customers over the next couple of months, eventually needing to bring in our support team to handle the workload. Through these interactions, I learned the key information that they were looking and where the page was lacking.
After turning the key information into landing page content and testing with Validately (see below) to ensure the content served our purpose, we used Google Optimize to A/B test the new changes.
It worked. We increased the number of customers from landing pages by 60%.
Validately**: Remote user testing**Before making live changes to your product, which is expensive in terms of engineering, time, and effort, you want to make sure your tests will accurately prove or disprove your hypothesis.
It is cheaper, easier, and faster to figure this out before pushing live code.
You can accomplish this with user testing, both in-person and remote. We do both. Our remote tool of choice, for both moderated and unmoderated testing, is Validately.
Validately lets you remotely test with your own customers or they will find some for you at $15 per tester.
- Test with your core audience (by filtering tester demographics), not random people who would never use your product.
- Get seven testers, which amounts to $105 per test. It is a small cost that is best paid up-front vs the extreme cost of a non-conclusive test.
- Push on any doubts or questions you have about the test.
- Do not get tricked into trusting the quantitative feedback with this small sample size.
Validately Example: We wanted to improve Temi’s landing page. Our goal was to increase the number of people who placed a trial order.
Based on learnings from FullStory, we knew people were searching for information elsewhere on the site and wanted to bring it all to the homepage.
We wanted to know that calling out audio quality on the homepage and changing the headline to more specifically describe the service would help visitors understand our service offering and address their concerns. We put up an unmoderated test for people who fit our target demographic and asked key questions:
- What are your first impressions?
- What stands out to you?
- Who is page this for?
- What can you do from this page?
After watching the sessions, we were satisfied that the new header and headline would do a better job of explaining audio quality
It worked. We decided to move forward with the test that increased the number of visitors submitting trials by 33%.
Summary
These tools will help you identify areas of opportunity and track the progress your team is making to improve the customer experience.
Disclaimers:
- No tool does everything on its own. You must use them in concert.
- There are many tools out there. You may find great alternatives to these specific tools, but the intent behind them stands.
If you are working on Growth or have questions about the above, feel free to reach out: Barron.Caster@gmail.com.
To see successful A/B test examples, deeper dives on these products, and future posts on Growth, click “Follow” to subscribe