Credits: Advancing Women in Product Team, Aneesha Prakash, Product Manager at Replicon, Jo Dong, BD Manager at Stardust, and Kanika Kapoor, Product Manager at PayPal
As the critical link between business and engineering, product managers need to develop and implement suitable product analytics strategies that can be understood by professionals from a variety of different backgrounds. This task is not as straightforward as we think: are we optimizing for a strategy that is universal, or something that is more data-driven? And above all, how should the product analytics strategy align with company’s business model? Those are just some of the important questions to start with. During the February 26th discussion during AWIP’s Deriving Product Insights from Data Analytics event last Tuesday at Mode, we discussed more with panelists in the field and further brainstormed the ideas in the on site case study session.
The panel was moderated by Thuria Narayan, Senior PM at GE Power Digital, and was joined by Jenny Li, PM at Mixpanel, Simon Tan, PM at Dropbox, and Nishi Patel, PM at Mode. Here are some of the highlights from their discussion:
How do you establish a product analytics strategy and how does it change over time?
Nishi pointed out the best strategy is to think of analytics during the process of product development, not just at the beginning of it. Doing analytics incrementally helps PMs make sure they are doing the right thing and are able to fix small bugs instantly.
Nishi also encouraged the audience to think of data collection beyond the sake of a dashboard. She instead spoke in favor of sharing ownership of the data and delivering meaningful results
How does the business model and/or stage of the company influence the product analytics strategy?
Panelists agreed that the size and stage of the company matter in this regard.
“For bigger companies, you want to run experiments on everything because you can’t predict consumer behavior at that scale,” said Jenny. After all, 1% difference in the experiment outcome could lead to millions of dollars loss in revenue.
Meanwhile, for smaller companies, focusing on dozen important users and finding out what they really want are more important than A/B testing. Nishi recommended breaking the process up into a few steps. Start by collecting all possible data and use it to tackle down some initial key questions. Then to spread the data culture across the whole company and ask the “why” question behind it, and eventually being able to predict the consumer behavior.
What data should you look at, Quant or Qual? How to define meaningful metrics?
Simon actually emphasized that they are equally important. “You use qualitative data, such as scalable consumer feedback, to figure out the trends and seek proof from quantitative data to form the entire picture,” he said. As the PM from Dropbox, he added that quant data is much easier to scale.
Jenny analyzed it from the perspective of cons. “You only get things that can be tracked in quant data,” she said. Qual data does add an extra dimension and help people understand what they are thinking. However, it is also hard to scale. More importantly. Opinions don’t always match actions.
Thus, a mix of both is probably the best bet.
How should a company or team reach the evaluation of the right tool to use?
Panelists agreed that it is based on the stage and resources of a company. For startups, it is easier to get started with the third party services or self-served tools. For big companies, building an in house team to focus on solving their specific questions is the better way.
To watch the full panel discussion and case study session that was hosted by AWIP, checked out our Facebook page.
Stay updated with our future events:
03/21: Amplifying Your Voice as A Women In Tech
03/26: Using Technology to Change Behaviors and Transform How People Live