How to define Product Goals & Metrics?

Setting goals for product features is an art in product building. A product is successful only if it achieves intended objectives or goals. Defining and measuring the product’s or its features’ success need a good understanding of company goals, business, product vision, strategy, roadmap, its users, end customers, and partners. Hence, it is also a standard question in PM interviews at all levels, including the Product Execution Interviews (at Facebook).

Here is a step-by-step guide about setting product goals and their success metrics before you build any product. It is also a treasure for PM interviews.

Goal Metrics Interview Problems :-

How would you go about setting goals for Instagram hashtags?  
How would you measure success for IG stories?
How would you set goals for Instagram Reels? 
What goals and success metrics would you set for buy & sell groups?
How would you set a goal for Facebook Lite?
What top 3 metrics would you track for Quora?
What should Uber eats goals be for the next 12 months?
What would be the goals & success metrics for video conferencing app (Hangouts, Zoom)?

So what does product success look like?

A product goal describes a long-term objective or future state of the product. Product metrics are needed to track the success of the product against its goals. These metrics help to identify existing gaps, plan a roadmap and guide the prioritization of the next features and enhancements. Metrics are extensively used during

• Leadership approvals (business case)

• Product launches (as success criteria)

• Annual, quarterly plannings (as KPIs - Key Performance Indicators or OKRs - Objective Key Results or OSTMs - Objectives Strategy Tactics & Metrics)

• Sprint exercises (as acceptance criteria)

To measure the success accurately, goals need to be SMART - Specific, Measurable, Attainable, Relevant, and Time-bound.

What are the different classifications of metrics ?

To measure the success of a product, three types of metrics are used -

• Business metrics like revenue, profit, efficiency
• User metrics like usage from various users
• Platform metrics like page load time, supply-demand matching, i.e., bookings

Metrics are also classified as -

• North Star Metrics (NSM or Key goal metrics) to measure the critical product or business value - GMV, Sales, or Daily Active Users
• Product Tracking (or Product health metrics) that support Northstar metrics and measure deeper product health like number of Users or number of likes, comments, and shares
• Guard-Rail (or Counter metrics) that are measures of trade-offs, risks, or possible cannibalization of another feature or metric e.g., NPS, CSAT

Moreover, leading metrics (leading indicators) are those which are forward-looking and can represent future outcomes. In contrast, lagging metrics (lagging indicators) represent current and past state and are not good predictors of the near future. E.g., When measuring sales, Revenue per user is a leading indicator while churn rate is a lagging indicator. Dropping revenue per user is an urgent sign of sales reduction, while a high churn rate of customers over a longer period will eventually impact sales.

Now, let’s discuss how we set goals and metrics to define a product’s success.

1) SITUATION

It is important that you understand the product, company, and problem space. Ask relevant questions to clarify the situation and develop a common understanding.

CLARIFYING QUESTIONS TO ASK :-

What is the product and the company ?
What is the stage of the product in its lifecycle?  
Should we focus on any geography or think globally ?
What is the competitive landscape ?
Is there a particular company strategy to build new goals ?
What are the key user values and actions ? 
Do we have any particular customer segment or situation like Covid I should focus on?
Is there any specific time frame for goals ? 

Note:- Not all questions need to be explicitly asked. You may make some relevant assumptions and state the same. E.g., if the product is Facebook Live, you can mention - “My understanding is that Facebook Live is a matured product and is aimed at driving engagement for global audience.”

2) MISSION

Reiterate the mission and vision of the company and/or product. This is helpful in building the right goal strategy; say - “Before I build an approach, let me touch upon the mission for Facebook ”

MISSION  :-
Facebook - Make world more connected and help people build communities
Instagram - Allowing people to capture and share moments of their life
To bring you closer to the people and things you love
Google - Organize the world's information & make it universally accessible & useful
FB Watch - allows users to view videos from friends, groups & thus bring them together

3) USERS

Product goals are tightly coupled with the users in almost all modern businesses. Identifying different users and their journey help to build a comprehensive list of user metrics and possible goals.

Users can be Supply-side - Demand-side; Primary users - Secondary users; Internal users - External users; Buying customers or Users using the product.

E.g., Instagram’s users are

• Content creators
• Influencers or Celebs
• Content consumers
• Advertisers or Brands

User Journey :- Discover > Interest > Desire > Reach > Find > Access > Learn > Test > Use > Complete > Validate > Share

E.g. User journey of a traveler:- Deciding Destination > Plan Itinerary > Activities > Express and Share memories

USERS :-
Artists, Content creators, Advertisers, Young Adults, 
Parents, Kids, Elders, Students, College goers, Religious, 
Working professional, Traveler, Affluent, Price Conscious, 
Risk Averse, Influencer, Follower, Volunteers, Activists, 
Conservatives, Digital Nomad, Hiker-Traveler, Educators

Detailed user segmentation is generally not necessary, but it is still best to ask if goals should be for any specific segment - read segmentation.

4) METRICS

Now that you have better clarity about the mission and the users, let’s build a comprehensive list of relevant metrics.

Generally, metrics are measured across the entire user funnel -

Product Funnel

• Awareness
• Acquisition
• Adoption
• Conversion
• Engagement
• Retention
• Monetization
• Satisfaction

At least two or three metrics should be identified across the funnel so that we can accurately prioritize the most appropriate metrics in the next step. It is also important to use an appropriate time frame for these metrics, which generally depends on the company’s performance review cadence (weekly, monthly or quarterly).

The metrics should be measured across all relevant users, supply, demand, business, and platform (whatever applicable) - consider 2X2 of these against the funnel metrics.

E.g., For Airbnb, the key metrics are

LIST OF METRICS :-

ACQUISITION OR REACH - User Traffic or hits ( direct, organic, referral, promotional )
, Bounce rates

ADOPTION - Install/downloads, Unique Users, Sign ups, # Sessions/DAC, 
%age of new users

ACTIVATION  - No. of groups, no of groups/users, Bookings/user, 
Avg days for first 3 bookings (measures key event after adoption)

USER GROWTH - channel traffic, new users, repeat users

ENGAGEMENT   
- %age of users using the feature on daily, weekly, monthly basis - DAU, WAU, MAU  
- Activity or User actions - creations-posts, click, session, view, like (passive), 
comment(active), share, subscribe,  session length (in terms of users or session)
, notifications viewed/ctr, card abandonment rates, #messages responded or viewed
- Response rate, response time, TAT 
- Avg reach of content - Number of views per post, Ad views per user

RETENTION - DAU/MAU, cohort - D7, D30, weekly repeat active users

STICKYNESS - DAU per MAU, x actions per month

CHURN - Not active for X days, not renewing

COVERSION - to goals like subscription, check-ins, Customer Acquisition Cost (CAC)

MONETISATION -  Gross Merchandise Volume (GMV), Revenue, CTR, Avg Revenue/User (ARPU)
Customer Lifetime Value (CLV), Subscriptions/conversions, Revenue per Ad

SATISFACTION - CSAT, SUS, NPS, User ratings, Referrals

Note:- Many of the metrics are heavily inter-related e.g., Facebook makes revenue via ads (monetization), and ad revenue is directly proportional to the engagement and conversion metrics of users like clicks, subscriptions, and also their retention on the platform

5) PRIORITIZE

In the previous step, we defined various metrics for each funnel stage that indicate movement towards multiple possible goals. Now, it’s time to build a goal strategy by prioritizing a goal and selecting the most relevant and impactful metrics.

Goal Strategy

First and foremost, the product lifecycle stage (new, growing, mature) is used to identify which part of the funnel to focus on as a key goal. Select the most appropriate goal from the funnel (awareness, acquisition, adoption, engagement, conversion, retention, monetization, satisfaction) which complements the company and product strategy.

For e.g. -

• If the product is new or is being launched, the primary goal is generally adoption (number of users). Hence, focus on acquisition, conversion, and engagement metrics instead of retention metrics.
• If the product is in growth stage, the goal is generally to increase engagement (e.g., Instagram reels), then the focus can be on engagement and retention funnel stages which are closer to the goal.
• For matured products like Facebook, the goal is mostly engagement (most important), retention of its huge userbase, and monetization.

Prioritized metrics

North Star Metric is the most important broad indicator (leading metric) to track the goal. It indicates whether the product is benefiting the users as well as impacting the business (i.e., satisfying core product value). Based on the strategy, different companies, even in the same industry and lifecycle, optimize for different goals and metrics. For the interview, it is recommended to specify one North star metric, and two to five supporting metrics that relate best to your identified goals. It is also advisable to provide a couple of counter metrics or guard rail metrics to track any counter affects and risks associated with the product or feature.

E.g., For the Instagram Reels feature, the company’s goal is engagement, and it is measured by the primary metric (North Star Metric or NSM) of Daily Active Customers actively engaging with Reels.

• North Star Metrics - Daily Active Customers, engaged with Reels (with >15 secs view time)

• Product health metrics - No. of User Engagements on Reels (likes, shares, comments), No. of Reels Created, No. of Customer Views, $ Revenue from ads

• Counter Metrics - User Session time or Time spent on Reels vs. other areas of Instagram

NORTH STAR METRICS :-

Instagram - Weekly Active Customers (engagement) 
Facebook - Monthly Active Customers (engagement) 
Uber - Bookings completed per week (conversion) 
Airbnb - Nights booked per week (conversion) 
Quora - Number of questions users answered (engagement) 
Whatsapp - Number of messages sent weekly (engagement) 
Youtube - Total engaged watch time (watch time >10 secs) (engagement) 
Netflix - Number of subscribers with watch time >1 hours in the month (engagement) 
Salesforce - Number of records created per account (engagement) 
Adobe - Revenue from subscribing customers (monetisation) 
Walmart - Purchases per week (conversion) 
Amazon - Purchases per Prime subscribers (custom strategy) 
Prodbee - Number of page views (acquisition)

Pro Tips :-

- Keep the prioritized metrics as simple as you can

- As companies evolve from new to growing and matured, their goal shifts in the funnel from Acquisition to Monetisation and Satisfaction

- New products have simple metrics around customer acquisition; mature products have more detailed/custom metrics like number of engaged users >30 sec watch time

- Do not use the vanity and non-actionable metrics

- Try to use absolute metrics than rates to avoid the base effect

- Do not forget the timeframe of measuring the metric (e.g., no. of new sign-ups weekly)

- Avoid competition benchmarking metrics like market share unless necessary. For e.g., new business launch

- You can also measure metrics as target vs. actuals like quarter to date

Tradeoffs

It is recommended to critique your metrics by discussing any broader risks and foreseen fallback. You can discuss the counter metrics to highlight cannibalization, overuse, and customer satisfaction issues. E.g., If a product feature is newly launched, it is important to closely track the customer reactions on social media, customer support issues, bugs, resolution time, and various platform performance indicators like downtime, # bugs, average load time, etc. Tracking edge-cases or failure rates such as no matches, no results page, and inactive users are also key metrics to measure the success of products depending on their maturity and business relevance.

Note:- In interviews, goal setting question is often followed by another Product Execution Question like -What would you do as PM if this product metric X goes down by Y %?

(see Solving a Product issue )

 "Happy Product" means "Happy users' & "Happy numbers"
Still have questions or need any help on Goals and Metrics ? Drop a comment
If you liked the article, please support by sharing this story or https://prodbee.com/

Also Published Here