A propensity to hope and joy is real riches; one to fear and sorrow real poverty. — David Hume
Abstract
Marketers invest a lot of time talking about the significance of getting the correct messages to the perfect individuals at the perfect time. Notifying or Emailing when the user is not interested may cause many users to turn off app notifications or report emails spam which blocks all future communications.
Marketing comes at a cost both financial and user experience. If there are 100k users on the platform it is wise to put in effort for only a subset of users who might be interested to purchase/converting.
The best way to identify who among your audience is most likely to actually make a purchase, accept an offer, or sign up for a service is a propensity model. Let us understand the propensity model better by working on a problem statement: Build a propensity model to determine if a user will purchase on their return visit.
Goal
Objective
- Propensity model to figure out the likelihood of a person buying a product on their return visit.
- We need to identify the probability to convert for each user.
Business Outcome
- A propensity score would help segment users and run different marketing campaigns so that the conversion rate increases.
- We may share mobile notifications or emailers with exclusive offers to high propensity users helping businesses with conversation rates.
Approach
What is a Propensity Score?
- A propensity is a natural tendency to behave in a certain way. We all have propensities — things we tend to do.
- Propensity modeling attempts to predict the likelihood that visitors, leads, and customers will perform certain actions.
- The propensity score can be defined as P( Target = 1 | Z ) where Z is the user’s behavior characteristic features on the website.
Feature Required
Data Analysis
Analyze your data understanding how many % of the positive class (user buy on a return visit) and negative class (user doesn’t buy on a return visit).
- Positive Class (1) i.e. user buy on return visit: 1.53% [High Propensity Customer]
- Negative Class (0) i.e. user doesn’t buy on a return visit: 98.47% [Low Propensity Customer]
Metric Selection
For our Marketing use case to improve conversion rate:
Cost of False Negative (marking High Propensity as Low) > Cost of False Positive (marking a Low Propensity Customer as High)
Hence our Metric should be such that: Recall is more important than Precision
A beta value of 2 will weigh more attention on recall than precision and is referred to as the F2 measure.
F2-Measure = ((1 + 2²) * Precision * Recall) / (2² * Precision + Recall)
Model Training
The propensity model is a binary classification problem, we would be using a Logistic Regression for our model.
Model Training Schema
Model Output
prob: is logistic regression probability of an event occurring, in our case event is user buying on a return visit or not.
Experimentation
We ran 3 different feature set experiments with logistic regression and found 2nd to be performing best on our metrics.
VISUAL of Model Evaluation (Best Model: 2nd in above Experiment Table | Positive Class Threshold: 0.0217)
The best threshold for positive class = 0.0217 means logistic regression probability ≥ the threshold is positive class (user will buy on return visit) else, negative class.
Results on Test Dataset for Propensity Model
On testing experiment model 2 with features Bounce, OS, TimeOnSite, Pageviews, and Country. We got a Recall of 91.7% and a Precision of 3.9%. A high recall relates to low False Negative cases and low precision relates to high False Positive cases.
Confusion Matrix on the Test dataset
NOTE: To build this model our objective was to maximize the conversion rate. We gave more importance to recall i.e. Cost(False Negative) > Cost(False Positive)
If marketing communication cost is high and business demands (equal Precision and Recall) then we would need to change the positive class threshold and metric such that Recall = Precision (take F1 Score as metric).
Scope of Improving the Model
- If the user is signed in on the website or not.
- Add Traffic Source; organic or ads brought them to the website
- Use time features: some users love shopping at night (case study link)
- Use Technographic Features: Device Brand and Model to target users better. There have been multiple case studies that back that users having costly/high-end devices convert more. (case study link)
- Consider experimenting with the model by adding more geographics of the user i.e. City/State. Users visiting the website from Tier 1 cities or state having higher per capita have a higher propensity to convert.
Conclusion
Now, using this propensity model marketing and audience targeting can be done more intelligently where chances of a user conversion (purchase) from the platform are higher. Also, it helps the marketing team in terms of cost as they no longer have to run campaigns/notifications/emailers on all visitors but rather focus only on a subset of users whose propensity score is high.
I hope you learned something new from this blog. If you liked it, hit 👏 and share this article. Stay tuned for the next one!
AUTHOR: https://www.linkedin.com/in/shaurya-uppal/
Newsletter: https://www.linkedin.com/newsletters/problem-solving-data-science-6874965456701198336/
Also published here.