Incrementality has been hailed as the gold standard in measurement for marketing. In the BCG-Meta “Measure to Grow,” report measuring incrementality is seen as the ideal for measuring the impact of interventions and spending, even though the ease of use is lower than click-based metrics. However, despite its significance, there are nuances to its measurement that can make it challenging for business users to extract the right insights. In this article, we will explore the intricacies of measuring incrementality and shed light on some key considerations that marketers should keep in mind.
How Incrementality Is Measured
To measure incrementality, a target group (TG) is identified to receive a specific intervention, such as an email campaign, while a control group (CG) is withheld from the intervention. By comparing the conversion rates of the TG and CG, the delta between the two can be calculated as the incremental conversion. This incremental conversion, when multiplied by the target base and revenue per conversion, provides a measure of the incremental revenue generated.
You might also like to read about The Ethics of Using Machine Learning in Marketing: What You Need to Know.
Some Challenging Nuances of Incrementality Measurement
Short-Term vs. Long-Term
It can be conducted in the short term or the long term. Short-term control groups are held out specifically for a campaign, while long-term control groups are held out for an extended period, typically ranging from six months to a year. Both approaches offer valuable insights. Short-term control groups help evaluate the impact of withholding a specific communication, while long-term control groups provide insights into the cumulative effect of not receiving a series of communications. While both approaches have their merits, long-term control groups often offer more actionable insights.
Handling “Unsuccessful” Campaigns
Occasionally, smart shopping campaigns may not yield the desired results, with the control group outperforming the target group. In such cases, convention dictates labelling the campaign as “unsuccessful” and setting the incremental revenue to zero. Reporting negative impacts at the campaign level is counterproductive since it does not provide meaningful information. Instead, considering the campaign’s impact as neutral helps maintain its integrity.
Granularity of Measurement
The level of granularity chosen for measurement can significantly impact the calculated lift. Whether measured at a campaign level per day, campaign level per month, or segment level per month, the decision affects the resulting incremental revenue. A more granular approach, such as measuring at a trigger level per day, tends to yield higher incremental revenue values. However, it is essential to strike a balance between granularity and practicality, as excessively granular measurements may become unwieldy and time-consuming.
Non-Comparative Nature
When assessing incrementality, it is crucial to recognize that trigger-level lift and segment-level lift may not align when measured separately for the same period. Different data cuts, such as triggers, segments, revenue centres, or periods, can influence the calculated incremental revenue. This can lead to complex discussions as stakeholders attempt to reconcile incrementality numbers that may not precisely match.
Reading about the Benefits Of Customer Life Cycle Management: How It Can Improve Your Business can also be quite helpful too.
Unreal Numbers
While incremental revenue can be quantified, isolating the specific transactions that contributed to it is challenging. Drill-down analyses, such as identifying incremental bills, customers, or products, are often not feasible. Incrementality focuses on the collective impact of interventions rather than pinpointing individual transactions. Therefore, it is essential to consider the overall incremental revenue rather than fixating on isolating specific components.
Impact of Intervention vs. Selection
Most incrementality metrics primarily measure the impact of interventions—targeted and relevant messaging over some time. However, it is crucial to distinguish between the impact of the intervention itself and the impact of selecting the right audience to target. If predictive modelling is used to identify the best customers for targeting, and the control group consists of similar high-potential customers, the lift observed is primarily attributed to the intervention rather than the selection process.
Conclusion
Understanding and leveraging incrementality effectively requires a thorough examination of its intricacies. Superficial analysis of reported incremental metrics may lead to confusing results upon deeper investigation or contextually inappropriate measurements that fail to provide true insights. At SOLUS, we have invested years in studying it and have incorporated practical and usable approaches into our methodology. Reach out to us to gain a comprehensive understanding of incrementality and lift measurement.