Blogs

The 2024 Guide to Incrementality

TL;DR

Have you or a loved one recently been inundated with messages about incrementality? Can’t figure out if this is the latest fad or a legitimate evolution in analytics? In today’s fast-paced and data-driven marketing landscape, staying ahead of the curve is essential for success. In a sea of buzzwords and trends, incrementality has emerged as a contender with significant implications for marketing strategy and performance measurement. In this article, we’ll cover what exactly this purported “silver bullet” is, and what marketers need to know. 

Section 1: What is incrementality? 

Incrementality refers to the measure of the additional (or incremental) lift that a marketing initiative provides compared to baseline expectations. In other words, incrementality seeks to answer the question of “what additional value did my marketing activities contribute beyond what would have occurred naturally?”

Incrementality helps answer this question by comparing the performance of those who were exposed to marketing campaigns (treatment group) with a similar cohort that did not see those same campaigns (control group). By measuring the difference in outcomes between the two, we can determine the additional value generated by the campaign. 

You’ll notice this is essentially describing the scientific method; like scientific experimentation, incrementality analysis begins with hypothesis testing, where marketers conduct randomized experiments to isolate the incremental impact of campaigns. This emphasis on empirical evidence is why incrementality is so revered: of all the popular analytical methodologies employed by marketers, incrementality is the only one that establishes a causal relationship as opposed to simply explaining the correlation between the independent and dependent variables. 

These experiments can take many forms, but they all generally involve some way of randomly selecting users into a treatment or control group. A geographic experiment, for example, would segment the country into similar pockets and then randomly assign them to a treatment and control group. This is also called a holdout test since we’re withholding marketing exposure from a portion of the target audience. 

Section 2: How is incrementality used?

Since we can rely on incrementality to establish a causal relationship, it’s often a great tool to validate results or justify marketing spend. Unlike other common marketing analytics tools that mostly conduct correlational analysis, this gives our data more credibility. The next time your finance department asks why they should keep spending dollars on a marketing initiative, you can simply refer to the results of your latest incrementality experiments with confidence. Without the marketing spend that you’re counting on, the company would be leaving money on the table. This really helps change the message from looking at marketing as an expense, and more as an investment. 

Incrementality is commonly used for budget allocation purposes because we can run various types of randomized experiments on our top channels and campaigns to test if those marketing dollars are truly driving new revenue instead of cannibalizing existing sales. Not sure if Facebook Display is actually driving positive results for your business? Design an incrementality test and find out. Repeat the process for your top channels and let those results help you decide how to divvy up your marketing spend. Tactically, incrementality is also well-suited for channels that your team doesn’t have much experience with. Let’s say your team wants to spend more on a new channel, but you’re not sure if additional spend will generate the necessary ROI. If we run a pilot and conduct incrementality analysis on the results, we can extrapolate and forecast what a scale up would look like.

Section 3: What are some challenges of incrementality? 

While incrementality testing is a powerful tool for measuring the true impact of marketing efforts, there are some challenges and limitations you should be aware of. 

  • Since incrementality involves experiments, a big challenge is ensuring the sample size is sufficiently large enough so that our results are statistically significant. Small sample sizes can lead to low statistical power, making it difficult to draw meaningful conclusions from the results. 
  • It’s impossible to remove all of the “noise” surrounding a marketing campaign so an omitted or hidden variable could dramatically skew your results. Did you hire more sales reps while the campaign was running? Did you increase more communication with the system so more sales occurred? When you randomly assigned treatment and control groups, were there hidden mechanisms that prevented true randomization such as inherent biases in the selection process? Remember that incrementality attempts to show causation, which means your chances of getting a bad read increase if there are important omitted variables. Incrementality also struggles to account for spillover effects (marketing efforts often impact individuals who were not directly exposed to the campaign, through word of mouth and other means) which are particularly important in digital marketing where these cross-channel interactions are more common. 
  • Incrementality is not a low maintenance, set it and forget it, type of tool. To get the most out of this technique requires a skilled marketing team that has a larger strategy and framework (in the form of a learning agenda or testing plan) that will guide what to test, when to test it, and most importantly, how to interpret those results and make actionable decisions off those insights. 
  • Note that incrementality experiments generally run for weeks not days, during which we cannot make changes to our campaigns because it will impact results. That means no swapping out of creative, A/B testing, or introducing any other variables while experiments are running. This is not ideal for larger channels because it requires us to disrupt our business as usual operations to accommodate a statistical technique. 

Section 4: Incrementality vs. other popular analytics methods 

Incrementality is often grouped together with other popular measurement methodologies including multi-touch attribution (MTA) and marketing/ media mix modeling (MMM). While all 3 can help marketers understand the effectiveness and impact of their efforts, they evolved for different reasons and thus are best suited for different things. 

Before the digital advertising schism moved us away from traditional advertising channels like linear TV, MMM was the primary form of media measurement because it doesn’t rely on granular data. MMM excels at capturing older channels and the longer-term impact on revenue by leveraging historical data (both media and macroeconomic data) to predict future sales based on past behavior. However, MMM is very expensive to spin up and often takes months to deploy with a reporting lag after each quarter to update the model, so other methodologies sprang up to make up for these deficiencies. 

MTA came next with the rise of digital media, along with detailed user tracking which gave marketers more granularity in terms of insights. By leveraging this richer user data to track customer journeys, MTA was far better adapted for the complex and cross-channel landscape and is the most bottoms-up approach as it collects data at the individual level and then aggregates this to create a model. However, with the rise of consumer privacy regulations, MTA may become less effective in the future if user-level tracking goes completely away (still looks to be sometime in the future given Google recently delayed cookie deprecation again) so marketers began to turn to incrementality experimentation. 

You’ll notice that none of these methodologies are silver bullets; each have their own distinct advantages and disadvantages. The truth is that the best marketing teams leverage some combination (or all) of these 3 to guide their decision making because this can make up for individual weaknesses. Incrementality is simply the latest in a long line of measurement methods that marketers are using to try and gain an edge. Although the promise of finding causal relationships is tantalizing, the challenge of setting up experiments correctly keeps many companies from effectively using incrementality. If your audiences somehow overlap, your segment sizes are off, or a number of other possibilities somehow skew your results, you’re using incorrect data to make very impactful decisions (potentially for the worse). 

If you want to learn more about the MTA, MMM and how they compare to incrementality, we wrote this awesome deep dive that goes into much more depth on what MTA and MMM are, and also provides a detailed comparison of the three. Click here to check that out.

Continue Reading

Illustration of pixel tracking concept with magnifying glass over mechanical gears, symbolizing data analysis, and Northbeam logo in the top corner. Text reads: 'What is Pixel Tracking?
Blogs

What is Pixel Tracking?

A woman, symbolizing digital marketers, manages glowing data cables, representing ad tracking parameters like gclid and fbclid, and their role in traffic attribution.
Blogs

Crack the Code on UTM Parameters for Ad Tracking

Blogs

Data-Driven Marketing Is the Future — Here’s Why It Matters

Blogs

Accounting Modes 101: What Marketers Need to Know

Blogs

Google Chrome Won’t be Deprecating Cookies — Yet

Blogs

2025 Marketing Trends to Watch Out For: Part 2

Blogs

2025 Marketing Trends to Watch Out For: Part 1

Moving Beyond Walled Gardens: The Shift in Digital Advertising

What is Marketing Intelligence Software?

Blogs

What is Marketing ROI and How Can You Improve It?

How The Oodie Tracks the Metrics That Matter with Northbeam

Unlocking the Truth Behind Incremental Performance: Why & How Attribution Models Can Make Or Break Your Business

Media Mix Modeling (MMM): Build or Buy?

The Marketer’s Guide to AI

Case Studies

The Best of Art and Science: Why Omnilux Uses Northbeam to Make Crucial Marketing Decisions

What is Media Mix Modeling (MMM)?

Blogs

Media Efficiency Ratio (MER) & Return on Ad Spend (ROAS): What are They & What’s the Difference?

Case Studies

How Northbeam Gives The Kizik Team Answers to Questions They Didn’t Know They Had

Blogs

The Future with YouTube Advertising

Blogs

Serious Data For Serious Growth: How Northbeam Enables HexClad To Stay Efficient with Scale

Why You Should Care About MTA, MMM, and Incrementality

What to Expect When Switching to Northbeam

Dr. Squatch X Northbeam
Case Studies

Profitable CACs, Lifetime Value, and "Red Zones": How Dr. Squatch uses Northbeam

What is Multi-Touch Attribution (MTA)?
Blogs

What is Multi-Touch Attribution (MTA)?

How to Not Get Fired: A 5-Step Guide for CMOs
Blogs

How To Not Get Fired: A 5-Step Guide for CMOs

What makes Northbeam data different
Blogs

What Makes Northbeam Data Different?

Northbeam is powered by AI, and so will you.
How to use Northbeam

How does Northbeam use AI?

Northbeam is the perfect tool for performance marketing agencies.
Media Strategy

5 Reasons Why Northbeam is the Perfect Tool for Performance Marketing Agencies

Northbeam's newest metrics include shipping, discounts, refunds and taxes.
How to use Northbeam

How to use Northbeam's Newest Metrics: Shipping, Discounts, Refunds and Taxes

Blogs

How Will Cookie Deprecation Affect Northbeam? An Explainer

How to use Northbeam

How To Use The Northbeam Sales Page

Blogs

Northbeam's Definitive Guide to Q5 2023

Cost per clicks
Blogs

Do Cost Per Clicks Matter?

Data Reports

Performance Benchmarks: Black Friday Cyber Monday 2023

Darlene Ghorbanian
The Media Buyer Summit

Incorporating Creative Strategy into Media Buying with Darlene Ghorbanian

Nehal Kazim
The Media Buyer Webinar

​​Scaling Ads with Benchmarks with Nehal Kazim

Barry Hott
The Media Buyer Webinar

Unleashing the Power of Ugly Ads with Barry Hott

Case Studies

Creative Iteration and Optimization at Vessi Through Trustworth First-Party Metrics

Case Studies

How Jones Road Beauty Unlocked Viral Growth by Choosing the Right Attribution Solution

The Media Buyer Newsletter

The Media Buyer Index

KITSCH and Northbeam
Case Studies

Unlocking Growth With Dependable Data & Clear Customer Journeys: KITSCH

via CNET - HexClad
Case Studies

How HexClad uses Northbeam's Indefinite Lookback Windows for Global Scale

MYHD DJ Store Improves Customer Acquisition Strategy Exclusively with Northbeam Data
Case Studies

Improving ROAS +84% With Data Driven Decisions at MyHD

PetMeds
Case Studies

Targeting High-Value Customers and Subscriptions for Sustainable Growth at PetMeds

The Ridge relies on Northbeam for intelligent ad buying
Case Studies

How The Ridge Used Tactical Media Buying to hit $300 Million in Wallet Sales

It's time to take your marketing

to the next level.

Chat with our team and discover how Northbeam can transform your business.
book a demo