A Guide To Google Search Ads Incrementality Testing including 4 ways to test lift on Google Search.
--- Each year in digital marketing has certain popular or trendy questions. But one question seems to be more than just the next “trend” in digital marketing. The question “How do I measure incrementality?” or put another way “How I do measure incremental lift on X channel?” has been one that is gaining prominence with digital markets and marketing leadership alike. But what is incrementality testing? Boiled down, incrementality is asking, “how much of this X (clicks, purchases, customers, etc) would I have gotten without running Y advertising.” This shift from more basic attribution models like the last click has been coming for a while. Both market trends and certain companies have helped to accelerate this. Amazon, for example, has always tried to use incrementality in their paid ad measurement. Meanwhile, Facebook has been telling advertisers to move their digital measurement to this sort of a model. Why? Well, big blue will say since Facebook creates intent, it’s a more nuanced and accurate way of capturing the value they drive. Internally, Facebook also knows it has a better chance to beat out Google for a share of digital advertising wallets if they move advertisers away from measurement that’s been fine-tuned to work with Google for the past 10 or so years. Whether incrementality testing is a totally altruistic trend is up for debate. What’s not up for debate is digital ad incrementality testing is important. It is also not always easy to do. Google search incrementality testing can be difficult. Below, I’ll highlight the most common ways companies generally test for incremental lift, specific to aid Google Search incrementality testing.
Datlab has tested incrementality a variety of ways. We've done our best to provide an overview of how to test Google Ad incrementality (while ranking the options from easiest to hardest).
--------------------
[Easiest Option To Test Incrementality] Paid & Organic Reporting - The easiest solution to understand incremental clicks is Google’s free Paid & Organic Report. After linking Google Ads and the Search Console, users can see how often pages from your website are showing in Google’s free organic search results, in addition to any Google ads and ultimately, how the two perform when they appear at the same time (vs. individually).
Pro - The major pro for a Paid & Organic reporting test is it’s simple to set up quickly get data
Con - The major con is that this only provides click based reporting (not conversions or any other metrics).
Pre/Post Test -- One of the most common methods for testing incremental lift on Google Ads has traditionally be pre/post tests. To run this sort of ad incrementality test, you first try to establish a baseline of performance. Once the pre-baseline is established, you either add or remove media exposure and measure the impact. The difference between the pre and the post-exposure is then the incremental lift.
Pro - the pro for this sort of a test is like Paid & Organic reporting, it’s fairly simple to run Con - the major con is the inability to control for confounding variables. Things like competition, seasonality, weather, promotions can all impact the test during either the pre or post stage creating an inaccurate picture of what the media’s effectiveness was
Match Market or Geo Testing- A Match Market test is one of the most ideal ways to test incrementality. In a Match Market, you look at different geos (countries, cities, states, DMAs, etc) to identify a cohort that tends to perform the same (i.e. the CPA in United States, The UK Australia tend to always be around $10). Once clear matches have been identified, one set of markets is then exposed to the media. The difference in performance between the exposed media market vs. the non-exposed is than the incremental lift. Ideally, several pairs of markets should be used during this testing to reduce any variability in 1 market due to other factors.
Pro - Unlike Pre/Post, Match Market tests are better at reducing the impact of confounding variables - such as seasonality, weather, external market factors - making this test one of the "cleanest" ways to test incrementality
Con - Match Market requires an extensive data set and it’s success relies heavily on the correct selection of market pairs. Sound experiment design is key.
-Match markets can fail if you identify two markets as analogous when in reality, they aren’t, they just appeared to be for that brief moment in time you analyzed.
-Match market also requires that the ONLY thing that changes between markets is the ad exposure you are testing for (for example, don’t also run a sale at the same time, etc). [Most Difficult Option To Test Incrementality]
User Level A/B Split - A/B Split Tests are the crown jewel of incrementality testing (and Facebook’s preferred method). Like Match Market tests, the goal of A/B user testing is to identify two groups of users who generally behave the same and then expose one of the groups to the media (while exposing the other to nothing or to Public Service ads like Smokey the Bear).
The reason Facebook prefers these types of tests is due to their large amount of signed in users and the ability to target users based on logins. This is also a competitive advantage as major networks like Google, due to privacy concerns, can only use user-id or email on certain networks (YouTube and Google Search). To run a test like this on Google Search, a company must first create at least 2 unique but analogous user segments (A/B groups) and upload them into Google Ads via Customer Match. You then target those specific user segments and either expose them to ads or ensure they see no media. Then, you measure downstream impact (revenue, purchases, etc) through your backend CRM
Source: Medium
Pro - Along with Match Market, A/B or User Level testing, if done correctly, will provide the most accurate data and impact results. Con - This is the most difficult and time-consuming way to test incrementality. It requires you to look at all your backend data, effectively split customer groups and re-upload those customers into Google Ads (Facebook can help do this automatically). Ideally, your match rate will be very high (meaning the users you uploaded are able to be found by Google). For example, a list of 100 users, when uploaded, still has 100 users. The other potential pitfall here is that you need to ensure the exposed group of users actually see an ad. For example, what if those 100 users who were meant to see ads don’t log on to Google and search for the period of the test? You wouldn’t have the opportunity to expose them to media. There’s really no perfect solution to studying incrementality. All of them require some time investment and the risk of a contaminated test from confounding variables remains high for almost all the different types of tests. The world we live in isn’t static and given this constant change, it can be hard to tell whether outside forces or the media was what caused a different response. It this reason that it’s important to run any of these tests for a long time and to ensure any test run hit’s statistical significance.
Interested in learning more about how Datlab can help with incrementality testing? Reach out today to learn more.