Taking into account the measurement approaches explored in this guide, this chapter sets out how companies can select the right approach for the business, and when it might make sense to adopt a hybrid approach.

  • Choosing an approach
  • Aligning the measurement approach with the business and objectives
  • Adopting a hybrid and blended approach
  • Evaluating the success of a measurement approach
  • Using AI and machine learning to support measurement approaches

Choosing an approach

There are many approaches available to marketers to consider when choosing a methodology for marketing measurement, from those that rely on econometrics modelling to brand studies, controlled experiments, and partnerships where brands work with other parties to understand the performance of campaigns. The impact of the decline of third-party cookies and device IDs has had a direct impact on many of the measurement approaches covered in this guide. Table 1 provides an overview of these.

Table 1: Overview of how the decline of the third-party cookie and device ID impacts digital marketing measurement approaches

Measurement approach Impact of the decline of third-party cookies and device IDs How challenges can be countered
First-party and zero-party data Not impacted.
Attribution modelling Multi-touch attribution models which rely on third-party cookies/device IDs to bring in data about the user’s customer journey will become less reliable.
. 		Last-touch attribution is still possible but this will not
take account of other channels a user has engaged in. Google offers data-driven attribution modelling and its Privacy Sandbox will offer an Attribution Reporting API. Custom attribution can be developed based on machine learning predictive attribution modelling.
ID-based solutions IDs need to be created either with the consent of the user or at an anonymous group level based on interests. A key challenge is scale with multiple parties required to share user data. Only use IDs that are based on first-party or second-party data sources. The IAB UK has details on different types of addressable solutions which include linked audiences, browser or operating system linked audiences and unlinked first-party audiences.
Data clean rooms Clean rooms become more useful as they offer a way to match and anonymise first-party data with second-party data and to measure cross-channel.
Data partnerships Not impacted since it uses first-party data given with consent.
Brand studies There will be challenges with creating exposed/control groups that rely on third-party cookies/device IDs to define who has or has not been delivered the ad. Create control/exposed groups out of tests, or by using zero-, first- or second-party data.
Panel-based methodologies Not impacted.
Controlled experiments and incrementality tests There will be challenges with creating exposed/control groups that rely on third-party cookies/device IDs to define who has or has not been delivered the ad Create control/exposed groups out of other data sources e.g. geographic tests or by using zero-, first- or second-party data.
Attention-based measurement  Not impacted.
Econometrics/marketing mix modelling (MMM) Mostly unimpacted unless the model depends on appended digital data from third-party cookies/device IDs (usually via a multi-touch attribution model).
. 		Check with the econometrics provider that they do not append third-party-dependent multi-touch attribution models into their product, and where possible incorporate zero- or first-party data for granular insights.

Aligning the measurement approach with the business and objectives

There is not one approach that will suit everyone. Companies should look to determine what approach is most suitable for their business and objectives. There are a number of questions companies can ask to help identify which might be the most appropriate measurement approach to suit their needs and whether they should consider a blend of approaches. Consideration will also need to be given to the budget required, particularly if a number of different techniques are used.

  • What are the campaign objectives and KPIs?
  • How much money is being invested in marketing activities, and how does this compare to how much money is available for measurement and testing? Does this reflect sufficient budget in terms of the proportion of investment in marketing activities?
  • What data sources are required for the approach and how can the data be further enhanced from other sources e.g. data partnerships?
  • Does the approach require historical data for analysis and how much data does the company have? (MMM requires a certain amount of data to work with, typically a minimum of two years.)
  • How many media channels is a company using? (If only using one channel, then MMM might not be suitable, but if considering using TV, that can be a key driver towards using MMM.)
  • How many campaigns are being run a year? (If only one campaign, MMM might not be suitable.)
  • Can the chosen measurement method scale in proportion to spend?
  • Can the model be validated through robust testing?

Adopting a hybrid and blended approach

It is clear from talking to brands and agencies that many companies are adopting a blended approach to get a better understanding of what is working.

“We use MMM to provide us with good media planning input, but we then use brand lift studies for our more tactical digital campaigns to make sure that we’re effective with our claims and creative. If we’ve got a hypothesis which is fairly small, we will usually do some kind of control experiment and shut off a geography, for example, so we can test different claims.”

James Sharman, Northern Europe Digital Acceleration Lead, Haleon

The challenges around attribution have led companies to adopt what Google refers to as the ‘trifecta’ approach, with this being hailed the ‘golden trinity’ framework. As described in an article by Marketing Week,[1] the ‘trifecta’ is where organisations combine experimentation, econometrics and short-term digital attribution to assess marketing effectiveness.

This blended approach is backed by Les Binet, Group Head of Effectiveness at adam&eveDDB,[2] among others, who describes it as the optimal blend of marketing effectiveness evaluation approaches. Google, in its 2019 report on measuring effectiveness,[3] highlighted that one of the ‘Three Grand Challenges’ was to successfully combine these complementary methods.

“As companies move away from third-party cookies in general, looking at marketing attribution on its own is taking a back seat,” observes Marina Peluffo, Head of Business Intelligence at Prima. “Instead, companies are shifting to a more holistic view that MMM can bring, and then they are validating what they see from the MMM studies through A/B testing and adopting a triad of techniques around measuring marketing effectiveness.” An illustration of this is shown in Figure 1.

Figure 1: Integrated triad approach to measuring effectiveness

Source: Econsultancy

Adopting the concept of an integrated and blended approach to marketing measurement enables companies to work towards gathering a complete understanding of the landscape and how they are performing.

“The first pillar of the trifecta is the marketing mix econometrics model. At Sky, we use the econometric models as our baseline view for all our media investments across all products and marketing channels. These studies are look back impact analysis, and it takes at least a quarter to present the recommendations as it needs input from offline media and other external sources which is not available in near real time.

“Digital marketing performance reporting is near real time as opposed to other media, which means measuring the effectiveness of investment in digital marketing requires investment curves which are near real time. So, we build additional digital investment curves in parallel with the econometric investment curves to allow for in-quarter optimisations and scenario planning.

“Econometrics can’t deal with the incrementality of granular activity like individual partners within a plan, or campaigns within a media channel. To solve such challenges, we employ the second pillar of measurement – media testing. The golden standard in media experimentation is geo-lift studies which remain our primary tool for all strategic questions, but we also use other forms of media testing, like at customer level, cookie level, or session level, depending on the hypothesis we are trying to prove. Typical questions include whether a technology, partner, activity or tactical change has been incremental in opening new headroom, driving media effectiveness, or delivering incremental sales.

“Finally, the third and last pillar is platform attribution. Unlike the first two pillars it can’t measure incrementality of media but analysed over time, this is a good measure of shift in effectiveness of digital media, though the loss of cookies poses a challenge to its accuracy, and newer techniques/technology will soon replace the traditional attribution measurement.

“Measuring incrementality is the only way one would know what the return on media investment is; the first two pillars of the trifecta are great at it, but they take time while platform attribution is near real time but is only directionally correct. This challenge is overcome by using the relationship between incrementality measurement and platform attribution thus helping us measure the incrementality of digital media in near real time.

Previously, we used to say that incrementality testing feeds into econometrics to improve their accuracy. However, in the current day and age when everyone in the industry is experimenting, it is a two-way dialogue between media testing and econometrics. Econometrics learns from media testing results, and media testing needs to apply econometric principles when isolating impacts. The trifecta of measurement has helped Sky ‘right size’ its digital media investment and isolate what is driving effectiveness.”

Kumar Amrendra, Head of Digital Marketing, Sky UK Ltd

The benefits of adopting a blend of techniques are being seen by a number of companies. This was a point made by Sage in Marketing Week and Kantar’s Language of Effectiveness report in terms of gaining insights into short- and long-term effectiveness. “It’s pretty complex – you’ve got a lot of data points and it takes time to build them. But if you can get that right, blending those three approaches allows us to improve what we do over the long- term and allows us to yield the most insight to inform our marketing decisions going forward.”

Mauricio Ferreira, Marketing Effectiveness Lead, at Confused.com, further supports using a combination of techniques to get a better picture of effectiveness. “We use three main sources of truth for marketing measurement and triangulate as many sources as available for each channel, from econometrics/MMM (via agencies); geo-tests (both agencies and in-house) and digital attribution/MTA”. This approach provides clearer results for Confused.com.

“We have explored multiple approaches, such as launching new channels using geo-tests and combining the results with econometrics. With clear results from two different methodologies, we managed to get buy-in from the senior leadership and invest further in those new channels. Also by re-evaluating current channels using a similar combined approach (econometrics plus geo-test), we were able to demonstrate to senior stakeholders that we are also assessing the effectiveness of the current channels on an ongoing basis.”

Mauricio Ferreira, Marketing Effectiveness Lead, Confused.com

Research carried out by Google and Kantar looked at developing a marketing effectiveness framework and reinforced the view that there is no single silver-bullet solution for measuring marketing effectiveness.[4] The research identified three stages of marketing effectiveness maturity: ‘establishing’, ‘developing’ and ‘advanced’.

Advanced organisations were found to embrace a blended approach in order to achieve a holistic view, and on average they used twice as many effectiveness evaluation methods as establishing organisations. The research also confirmed that marketing effectiveness maturity is driven by the golden trinity, with controlled experiments being the last method typically to be adopted by advanced organisations, alongside digital attribution and econometrics.

Advanced organisations are also integrating brand measurement to evaluate lasting effectiveness in order to plug a perceived gap in the trinity. This is a reflection of the importance of long-term marketing impact which advanced organisations are five times more likely to believe in, when compared with establishing organisations, and they are two times more likely to integrate brand measurement into their effectiveness ecosystem.

The research demonstrates that ROI has a constructive role to play, but this relies on being used in combination with other meaningful metrics. For establishing marketers ROI is the primary metric used, but as organisations mature, they become less reliant on ROI, instead using it alongside KPIs such as media-driven commercial metrics (revenue, profit), delivery metrics (reach, frequency), performance metrics (CPA, CTR) and brand metrics (awareness, consideration).

Marketing Week | Three Steps to a Marketing Effectiveness Framework[5]  

Gabriel Hughes, CEO and Founder of Metageni, describes how the agency’s clients are increasingly having to rely on a fragmented picture supplied by ad platforms and are struggling to reconcile numbers. This has led to an increased demand for custom measurement solutions and services. In response, Metageni uses “hybrid approaches combining elements of predictive multi-touch attribution and market mix modelling, based on a brand’s first-party data, in a custom approach.” This approach is described in detail in an article written by Hughes, which discusses how together these techniques are complementary, providing both a strategic and tactical view of a company’s marketing.[6]

“Our approach is to only work with a client’s first-party data for our custom attribution solutions, combining raw click-level site traffic analytics with transaction data and marketing platform data. This is used to train machine learning-based attribution models. Each one is entirely unique to each brand, and leverages customer signals, with different attribution for different customer types.

“We then bake in models of impression effects which are estimated using econometric techniques and reconciled with the attribution data on a daily basis. Again, these models are unique to each client, allowing the upper funnel impacts of social, display and video to drive low-funnel navigational sales via channels like brand search, organic brand, affiliates and of course direct. This way, the analysis is as robust as we can get, and the numbers add up.

“The model’s predictive accuracy is transparent and tracked daily, so we will retrain models when accuracy falls below an agreed threshold.

“With digital data granularity, it is possible to do all this with historical data of just a few months if the sales volume is there, making this a viable option for most major consumer brands online.

“Brands with significant marketing are well advised to run both expert-led MMM and custom attribution in combination, as well as consider an experimental framework to validate new marketing tactics for both strategic and tactical decision-making.

“An example of a customer adopting a hybrid custom approach was Toolstation, one of the UK’s fastest growing providers of tools and building materials. They identified online revenue opportunities of 7.5% of total revenue from paid digital channels, across the trade and DIY business. Analysis revealed that an additional 29% of conventional branch sales occur following online research, supporting the optimisation of the online marketing activity as being crucial for Toolstation. Deploying marketing mix modelling unveiled +18% ROI opportunities which were used to optimise their 2022 budget ahead of the ecommerce downturn that year.”

Gabriel Hughes, CEO and Founder, Metageni

Evaluating the success of a measurement approach

Having chosen a measurement approach, it is then important for a company to have confidence in their decision and identify ways in which to improve its effectiveness.

“Determining the effectiveness of new measurement approaches, especially in the context of integrated channel performance and attribution, is crucial,” says Amy Blasco, Partner of Enterprise Data, Experience and Marketing Lead at IBM. She has set out some key signals companies can look for when evaluating new approaches.

  1. Consistent and predictable ROI measurement: There should be a clear connection between marketing spend and the resulting return on investment (ROI). A successful approach will help marketers make decisions that consistently yield a positive and predictable ROI.
  2. Granular channel insights: A successful methodology will provide granular insights into how each channel is performing.
  3. Decreased cost per acquisition (CPA): Over time, as the accuracy of the measurement method improves, marketers should observe a decrease in CPA, as they can now allocate budget more efficiently based on precise channel performance.
  4. Improved customer lifetime value (CLTV): An effective approach not only focuses on immediate conversions but also on long-term engagement. A rise in CLTV would indicate that the approach helps in retaining and nurturing customers, thanks to a better understanding of the integrated channels.
  5. Feedback loop and continuous learning: As campaigns are adjusted based on insights, the subsequent performance should inform the next set of decisions. Continuous learning and iterative improvement are signs of a successful approach.
  6. Integrated view across channels: A successful approach will reveal how channels influence one another. For instance, did a user see a YouTube ad before clicking on a Google Search ad? Understanding this cross-channel interplay is crucial for integrated marketing.
  7. User engagement and retention metrics: Beyond just acquisition, understanding how engaged and loyal customers are is crucial. An effective measurement tool should provide insights into metrics like churn rate, session length, and repeat interactions.
  8. Positive market comparisons and benchmarks: Compare the performance metrics with industry standards or competitors using similar approaches. If your metrics are favourable in comparison, it is an indicator of an effective measurement and strategy.
  9. Positive stakeholder feedback: While data is crucial, feedback from stakeholders, like sales teams or customer service, can provide qualitative insights. If they report better lead quality or improved customer interactions, it corroborates the data-driven insights.

While adopting newer methodologies for measuring digital marketing effectiveness, it is crucial to maintain a balance between quantitative metrics and qualitative insights. Over time, the consistent achievement of key performance indicators, combined with positive stakeholder feedback, will validate the success of these.

Amy Blasco, Partner, Enterprise Data, Experience and Marketing Lead, IBM

Using AI and machine learning to support measurement approaches

As technology continues to advance and data capabilities develop at pace, there are even more opportunities for how artificial intelligence (AI), machine learning (ML) and predictive analytics can support measurement analysis and empower data-driven decision-making and marketing activity.

“We’ve recently witnessed the ability of machine learning to analyse online consumption patterns across billions of data touchpoints, providing a multidimensional understanding of audiences. This allows marketers to anticipate shifts in behaviour, interest and intent, rather than reacting to stale data.

“Attention is a great example of a metric that is gaining momentum thanks to advances in ML, combining real human data points with modelled data to provide us with a better understanding of user engagement.

“Moreover, AI and ML can be used to uncover which elements of a campaign are driving success or holding it back through advanced measurement capabilities. This provides marketers with a deeper understanding of their audiences and campaigns, enabling them to optimise their decision-making and implement strategies that deliver against key metrics.”

Chloe Nicholls, Head of Ad Tech, IAB UK

One of the biggest pain points with marketing mix modelling historically has been around data collection, but this has been improved through big data capabilities, better tracking and the ability of AI and ML to support the analysis of big datasets. “I see this as creating an environment that is more conducive to performing marketing mix modelling and companies are also developing their own teams and talent in this area,” says Marina Peluffo, Head of Business Intelligence at Prima.

There is also a need to ensure that calculations are not too simplistic and that they take account of the business and the context in which it operates, along with any factors influencing this. This was a point made by Gabriel Hughes, CEO and Founder of Metageni, who highlighted it was key to ensure models using AI and machine learning be tested for their accuracy.

“The ability to quickly deploy these models, and get high accuracy, has increased massively in recent years. We employ our own optimisation using a genetic (evolutionary) algorithm to help discover the best model. We also test new models of ‘holdout’ data, which is data the model is not exposed to in advance, to ensure high standards of data science best practice. Model accuracy is tracked for each client so we can retrain models as and when model accuracy changes.

“Recently we have also been exploring the use of the new generative AI capabilities to support our live Report and Insight platform, which we call ‘Creating Data Stories’.  The main blocker is that brands do not want to share their first-party data with the big AI platforms, so any generative language models need to be hosted and sandboxed independently, for each client. But this work is progressing well and the potential is huge.

“Finally, we see huge potential in predicting how customers will behave to target potential buyers. We ran a successful project with AO doing this in 2020, which won us the ‘Best Use of AI’ award in the Ecommerce Awards that year. We are taking similar approaches with our other clients as the predictive capability continues to improve.”

Gabriel Hughes, CEO and Founder, Metageni

  • There is no one-size-fits-all approach to measurement, so marketers will need to consider the impact of the decline of third-party cookies and device IDs on each of the digital marketing measurement approaches.
  • Decide if it makes sense to adopt a blend of techniques, and use a ‘trifecta’ of experimentation, econometrics and short-term digital attribution to assess marketing effectiveness.
  • Establish measures to determine the effectiveness of measurement approaches, considering key signals such as ROI, customer lifetime value and positive stakeholder feedback.
  • Look for opportunities to use increasingly sophisticated AI, machine learning and predictive analytics to make measurement more efficient and support decision-making.
  • When using algorithms and predictive tools, make sure calculations take the whole business and business environment into account and that they are rigorously tested for accuracy.

This guide is based on primary research which involved exploring findings from two reports:

  • Econsultancy’s 2023 Future of Marketing report, which was based on a survey of 835 client, vendor and agency-side marketers. The survey was fielded to Econsultancy and Marketing Week’s audiences between 9 June and 3 July 2023.
  • The Language of Effectiveness 2023 report has been produced using responses to an online survey of 1,369 qualifying marketers conducted by Econsultancy’s sister brand Marketing Week between 27 March and 28 April 2023.

In-depth interviews were carried out with industry experts. Econsultancy would like to thank the following interviewees for their invaluable contribution of time and expertise to this guide:

  • Kumar Amrendra, Head of Digital Marketing, Sky UK Ltd
  • Amy Blasco, Partner, Enterprise Data, Experience and Marketing Lead, IBM
  • Laura Chaibi, Director, International Ad Marketing and Insights, Roku Inc
  • Sebastian Cruz, Regional Digital Marketing and Media Director, Shiseido, Asia Pacific
  • Gary Danks, General Manager, AIM, Kochava
  • Mauricio Ferreira, Marketing Effectiveness Lead, Confused.com
  • James Hurman, Founding Partner, Previously Unavailable
  • Gabriel Hughes, CEO and Founder, Metageni
  • Dr Grace Kite, Economist and Founder, Magic Numbers
  • Chloe Nicholls, Head of Ad Tech, IAB UK
  • Roxane Panopoulos, Group Manager, Regional Measurement & Insights – Netherlands and Nordics, Snap Inc
  • Marina Peluffo, Head of Business Intelligence, Prima (speaking as industry expert)
  • James Sharman, Northern Europe Digital Acceleration Lead, Haleon
  • Steven Silvers, EVP, Global Creative and Media Solutions, Kantar

Lynette Saunders is a Senior Analyst at Econsultancy, where
she works on delivering industry-leading research, briefings and
reports for the digital marketing industry and speaks at a number
of external conferences.

Lynette’s previous experience includes delivering web analytics, measurements and insights, as well as leading usability and
customer experience programmes focusing on improving the
overall online customer experience for Cancer Research UK
and the Royal Mail Group.