Marketing measurement methods: Combining approaches for optimal results

The Institute of Practitioners in Advertising (IPA) released a comprehensive report on marketing effectiveness measurement last year, emphasizing that combining multiple approaches delivers the most accurate picture of advertising performance. The report outlines a structured framework for marketers seeking to navigate increasingly complex measurement challenges.

In an era where attribution has become increasingly difficult due to privacy changes and fragmented consumer journeys, marketers are advised to implement a systematic approach that combines marketing mix modeling (MMM), experimentation, and attribution methodologies. Each approach offers distinct advantages in different contexts, according to the whitepaper authored by Simeon Duckworth, Neil Charles, and Duncan Stoddard of the Melt Collective.

According to the IPA report titled "Making effectiveness work," launched on October 9, 2024, marketing effectiveness measurement requires a multi-faceted approach. "As data and advertising become ever more fragmented, a learning culture is essential to measuring marketing effectiveness. A learning culture reduces silos, empowers marketers and ultimately allows them to ask better questions," states the report.

The IPA researchers discovered that while marketers have powerful tools at their disposal, no single approach can address all measurement challenges. MMM excels at understanding long-term effects but lacks granularity. Attribution delivers detailed performance metrics but struggles with privacy constraints. Experimentation provides causal clarity but can be expensive to implement at scale.

Dharmesh Damani, UK Data & Measurement Lead at Google, highlighted this relationship in a recent professional post, stating, "While there are strengths and weaknesses to each solution it is important to combine all 3 to evaluate the true performance of your advertising."

Detailed analysis of measurement approaches

The comprehensive analysis published by the IPA reveals distinct characteristics of each measurement approach:

Marketing Mix Modeling (MMM)

• Strong in measuring long-term and multichannel effects
• Provides holistic view that disentangles media interactions
• Privacy-robust as it works with aggregate data
• More cost-effective to scale than experiments
• Less granular than attribution methods

Experimentation

• Delivers causal insights with limited bias
• Moderate ability to detect small effects
• Reasonable privacy compliance
• Mid-range cost structure
• Can measure short-term effects but less effective for long-term analysis

Attribution

• Highly granular, detecting small effects
• Less effective at measuring long-term impacts
• Faces increasing privacy challenges
• Lower implementation costs
• Struggles with multichannel measurement

The visual presentation of these characteristics highlights how the three approaches complement each other, with MMM strongest in holistic measurement and long-term effects, experimentation excelling in causal accuracy, and attribution providing superior granularity.

The MESI framework: A systematic approach

The IPA report introduces a structured methodology dubbed MESI (Model-Experiment-Simulate-Implement) to create a cohesive measurement strategy. This approach starts with modeling to establish baseline understanding, followed by targeted experimentation, simulation of potential outcomes, and implementation of findings.

"Start with a model, this could be MMM, data-driven attribution or consumer modelling, to map marketing effectiveness. Use this model to highlight where there is evidence to suggest changing the communications plan may be more effective," explains the IPA report.

John James, a commercial strategy specialist commenting on the findings, noted an important consideration: "Yet none of them deal with the elephant in the room. Political reality." This observation underscores that even the most sophisticated measurement approaches must navigate organizational dynamics and established practices.

The critical role of learning agendas

Beyond technical methods, the IPA research emphasizes the foundational importance of establishing formal learning agendas within organizations. These structured approaches to knowledge acquisition differ from typical research programs in several critical ways:

1. They focus specifically on information that changes decisions
2. They establish clear learning goals that transcend organizational silos
3. They identify knowledge gaps that require multi-source solutions
4. They commit organizations to experimentation and innovation

"It is more focused than a programme of research and analytics," states the report. "Provides the pivotal information that changes minds and decisions, creating clear long-term learning goals and actions that cut across silos."

For measurement practitioners, the report advocates progressive innovation and development of new approaches. The ultimate goal extends beyond collecting data to creating an evidence-based culture that drives better decision-making.

Implementation guidance for marketers

The IPA report offers detailed implementation advice for different stakeholders:

For marketers designing measurement strategies, the report provides pragmatic guidance on structuring programs that maximize learning while minimizing fragmentation. It emphasizes balancing established knowledge with exploration of new approaches.

Agency planners receive a detailed overview of core techniques and introduction to the MESI process framework that can structure learning across tactical, campaign, and strategic decision levels.

Measurement specialists gain stimulus for improved briefing processes and a roadmap for continuous innovation in their methodologies.

"If you are an agency planner, the report provides a detailed overview of the core techniques. It introduces a disciplined measurement process, MESI (Model, Experiment, Simulate, Implement), that can be used to drive learning for tactical, campaign and strategic decisions," explains the IPA.

Addressing the challenge of data fragmentation

A core challenge identified in the research is how different measurement approaches can lead to knowledge fragmentation and organizational silos. The report recommends a "culture-first approach to measurement strategy" to address this division.

"The challenge is that different measurement approaches can result in knowledge fragmentation and silos. The report recommends a culture-first approach to measurement strategy to heal this divide by adopting formalised Learning Agendas and an active learning approach," writes the IPA.

Kevin Meyer, a brand and marketing manager reviewing the findings, emphasized the value of this integrated approach: "I love seeing more movement towards MMM and experimentation! The more accurate our understanding of reality, the more potential we have for good decision making."

Open-source developments enhancing accessibility

An important development highlighted by industry experts is the increasing accessibility of sophisticated measurement methods through open-source solutions. "What's more is that open-source MMM solutions like Meridian have made MMM far more accessible," noted Damani in his professional network post.

Meridian, an open-source MMM built by Google, provides innovative solutions to key measurement challenges, making advanced analytics more accessible to marketers without extensive data science resources.

Expert perspectives

Industry experts have provided nuanced perspectives on the findings. Simeon Duckworth, founder of the Melt Collective and co-author of the report, stated: "There is no silver bullet to measure marketing effectiveness. Each approach has its role and its limitations. For most marketers, the solution is to focus more on the process and culture of active learning rather than on specific techniques."

Nico Neumann, Assistant Professor at Melbourne Business School, observed: "While the technical aspects of measuring marketing effectiveness often generate the biggest fears, it's usually the mindset and culture of an organisation that determine whether analytics projects succeed."

Laurence Green, Director of Effectiveness at IPA, concluded: "This report aims to help steer marketers through the effectiveness thicket, using the core techniques of marketing mix modelling, experiments and attribution. What they will find is that the solution lies more in establishing a decisive, effectiveness culture than in chasing the perfect evaluation technique."

A forward-looking strategy

The IPA report stands as a comprehensive guidance document for marketers navigating the complex landscape of effectiveness measurement. Released in conjunction with the IPA Effectiveness Conference in October 2024, it emphasizes that success depends not simply on technical solutions but on cultivating an organization-wide approach to measurement.

"Making Effectiveness Work is a standalone publication, but it sits alongside the IPA's broader initiative to foster an Effectiveness Culture with agencies," notes the IPA on its website.

For marketers facing increasing pressure to demonstrate return on investment amid economic challenges, the research provides timely guidance on structuring measurement approaches that combine technical rigor with organizational learning.