With more than $70 billion of annual television and video spend at stake, the video advertising industry is at an inflection point as market providers have faced serious questions about the accuracy of their measurement and their foundational methodological approach. Amid these ongoing concerns about TV/Video measurement reliability and accuracy, Comscore, a trusted partner for planning, transacting, and evaluating media across platforms, has published new research that reveals how traditional panel-reliant media measurement – which in many instances does not fully account for margins of error or confidence intervals – means that many decisions in media placement are essentially equivalent to tossing a coin.

In a new blog post titled “Relative Errors in Television Audience Measurement: The Future is Now”, Comscore Chief Research Officer Michael Vinson examines the limitations of panel-dependent methodologies and outlines why advertisers who utilize panel derived estimates for planning, programming, or content acquisition strategies need more precise and granular information to properly capture the ROI of their ad spend.

Marketing Technology News: ReelTime to Launch Roku Channel ReelTime TV to 51.2 Million Monthly Active Users, Expanding Content…

Research highlights: fundamental flaws of a panel-dependent approach

• Vinson’s analysis demonstrates how small samples today are extremely unstable and not able to properly support the decisioning of media spend that is in the market. An alternative is needed to ensure advertisers get the results they deserve.
• Common estimates place panel participation rates in the US around the 20 to 30% range and declining. Put differently, around 70% to 80% or more of the population contacted refuse to be a part of a panel. It becomes harder and harder to believe that the resulting panel is a probability sample of the population.
• The dynamics of panel refusal have almost certainly changed during the COVID-19 pandemic, particularly when empanelment requires in-home interviews or installation. Thus, even if there may have been some attempt to understand the effect of nonparticipation bias before 2020, in the pandemic and post-pandemic worlds those conclusions, particularly for methods that include in-home visits, are now largely inapplicable.
• Today, with hundreds of networks and stations available, not to mention time-shifting, streaming platforms and video on demand, the size of a panel required to plausibly measure most content has become impractical.

“Comscore is firing the starter pistol on a long-overdue conversation about the need to move beyond panel-based point estimates in media measurement. The relative reliability of measurements and methodologies needs to be a primary consideration when evaluating measurement,” said Michael Vinson, Chief Research Officer, Comscore.

“For a variety of social reasons, the collection of data from human populations using probability samples has become problematic,” said Dr. Charles D. Palit, professor emeritus at the University of Wisconsin and a widely recognized expert in statistical sampling. “Because of this, many statisticians now look for new and better data gathering processes. Comscore’s data system is one such example. The sheer mass of the data driving Comscore’s solutions allows detailed estimates for complex viewing behavior that would be fiscally prohibitive if attempted on a probability sample.

Marketing Technology News: ADvendio Launches a Money-back Guarantee to Accelerate Digital Transformation for Advertisers,…

Comscore’s approach to measuring television viewing behavior produces data which is more robust and valid than the data produced by current probability sampling methodology, and better suited for measuring television in the United States than a probability sample.”

“We know that media will never be the same, so neither should its currency. With billions of dollars at stake across the industry and with viewing habits changing rapidly, advertisers should no longer rely on outdated approaches to audience measurement. This is why Comscore has built a currency designed for today’s environment, using passive collection techniques and measuring 1 in every 3 homes in the U.S. to provide more granular and stable viewership for brands and agencies to make more reliable buying decisions,” said Bill Livek, CEO, Comscore.