Audience Measurement: Pursuing Transparency in Metrics
The IAB Helps Improve Visibility into Online Audience Measurement
This past year has been marked by several historic milestones in the evolution of audience measurement. In April 2007, the IAB took a leadership position in calling on third-party, panel-based audience measurement companies to have their methodologies audited (read more here). And by Fall 2007, the two leading audience measurement companies, comScore and Nielsen Online (formerly known as Nielsen//NetRatings), had demonstrated their commitment to transparency and industry leadership by beginning audits with the Media Rating Council (MRC).
See accompanying interview with George Ivie,
CEO and Executive Director of the Media Rating Council.
This momentum continued on November 29, 2007, with the first annual IAB Audience Measurement Leadership Forum. Senior practitioners from worlds of audience measurement joined executives from media, agencies, marketing and research to explore the greatest advantages that interactive media offer advertisers—the opportunity to target audiences with greater degrees of refinement and to measure the results of marketing communications activity with greater accuracy than other mediums allow.
Randall Rothenberg, President and CEO of the IAB, opened the day by setting out the vision—“combining sample- and census-based research can create a richer portrait of people’s lives.” (See Randall’s remarks from the event, at the IABlog.) Steve Wadsworth, President of the Walt Disney Internet Group, called in his keynote address for a focus on “clarity, simplicity, and actionability” in metrics to spur greater industry growth, a position that was echoed by speakers throughout the day.
The heads of the leading audience measurement companies told the standing-room-only crowd about the differentiated products they offer to the market, sharing their methodologies and insights about the future. Manish Bhatia, President, Global Services and U.S. Sales, Nielsen Online, talked about the advantages of panel-based measurements combined with other data points that can help marketers get a clear understanding of their audiences when there is a “problem of plenty.” Josh Chasin, Chief Research Officer, comScore, observed that “the most measurable medium means that we have the most measures.” Konrad Feldman, Co-Founder and Ceo, Quantcast, announced his belief that “the future of advertising is addressability.” Chris Maher, President, Hitwise, explained that marketers need information that “complements, adds depth, and context” to data.
Stephen Kim, Director of Advertising Research at Microsoft Digital Advertising Solutions, described the present state of metrics as “a world of plenty,” where “multiple data sets are the key” to understanding audience. On the agenda throughout the day were panels about Turning Data into Insights—Real World Case Studies, Cross-Media Platforms, New Audience Metrics in the Digital World, and Emerging Media Platforms Measurements. For more background on the forum, go to IAB Audience Measurement Leadership Forum. For some early press coverage of the event, see Mediapost’s Mark Walsh and some key points covered by PaidContent.org.
Visit adage.com for a video clip from the event. Read coverage of the event from AP business reporter, Seth Sutel.
As we look ahead, 2008 is shaping up to be an even more important year for audience measurement, as the industry moves toward a common framework for online metrics.
Defining the Issues in Audience Measurement
The Internet, the most measurable of all media, has been marked by two major, and different, ways to measure audience size:
• Panel-based measurement, exemplified by comScore and Nielsen Online, which extrapolate site audiences based on tracking the activities of a sample of users; and
• Server-side measurement, wherein companies analyze their own server logs of site traffic to determine the size of their audience.
Discrepancies arise from these differing methods. Frequently, the audience metrics generated by the panel-based providers have differed from one another, and they are often significantly different from those calculated by server-side techniques.
While these discrepancies have existed for as long as the commercial Internet, 2007 marked a turning point in the industry’s commitment to pursue solutions.
The IAB recognizes that there is validity in both panel-based and server-side measurements, and has encouraged the industry to consider "convergent validity" – a phrase coined by Peter Daboll, Yahoo!’s chief of insights and head of global market research, and a former president of comScore’s Media Metrix division. Convergent validity holds that combining different yet reliable data sources offers a richer understanding of the total audience. By adjusting for differences in each of the data sets, a clearer picture of the composition, segmentation and size of the audience can be achieved.
Opening a New Dialogue in 2007
Following its call to comScore and Nielsen Online to have their methodologies audited, the IAB held a historic audience measurement summit on May 16. This cross-industry meeting included executives from comScore and Nielsen Online, staff and members of the IAB, and representatives from industry trade associations, including the Advertising Research Foundation (ARF), the American Association of Advertising Agencies (AAAA), the Association of National Advertisers (ANA), the Magazine Publishers of America (MPA), the Media Rating Council (MRC), and the Online Publishers Association (OPA). The dialogue at the meeting enabled the key stakeholders to better understand one another’s positions, paving the way for future conversations. The participating organizations committed themselves to becoming the foundation of a new cross-industry effort that will provide the marketing and media ecosystem with greater transparency and educate it on best practices for the use of audience data in interactive planning, buying and selling.
At the meeting, Nielsen Online announced that it was moving into a full audit. comScore announced it was in final stages of a pre-audit. Both companies had started their full audits with the MRC by Fall 2007. A lengthy and complex process, the MRC audit of their methodologies will help identify and/or eliminate some of the sources of the discrepancies between panel-based and server-based audience measurements, establishing a foundation for future work toward reconciliation of the data sets.
For its part, the IAB announced its continued commitment to furthering transparency in audience metrics by educating the marketplace about the benefits of both panel-based and served-based data and developing definitions for unique visitors, page views and time spent. The IAB also affirmed it will continue to advocate for auditing against all measurement guidelines to increase consistency and reliability in interactive metrics.
Pursuing the Solution
Barring unforeseen developments, it is likely that the audits of Nielsen Online and comScore will be completed by late 2008. The accreditation of these companies will help decision-makers better understand the data, and therefore make better buying decisions.
Also in 2008, the IAB Audience Measurement Working Group expects to release the guidelines that will include definitions of unique visitors, page views and time spent, and will review the impact of cookie deletion, international traffic, spiders and bots, and other factors that potentially affect server-based measurement. Once these guidelines are established, organizations performing server-side measurement can begin to be audited based on them, increasing the reliability and comparability of server-based measures.
As work continues across the entire ecosystem to deliver the best measurement data and increase certainty for buyers and sellers, the IAB is dedicated to creating better understanding of how to use panel- and server-based measurements in combination, benefitting all participants in the interactive advertising marketplace.