Recently in Metrics Category

Drifting Away from the Realities of the Media Business

| | Comments
metrics-blog-header.gif

iStock_000000736447XSmall.jpgThe IAB loves Doug Weaver. Our members love Doug Weaver. Doug speaks at our conferences. I personally have nothing but the highest regard for Doug and have agreed with much of what he has written and said over the years.

Last week, he wrote a surprising piece in his blog, The Drift. The piece entitled, “The Myth of Digital Measurement” is well written, but stunningly out of synch with the realities of competitive media selling - and ignorant of the work the industry is successfully prosecuting to make measurement make sense.

Doug builds his argument on four “facts.” Unfortunately, it is his facts that contain myths. Let me detail his errors- and then tell you how a united marketing and media industry, led by the IAB, the ANA, and the 4A’s, are going about addressing marketers’ need for actionable measurement standards in digital media.

In his blog posting earlier last week, Weaver argued that, “Fact One: Digital is Permanently Dynamic. Not only do digital capabilities constantly evolve, but the pace of that evolution is constantly accelerating. Which leads to……Fact Two: ‘Measurement’ Can Never Keep Up”. By Doug’s reasoning, digital media are separate from other media for the long term because their measurement is a myth. This kind of thinking keeps us forever on the experimental lines of big brand ad budgets. Finding a way for brand advertisers to evaluate digital media in terms they understand and can use to justify investments, by definition, does not either stop or deny innovation nor dynamism.

In “Fact Three”, Doug writes about being onto the next things as soon as something becomes stable enough to be measured in a scalable way. And, he links that to commoditization. The reason other media, particularly television, have avoided commoditization is because they built simple, transparent currencies for the basics and then dedicated resources and creativity to proving why a rating point was not equal all the time in every piece of content.

Commoditization happens when you lose sight of how to prove value. In a world of intangible goods and services and make no mistake, the selling of advertising and the placing of it are intangibles, the smart money is on differentiation through measurement. Without a baseline, you cannot be different or better than something else. Metrics determine benchmarks and baselines against which to differentiate and to do so in transparent degrees of magnitude. Differentiation permits markets to ascribe higher and lower value.

In his “Fact Four” Doug makes the point that “Measurement Supports What We Already Want to Do” and he mentions that marketers use research for support rather than illumination. Perhaps that is true for some. I have seen quite the opposite since the IAB, ANA, and 4A’s launched “Making Measurement Make Sense”, facilitated by the consulting firm Bain & Company and the strategic advisory firm MediaLink. I have been in the room with marketers who believe in digital media’s power to build brands, marketers who want insights and want to understand how to use digital media better. These are not people looking to substantiate dusty old models; these are people who want tools to show that innovation pays.

Had Doug looked into Making Measurement Make Sense and had he talked to either the IAB, the ANA or the 4A’s about the who, what, when and how of this groundbreaking collaboration, he would have been pleasantly surprised. More than 40 senior executives, business leaders, functional experts and thought leaders, from across the ecosystem, have come together to define metrics and currency that facilitate cross platform transactions and also to identify the metrics that put value on elements of interactivity that contribute to building brands. Moreover, the fundamental underpinning of this initiative is to develop a structure for a measurement governance body that will manage change, develop standards, and provide a rapid mechanism for cross ecosystem business needs to be addressed by measurement systems.

Doug, dear colleague, please rethink how measurement, currencies and markets work in disruptive times.

Sherrill Mane is Senior Vice President, Industry Services, at the IAB.

The Moment of Truth Is Upon Us

| | Comments
metrics-blog-header.gif

“All truths are easy to understand once they are discovered; the point is to discover them”

— Galileo Galilei

Galileo’s quote is timeless. In life and in business, the journey—the process of discovery—is the hard part. In life, it is also frequently the beautiful part. In business, getting to that crystal clear, diamond sharp piece of insight that just simply dazzles is the hard part. Making money using it is usually quite simple.

In today’s busy world, where businesses seek short cuts and lowest possible costs and where vested interests often overshadow the best interests of all concerned, discovering truths seems like an overly lofty aspiration. Why expend financial, intellectual and time resources when the status quo, as imperfect as it is, is tolerable?

In digital measurement, why upset the chaos that sometimes cloaks itself as innovation? Why seek truths when various conflicting facsimiles of truths that cost money and render the supply chain inefficient are already part of how we do business?

Ultimately, we pay a high price for not investing in the discovery of truths about measurement. Total media spend could increase by a net 2% or $8 billion (on a 2010 base) in cross platform spend. Interactive advertising spend could increase by as much as 10% if we uncover the fundamental truths of measurement.

In the coming weeks, IAB general members will be called upon to support the cross ecosystem measurement initiative we have been previewing for some time. The opportunity for a coalition of marketers, agencies and media sellers to define the metrics and systems needed to do business efficiently is upon us. The ANA, the 4A’s and the IAB are partnering to develop the next generation of measurement and have selected an independent third party to facilitate the discovery of measurement truths.

This is the moment to embrace Galileo’s thinking. This is the moment to finally discover simple measurement truths that will allow for the right media allocations to flow through a simplified supply chain. This is the moment to invest in the best interests of the entire ecosystem.

We urge you to generously support the effort to discover the simple truths of measurement.

Sherrill Mane is Senior Vice President for Industry Services at the IAB.

Source for estimates: various studies and forecasts developed by Ernst and Young, 2010.

The Very Visible Consequences of Bad Methodology

| | Comments
metrics-blog-header.gif

For the record, interactive media are growing and transforming the way people communicate with brands, with content and with each other. What interactive media do not need is badly executed and poorly reported research. The media themselves already resonate with consumers and the marketing ecosystem.

What specific piece of poorly executed research prompted this? On December 13, Forrester released its North American Technographics Report to great fanfare. Much was made of a finding that simply does not synch up with everything we know using other data sources. According to Forrester, consumers reported that they spent 13 hours a week watching TV and 13 hours a week using the internet in January and February of 2010. Fanfare surrounding the purported parity was nearly matched by the furor over the fact that all other data sources show nothing of the kind. For the same time period, comScore reported that people spend 7 hours and 24 minutes a week using the internet.

The IAB does not need to shill for TV so the fact that Nielsen reports that TV viewing levels are in excess of 30 hours per week is not the point. The point is that we, the IAB and the overwhelming majority of members, stand for transparency and consistency in measurement. We sincerely believe that good measurement must be the barometer against which all media are evaluated.

Putting out numbers that confuse the already confused marketplace—numbers that are based on a methodology that every expert would say is inferior for the purposes of measuring media usage—serves no one. Self-reported usage data rely on consumers’ memories and on their desire to disclose what they do with media. Everyone knows that most people are using the internet at work and spending significant time doing so. We all know that being a “couch potato” is not perceived to be as cool as using new media. We know that people watch TV content on digital media. And, we know that most people are not sitting around making many of the distinctions about media that the professionals do. So why would we expect that people can accurately report their media usage using our artificial business distinctions?

A really thorough review of the Forrester study would also require thinking about the sample. Is it representative of broad population groups? Can the findings be generalized beyond the sample? These complexities are beyond the scope of today’s blog.

Planning and buying media for the purposes of marketing to consumers relies on what people do, not on what they say. To those who continue to confuse the marketplace, we say please consider the disservice you do to all of us. To those who accept counterintuitive information, we say caveat emptor.

Sherrill Mane is Senior Vice President for Industry Services at the IAB.

Nielsen & The Measurement Mess: Why We Need a Fix Now

| | Comments
metrics-blog-header.gif

Last Thursday, we learned that for some time now, Nielsen has been undercounting Internet usage. Nielsen informed clients in a carefully worded message that the company is investigating an erroneous decline in its Internet usage data. The notification went on to say that Nielsen would be working closely with the Media Rating Council (MRC) on remedying the situation. Last week, Ad Age covered the Nielsen story quite clearly.

What is remarkable is that while for approximately eight months Nielsen clients have been alerting Nielsen to the fact that inexplicable, counterintuitive declines in usage have been occurring, the company chose to investigate and communicate findings at its own unacceptably slow pace. When Nielsen did communicate that the data that have been out there for months are wrong, the sophistication and good citizenship shown by working with MRC were appropriate.

All of this begs the question of why doing the right thing happens so very slowly and poorly in today’s ecosystem? More importantly, what can we do to rectify the situation? A voluntary governing body, a FASB (Financial Accounting Standards Board) like MRC with broader empowerment by the entire industry could make a significant difference. For example, an organization like that could have a three month rule, that is, if data appear wrong for three consecutive months across multiple sites, the measurement service producing the data would have to communicate that the numbers are off and explain the steps being taken to rectify the problem.

The latest news from and about Nielsen online measurement just confirms what we at the IAB and our partners at the ANA and the 4A’s already knew: measurement is in dire straits and will stay there unless and until we as a community fix the process. Next time too few people are using the Internet for too little time we should not be looking at almost a year’s worth of bad data and trying to assess the damage.

More on this story from today’s Ad Age »
More on Metrics from the IABlog  |  Previous commentary on Governance in Measurement

Sherrill Mane is Senior Vice President for Industry Services at the IAB.

The Ecosystem Demands It / Your Clients Demand It

| | Comments
metrics-blog-header.gif

You’ve heard it before from measurement experts, media exec’s, and from me.  Today’s digital measurement does not serve our business needs and perhaps its undue complexity actually prevents digital media from achieving full potential as advertising media. As unprecedented change in what technology, media and consumers do together and with each other continues unabated, we are still fighting to establish constructs, metrics, measurement systems and workflows that should have been established more than a decade ago.

You’ve heard/read this before from us (how’s that for building frequency?), but, did you know that the advertisers are issuing a clarion call for Making Measurement Make Sense?  Yes, your clients and in trade association parlance, our partners across the ecosystem agree, in fact, demand that measurement and accountability improve.

In a videoblog posted on October 11, 2010, Bob Liodice, President and CEO of the ANA says, “Marketing effectiveness increases substantially when it is fully and completely accountable. In order to do so, we need fully accountable measurement systems “. Bob goes on to describe initiatives that are underway to improve measurement and thereby make marketers far better and far more accountable.

The one point that Bob does not elaborate on is that the IAB, ANA and 4A’s measurement initiative is as much about changing and reinventing a business process as it is about the specifics of measurement. Making Measurement Make Sense is platform agnostic since all media are digital.

Moreover, Making Measurement Makes Sense seeks to manage the inexorable change that we are witnessing as new devices enter the marketplace and creativity in consuming and communicating abound. Along with the ANA, the 4A’s and other potential partners, the IAB will continue to strive to establish metrics, systems and processes that make sense for today and tomorrow.

Sherrill Mane is Senior Vice President for Industry Services at the IAB.


Promoting Differentiation Through Metrics That Matter

| | Comments
metrics-blog-header.gif

Following the publication of my Op Ed piece in Ad Age Digital Next on September 20, entitled, “Why Measurement Is Still Screwing Up the Online Ad Business: The Problem Can Be Fixed, but It Will Take Your Help” (see the IABlog for the long version), my inbox filled with all manner of encouraging notes. Ad Age received some interesting comments as well.

iStock_000014233733XSmall.jpg

Encouragement came from many quarters throughout the ecosystem and varied from high praise to offers to heed the call to action. On the less encouraging side, it seems that there a misperception by a few about the Making Measurement Make Sense initiative and how digital media measurement should look relative to other media. The misperception is that we advocate measurement that diminishes from today’s data and systems’ capacities to capture what is uniquely powerful about digital media.

In no way, shape or form, do we recommend decelerating the pace of transformation, innovation or the capacity of digital media and digital consumers to change marketing, media, content, brands and consumer relationships to them. If anything, we propose that all media be viewed as digital media and that digital media as we know them today have metrics, data, systems and standards that permit efficient transactions within and across digital and traditional media. The outcome to this process could well be changes to measurement for both digital AND traditional media. All options are on the table.

We openly advocate identifying and developing metrics and systems that will permit efficient transactions. This implies that digital media be in the consideration set for all media plans because digital media will be bought and evaluated in a manner that facilitates the right budget allocations.

The right budget allocations are the ones that work best for the brand at the time. Getting to them involves tools that permit evaluating all options within common parameters and finding the ones that stand out for the brand, the budget, the time…the ones that are differentiated and that permit the brand to be differentiated. Today’s convoluted digital-only metrics don’t so much differentiate interactive advertising as silo it away from the rest of the media world.

Each segment of the marketing ecosystem deals with differentiation in its day to day business activities. Most businesses do. The trick to understanding how an individual item is differentiated from others in a given category is having metrics with which to do so. To fully understand the richness and the power of digital media in the marketing mix, we must create a common language around metrics and measurement that permit identifying differentiating characteristics. That is what Making Measurement Make Sense will do.

Sherrill Mane is Senior Vice President for Industry Services at the IAB.


metrics-blog-header.gif

Almost as outdated as “hits” as an audience metric, page views have somehow managed to hold on far after their time.  ComScore and Nielsen still report them, agencies still look at them, and publishers that are heavily rich media-oriented struggle to derive “page view equivalents” from their audience numbers.  All for a metric that had relevance when we lived in a world of HTML 1.0, but really, not so much today.

93436194_7fd9374236x350.jpg
Photo source: http://www.flickr.com/photos/richy/93436194/sizes/o/in/photostream/

During the process of creating the IAB’s audience reach measurement guidelines last year, page views were on the table, but the consensus was that in the rich media world of today’s web, it does more harm than good to reduce things to “pages.”

Now Google’s made the page view even more obsolete with its new Google Instant Search capability.  If that catches on and becomes the way most people use Google’s search engine, Google’s page view counts will plummet:  instead of a search page and a search results page, both the query and the results appear on a single dynamic “page.”

I realize that Google’s revenue model doesn’t depend on the page view metric, so it probably doesn’t care what Instant Search will do to its page view count (By the way, it’ll also be interesting to see what Instant Search does to Google’s time spent metric.) But as the web continues to get richer, UI changes similar to Google’s will proliferate in large and small ways on other publishers’ properties, and page view counts will grow ever more misleading as a way to understand audience size, activity level, and advertising opportunities.

I’d love to see agencies, measurement vendors, and publishers overcome the collective industry inertia that seems to resist abolishing the pageview.  Clinging to it just because it’s the metric you looked at last year (and the year before that) isn’t a good enough reason to keep it around when so many better and more representative metrics are available.  Granted pageviews do have a legitimate use as a metric for comparing between server-side analytics and panel-based measurers—but that’s a wonky, internal reason to keep them around, andt doesn’t justify their use to assess how media companies are doing, or as a part of the planning process. 

Anyone want to defend the pageview?  Or suggest a good replacement for it?  I’d love to hear from you.  Feel free to post your thoughts here.

Joe Laszlo is Director of Research for the IAB


Radical Rethink: IAB Proposes Solution to Digital Measurement Mess

| | Comments
metrics-blog-header.gif

An abbreviated version of this article was posted to AdAge.com on September 20, 2010.

The IAB, the ANA, the 4A’s and our member companies, and the entire marketing ecosystem for that matter, recognize that it is imperative to provide solutions to one of today’s most vexing marketing and media industry challenges: how to measure and evaluate interactive media. While there may be infinite ways to creatively deploy digital media, quantification of effects from basic exposure to brand-building, is seriously hampered, compromising the ability of marketers, ad agencies, and publishers to be accountable for advertising expenditures.  The lack of consistent approaches for measuring exposure and assessing the effects of digital media has led industry to a cacophony of competing and contradictory measurement systems, contributing to a complex and costly supply chain.

For too long, we as an industry have fueled the problem and done little to suggest solutions.  Some of the reasons for the digital media got to the today’s measurement mess have to do with both the entrepreneurial spirit that is characteristic of digital media and the innovation that Professor Clayton M. Christensen (Harvard Business School) describes in his work on disruptive technologies.  Digital media are now at scale. It is time to solve the measurement mess and live up to the promise of being the most accountable medium. The ANA, the 4A’s and the IAB have a proposed solution and it requires the support and collaboration of the entire ecosystem.

As all media become digital, and more and more consumers migrate to digital media, today’s measurement complexity is preventing marketers from following their consumers with the appropriate degree of comfort that their marketing objectives can be met.  The confusion surrounding measurement and the related lack of currency adds costs to advertising agencies as they are forced to use, subsidize and staff around more and more metrics and data in order to plan, purchase and post-analyze their media buys.  Media companies have difficulties pricing their time, space and services, and digital media are not able to compete effectively for share of budgets.

Digital Media Measurement Today

Online media are blessed and cursed with an abundance of metrics.  There are multiple vendors producing ever-growing quantities of data that often do not agree with each other, even directionally. And, more importantly, there is no coin of the realm - no “currency” - that governs the planning, buying and delivery of guaranteed audiences for advertising flights. 

Media planning, the critical stage in translating marketing strategy into media outreach is done using syndicated audience measurement data that is often wildly disparate.  Syndicated measurement services are the providers of competitive data and thus, of the currency of media planning.

Sellers use syndicated data as well as site-census data.  Frequently, sellers rely on census data for traffic counts.  Census data always yield different numbers than syndicated measurement, which incorporates samples.  And the sampling methodologies may or may not consistently yield representative samples.

The currency for actual advertising buys is ad impressions served by ad servers, a dataset that does not coincide with the other datasets used in the planning and buying process.

Furthermore, internet measurement confusion is confounding the process of planning, buying and evaluating cross-media platform advertising campaigns.  Current tools are inadequate and continue to frustrate the marketplace.  For example, the debate over whether or not Gross Rating Points (GRPs) and reach and frequency calculations should be used across platforms still goes on, and is even more sharply felt when attempting to compare, buy, and analyze online video and offline TV advertising.  All the while, it is still difficult to determine R&F’s of display ad campaigns across sites and ad networks, or stated another way a simple R&F within digital media alone is challenging to obtain.

The cacophony of confusing numbers is amplified by the fact that digital media also are evaluated and bought on direct response metrics like clicks and actions, contributing to the misperception that online media are not brand-hospitable.

Brand metrics that are agreed upon and capture data that marketers want are lacking.  In addition, there are no widely accepted metrics that permit quantifying the relative contributions of online and traditional media advertising to building brands.  Thus, there is no measurement approach or system for assessing brand effects and ascribing value to the various media.

Confusion about metrics contributes to two key areas of risk for the media and advertising industries.  The first is the risk to marketers and their brands if they are unable to follow consumers’ migration to interactive media. Currently, according to Forrester, consumers devote approximately 34% of the time they spend with media to interactive.  Yet ZenithOptimedia data show that in 2009 US marketers disbursed only 5.7% of their total ad (media plus marketing) expenditures on interactive media, a mismatch that speaks in part to the difficulties in planning, buying, and analyzing the effectiveness of those spending decisions.

Another risk is the threat of government intervention in digital media measurement. The Federal Government has intervened in broadcast advertising measurement in the past, and is currently exploring, at the state and Federal levels, industry and interest group complaints about miscounting of minority audiences in broadcasting. Minority audience measurement may well become a challenge to digital media, especially as consumers spend increasing amounts of time consuming video via interactive devices. Already, the Federal Government’s scrutiny of consumer privacy in interactive advertising environments threatens to undermine industry’s ability to obtain accurate site-census counts of audiences.

The Solution

The ANA, the 4A’s and the IAB believe that the solution requires a radical rethinking of how measurement is viewed and defined by the marketing and media businesses.  We need a cross-industry coalition of experts drawn from consumer marketing companies, advertising agencies and media companies that will identify and create consensus around the metrics and systems that are needed to run our businesses.  While inspired by the findings of a report (“Digital Marketing Metrics:  How Can Innovation Unlock More Dollars?”) done for the IAB and the 4A’s by McKinsey & Company, the structure, process and objectives of the joint solution are a distillation of many voices of many thinkers throughout the ecosystem.

The solution must embrace the needs of all digital media in a platform agnostic fashion.  There is room to embrace and refine TV and print and display and mobile centric approaches.  This figurative hug can only be extended if the more platform specific approaches can be crafted into a broad solution for counting and evaluating digital media and then going cross platform with metrics that matter.  Ultimately, we as an ecosystem, as a community of leaders must create metrics and measurement systems for transacting business in way that yields the right budget allocations.  Good selling is not about asking for fair share; it is about proving the value of the right share.

Measurement should be conceptualized as a business process that incorporates research science rather than as a research process that seeks support from the businesses.  And that business process must be one that incorporates a vision and a structure for change management, innovation, and quality assurance.  Discrete research RFP’s alone cannot fix the problem.  In short, we need a governing body, the media measurement equivalent of FASB - the Financial Accounting Standards Board - an industry-wide body that sets standards and ensures that measurement will be consistent and easy for the sellers and buyers of advertising to apply transactionally.

FASB was formed over 35 years ago and is the designated private-sector organization to set standards of accounting and reporting applicable to non-governmental entities.  FASB standards are officially recognized as authoritative by the Securities and Exchange Commission and the American Institute of Certified Public Accountants.  Centralized standards of this type are important to business because they establish a trusted and understood framework for allocation of resources in our economy.  FASB is independent from business and trade associations but seeks wide input into its decisions through a broad set of advisory groups.

In our industry the Media Rating Council (MRC) could play the role of FASB, but it would need to be empowered to do so across media which would take broader support from marketers, agencies and media organizations for this governing role.

It is mission critical that measurement and metrics stop being the favorite scapegoat of the media business.  As an ecosystem, we must stop talking the talk and begin walking the walk.  Media, both traditional and digital, have long blamed measurement for multiple revenue maladies.  The appetite to systematically tackle the business process, the governing body, the identification of metrics and their conceptual and operational definitions and building the systems we need may not be genuine.  “What would we do without measurement as we know it and so love to hate?”

As business leaders from across the ecosystem, we must identify and define the metrics that matter, metrics that take us from counting exposures to valuing exposures to contributing to brand health to putting paid and earned media together in creative cross platform campaigns, and providing easy to use post buy analyses.  Yes, you just read that long bombastic sentence.  And you will read it again because it captures the dire needs we have today.  They will be exacerbated if we do not solve them NOW.  

A Holistic Approach

While the IAB along with its members and the MRC have made much progress in creating standards and transparency in digital measurement, the work needs to be accelerated and to tackle the broad issues outlined here.  As an industry, we must further integrate efforts to fix measurement.  Only a holistic business process can get the necessary traction and only the right governing body can assure that good work is ongoing as continual change forces innovation.

The costs of proceeding independently with various and sundry initiatives with no overarching business process are high. An already confused marketplace may become further confused.  The costs of doing business on the agency side may increase, thus reducing the agencies’ capacity to fully assimilate the data streams and types into media buys, thereby hurting the ability to buy right for their clients and ultimately reducing the consideration set of media entities that go into the planning and buying process.

We invite your support for Making Measurement Make Sense NOW. Join us as we solve the problem and build the process and governing body to make this happen FAST.

Sherrill Mane is Senior Vice President for Industry Services at the IAB.

Online By The Numbers

| | Comments
metrics-blog-header.gif

The IAB is pleased to announce a new members-only feature on IAB.net: brief reports highlighting useful and sometimes fascinating nuggets about online audiences, based on audience measurement data.  They discuss major trends, examine key demographic or behavioral segments, and examine the impact of real-world events on online audiences.

Our inaugural report looks at the World Cup’s impact on audiences.  According to Akamai, the 2010 World Cup drove the three largest traffic spikes since the inception of their News Net Usage Index five years ago.   We looked for the impact of the World Cup in the monthly data: web audiences to sports sites were 21% higher in July 2010 than they were in July 2009, driven at least in part by the World Cup.  Yahoo! Sports, ESPN.com, and Univision.com also saw year-on-year traffic increases attributable to the World Cup.  FIFA.com increased its audience particularly dramatically, becoming the eighth largest sports site in the US for July.

In addition to the World Cup data, the first IAB audience metrics report looks at the overall top ten sites as measured by Nielsen and comScore; the properties with the largest video audiences; and online audience data for some specific demographics (younger men and women, and affluent users).

View the September 2010 Audience Trend Report

I welcome questions and creative suggestions for future report topics and directions. Feel free to contact me at [email protected] or use the comments below.

Joe Laszlo is Director of Research for the IAB

Improving Research Practices and Reducing Inefficiencies

| | Comments
metrics-blog-header.gif

If you follow the world of media and advertising research, you may have heard something about a debate set off by last week’s release of an IAB commissioned report entitled, “An Evaluation of Methods Used to Assess the Effectiveness of Advertising on the Internet.” If you’re not a media or advertising researcher, you probably skipped over the Twitter stream, the trade press and the numerous blog posts about the paper. As a digital salesperson or business development executive or agency chief or CMO, you likely figured it had nothing to do with you.

That assumption would be incorrect. This methodological review, a completely independent one, at that, conducted by the esteemed social scientist, Dr. Paul Lavrakas, has everything to do with you and your company. Here’s why…

With permission from Dan Murphy, SVP Interactive Research & Ad Traffic, Univision Interactive Media, we cite the following. “The IAB study was initiated a while back because publishers were being inundated with large numbers of studies - in some cases so many, users were getting multiple survey requests on the same User Session. (Pop-Up blockers anyone?) If the publisher didn’t accept the study, they were told - No Campaign. Putting the issue of recruiting sample with a pop-up window (i.e., a variation of a pop-up ad) to the side, many had questions about sample frame issues from Power-Users, Shared Computers, Users of various bandwidths and so forth. Concurrently, many of us (both buy and sell-side) were seeing peculiarities in the output.”

Simply put, the independent review was taken on to solve a business problem that had become more pressing over time. Should the industry be forced to continue to use research that may or may not be doing the job in order to be included on a buy? Does the research need to be improved or changed in order to preserve the quality of the consumer experience? After all, if as an industry, we do not make the consumer the center of our universe, what kind of businesses will we have? And, lastly, what guidance is being given to the agency staffers who are forcing these studies upon the sellers?

No research for any medium is perfect. We at the IAB did not expect perfection when we commissioned Dr. Lavrakas. We got what we expected: an objective evaluation that clearly expresses confidence in what is good, critiques what is bad and goes on to suggest steps for improving site intercept ad effectiveness research. In addition, the IAB and our member companies are committed to using the paper as the foundation for Best Practices work that force both buyers and sellers to be wiser about how the studies are used.

It is time that as an industry, we invested in making ad effectiveness research better; in moving from ad hoc solutions that sell well to buyers who may not understand of the limitations of studies; in truly building a body of work that proves how well the online media build brands. We cannot get there if we continue to ignore the need to think big, get out of our silos and recognize that we must raise the standards, the practices, the discourse itself to a new level.

Let’s embark upon learning and applying the learnings of Dr. Lavrakas’s work to solutions that make online media attractive destinations for consumers and for brand advertising dollars. And, in the spirit of collaboration shown by the vendors who let us look under the hood for this report, let’s move forward together so that we really can get to the next level.