Joe Laszlo: October 2010 Archives
Over the past couple of years, several IAB members have come to me with examples of measurement vendors claiming they have been endorsed, approved, or otherwise given the blessing of the IAB. This bothers me, because the IAB is (and I hope it’s perceived as) neutral when it comes to vendors. We may specify guidelines or best practices for a given measurement, but we don’t take sides when it comes to the companies doing the measuring. So if any vendor tells you they’re our BFF or whatever, you should push back (and let me know).
An even more insidious issue has come up lately related to the report Dr. Paul Lavrakas did for the IAB evaluating the methodology of certain types of Internet Ad Effectiveness research. I’ve seen a couple of instances, and heard of still others, where vendors using methodologies outside the scope of Dr. Lavrakas’s report are citing that report as a way to validate themselves. This is a terribly misleading assertion! I want to strongly caution against this—as indeed Dr. Lavrakas himself does at the start of his report:
Dr. Lavrakas’s findings should not be used to validate or invalidate any methodology other than the ones he explicitly examined: i.e., long-form questionnaire-based surveys taken by samples of exposed and non-exposed consumers recruited via either online site-intercept invitations or via an online panel.
The statements I’ve seen boil down to “Methodology Y avoids sources of uncertainty identified in a review of methodology X, therefore methodology Y is better.” Of course, this says nothing about other issues that methodology Y may have, which may be even more serious. Wonky research people and born skeptics will know this instinctively and this kind of claim should raise red flags for them. But folks outside the research community may be taken in.
I could name names in this post. But I won’t. However, I do want the vendor community to know that the IAB is being vigilant on this front—and we’d like you to avoid referencing Dr. Lavrakas’s report for marketing purposes.
Joe Laszlo is Director of Research for the IAB