At the ARF senior executive workshop in Chicago last week, Steve Levitt, author of Freakonomics, turned the tables (as he is known to do) and asked the attendees a question during his Q&A.
"If you know how to measure advertising ROI, why don't you do it?"
The room was silent.
And many of us who were there continue to be haunted by both the question and the importance of finding the answer.
Steve addressed his question to a room filled with very impressive senior executives representing a host of blue chip advertisers, agencies and research companies plus a number of distinguished academics.
Why would nobody engage Steve in this discussion?
Finding the answer to that question may be the biggest challenge our industry has to deal with these days, and that is why I am writing this piece.
As the silence became deafening, I asked the president of a research company that specializes in ROI consulting to comment. And, he graciously agreed to do so.
He said, "Indeed, my company knows how to measure advertising ROI, and we have been doing so for over 15 years for many major advertisers.
Steve Levitt then asked him, "What percentage of the $200 billion plus spent on advertising is subjected to ROI analysis?"
He thought it was "about 1 or 2%". A recent CMO Council survey reported that only 17% of companies have "measurement systems in place that perform this calculation satisfactorily."
So, what is going on?
The ensuing discussion by the group unearthed some possible hypotheses.
The most concerning, if it is true, is that the principals involved in advertising decision- making don't want to know the ROI; no matter how obsessed their ceo's may be about finding that out.
What do we do if we find out that it doesn't work?
What if we learn that advertising's economic value is far less than we thought?
What new models and processes do we then have to develop? What do we say to our ad agency; to our media agency with its sophisticated optimizer models; to our research partners whose tests scores predicted that the advertising would be effective?
And, how do we convince top management to keep spending all that money on advertising year after year?
"Stonewalling the ROI project may to be the best survival strategy", somebody opined.
Another hypothesis is that the ROI answer is not easily found in a drawer somewhere.
"We use marketing mix modeling to guide us on switching money from expensive TV to inexpensive search which makes our media delivery more efficient which is a good surrogate for improved ROI", someone said.
Beyond the increased use of modeling, there are other initiatives going on.
The ARF, ANA and the AAAA's are collaborating on the study of engagement as the holistic "GRP of the 21st century".
The ARF has been working hard for over a year on the development of cross-media measurement guidelines.
There are two more ARF "Advertising: What's Next?" senior workshops scheduled for November and December that will continue to address the issue. Both of these will feature more from Steve Levitt on the subject.
Steve Levitt told the Chicago group, with no equivocation, that he believed that advertising ROI could be measured and the question "does my advertising work or not" answered.
"I am sure it can be done. And, I'm willing to help the industry do it", he said.
Steve's hope is that we can find advertisers in 2006 that are willing to collaborate with him and engage in serious experimentation, data analysis and hypothesis testing.
As an industry, we cannot just leave the issue sitting out there. We have to find a new way to think about it, and there is no better partner than Steve Levitt to help us.
Let's make that happen in 2006.