Interview: Jack Honomichl on the State of Online Research

PDF version available here.

Jack Honomichl is considered one of the foremost authorities on the research industry, and for good reason. The Market Research Council, an organization founded in 1927 to foster growth of the advertising/marketing/public opinion research industry, credited Jack Honomichl as the person who “defined the market research industry” at his June 21, 2002 Hall of Fame induction ceremony at the Yale Club in New York City. Mr. Honomichl shares Hall of Fame status with other inductees who have also been recognized for their extraordinary contributions to the industry, including renowned research authorities: Arthur C. Nielsen Sr., George H. Gallup Sr., David Ogilvy, Marion Harper, Daniel Yankelovich, Daniel Starch, Ernest Dichter, Alfred Politz, and Elmo Roper.

Since 1969 Mr. Honomichl has written about the market research industry, as publisher of the first research trade journal, The Analyst, and Advertising Age research industry columnist for 18 years, and American Marketing Association’s (AMA) Marketing News research columnist for five years. He authored an industry textbook, Honomichl on Marketing Research, has written nearly 400 articles about the industry in the trade and academic press, and has been a featured speaker at national research industry conventions and trade association conferences, and has guest lectured at several universities.

On October 9, 2007 JOR Managing Editor Steve sat down in conversation with Jack to get his perspective on the state of online market research.

Transcript Provided By:
Transcription Services, Inc
(909) 484-8185
www.tsitranscripts.com

SA: Thank you for joining us today, Jack. I’d like to start off by taking stock of the current state of online research and how it currently fits into the market research mix. Online has been a viable option for fielding research studies for almost a dozen years now. ESOMAR reported that in 2006, out of $24 billion US dollars spent worldwide on market research, $3 billion US was spent on online market research, a 14% increase on the year before. What is your take on these numbers? Do they match what you are seeing? Is there more to the picture of where we are now with online research?

JH: Yes. The data that we prefer to use is that developed by Inside Research where we get the major users to provide us in a confidential basis, their spending via various methodologies including online. We break that down by the United States and Western Europe. Let’s talk US only for a minute. In 2006, online spending was up by 19% and the estimates we have for 2007 are up 15%. That’s pretty healthy, but it shows the start of a topping off if you will. The growth rate was very; very steep in percentage basis in previous years, but now logically it will start to plateau. In Europe we see pretty much the same picture. The one basic difference is we’re talking Euros instead of dollars. Growth in all the Western European countries put together was about 54% in ‘06 and 29% estimated for ‘07. So you can see that it’s following kind of the same pattern. But it boils down to this, that in the US about 36% of all survey work done was via online data collection in ‘06 and the estimate for ‘07 is 39%. Those are the numbers we go by. We have reason to believe they are accurate except of course they are not all inclusive. They don’t represent every last penny that’s spent out there. But as far as the growth rate is concerned I’m sure they are quite accurate. People seem to agree with us on that. There is a difference though – it might be of interest to your readers – there hasn’t been universal acceptance of online for various types of projects. It’s different in Europe than it is in the United States. For instance, the Europeans apparently have a different take on the value of online. In the United States, the leading usage is concept and product testing. In Europe, the leading use is also product and concept testing. But when you come to the next most important thing, in the United States it’s sales tracking and in Europe it’s customer satisfaction measures. So you go on down all the possible applications: A&U studies, advertising brand tracking, opinion polling, sales tracking and so on, there is quite a bit of difference. For instance, in the United States, sales tracking is number two. In Europe it’s way down the list, only about 6% of the total. So they’ve adopted it, but with a significant difference in the application they put it through.

SA: So concept and product testing is still the number one use on both sides of the pond.

JH: Both sides, yes. If you add them together for the world, that’s it by far. And I think the real message for this is that when we talk about the embracing, if you will, the concept, the data collection online, the beauty is inside the beholder. They see it as – their experience has told them it works well for this application, but it is not so good for other applications that sorted itself out over time; you know, trial and error. This is the way it has kind of shook down. In the United States, just to refresh your memory, the listing goes concept product testing, sales tracking and A&U studies, brand test tracking, CSM (which is very big in Europe), copy testing, site evaluation, user profiles, opinion polling, qualitative focus groups and others. But by far, a third of it is just concept and product testing.

SA: You mentioned qualitative such as focus groups. Are you seeing any changes in that? Is there more wide adoption on the qualitative side of the coin as well as the quantitative side of the coin?

JH: It’s interesting because if we can talk tangentially, as you well know, there has been a splurge of interest in recent years with the so-called “proprietary panels” and especially those that are called “community panels.” For those people who aren’t familiar with community panels, a client owns them. They recruit them. They might recruit them from their own customer body and they are really used as sounding boards. They are a two-way conversation link with knowing customers perhaps. These community panels are almost entirely used for what I would call “qualitative type insights” as to how your best and most important customers are reacting to new product concepts or a new advertising campaign or a new product, whatever. So in that sense, the qualitative end of it is getting a very special shot in the arm because of the growing acceptance and usage of these so-called proprietary panels which are owned by the client, and are focused on a very specific population. Quite often that population is comprised of their own customers selected or taken from a sales list or recruited at the point of sale, or something like that. It’s not so much the quality of the sample as it is the very richness or concentration of the sample in the sense that you know these are our heavy customers. Now we just want to tickle them with a new logo, a new this or a new that or a new product concept and see how they bounce. But we want to have two-way communication always through the process. The growing use of online for what I’ll call qualitative inquiries will have a big impact over time.

SA: Are you seeing any numbers around that in terms of percentage growth in that particular…

JH: We set up a panel of people who are – companies that are specializing in the sale and development of these so-called proprietary panels of both community and non-community. That was up by 26% by midyear in ‘07 and its projection should be up 37% by midyear ‘08. And the projection for October is up 37%. Revenue is getting close to $100 million, which is pretty sizable. I liken it very much to the original growth of customer satisfaction measurement which started off with just a few firms, highly specialized, developed a concept and methodologies and then everybody and their brother jumped on the bandwagon and said, “We can do that, too.” It became a big growth sector for the industry. I think these proprietary panels could be the same thing; they show all the signs of growing and developing, just like customer satisfaction measurement did.

SA: That’s very interesting that qualitative is starting to make an impact as well. Speaking of impact, can you assess the impact – favorable or otherwise – that online has had on the market industry in general in terms of benefit to methodology – are we doing more useful work, serving our clients better? And in terms of how online is growing our industry as well.

JH: Well my impression – and I underline the word impression, is that online was embraced the way it was for basically two reasons. One is that most research managers are being pounded by their masters to be fast and quicker; do things overnight and online promised that. The second was cheap, it’s not as expensive. And that was a big lure: fast and cheap. They jumped in because both of those were huge incentives. Then it was only after they got into it, that they started to realize “Well, it can be fast and cheap, but no good at the same time.” Then the sorting out process I described earlier – it’s good, okay for concept, product testing, but it’s no good for say polling, kind of sorting out came. And now we get down to that it’s matured to the point where people can look at it and say, “Well, it’s a fact of life, it’s established. We kind of sorted out the types of studies we’ll be using, but now what do we think about the quality of the data?” It turns out that’s quite a concern. We, being Inside Research, have built a panel of directors of research, very major buyers – 35 people who run the largest research company departments in the US, basically. It’s not exactly true, but that’s the essence of it. We asked them recently about their concern about the quality of online panels. The majority of them said it was one of their special concerns. Over half of them said it was “one of their special concerns” in running their operation and about a third of them said it was “the most pressing concern.” We asked them if they’ve seen improvements in online quality. The answer is “No, not necessarily. It’s staying about the same and, if changing at all, it’s probably deteriorating,” How do end clients face up to this? . Well, some of them – first of all, they do the usual thing; they say it’s the vendor’s problem. “We expect the vendor to solve it.” But the essence of it is that they have been very reactive to the vendors saying, “We are doing our own R&D. We are concerned about respondent quality, representatives and so on. Here is what we are doing; here is how we are trying to improve or even eliminate the bad stuff, the professional respondents and that sort of thing and improve the quality of the sample.” Now the research companies, the vendors, who talk that way and bring R&D projects in and so on, are getting the business. The other guys who aren’t paying much attention to that are just saying we are cheap and fast, might get the business in the first place, but eventually they will lose out to someone who says, “Hey, we’re taking this seriously, this sampling problem, we’re trying to rectify it or at least keep it certain parameters.” These guys say that they very definitely have dropped vendors who didn’t show that willingness to work on sample quality and give more business to those who are in fact doing in-house R&D, trying to find ways to improve on the process. That’s the way the world works. People eventually move their business to where they feel more comfortable. They want to deal with people who are serious about the problems, who really want to do a good job. They gravitate away from companies whose main pitches were all about ‘cheap.’ That should be no surprise. But there is no question that to research directors, this is a real worry. And especially some are after the revelations of how much overlap there was between the online panels – and we have professional respondents and people doing this to make a living practically, getting remunerated and being parts of five or six different panels. That’s been documented by Comscore. And that was a very sobering piece of information and it shook a lot of people up. Of course there have been all sorts of symposiums and seminars and conference programs devoted to the subject in the last few months and into the future on that very subject. So people are worried but as usual the buyer says, “Well it’s the seller’s problem and he’s got to do whatever is necessary to solve it.”

SA: Is there any indication of willingness on the buyer’s side to pay a little more for what they perceive as more quality, or is price pressure still there?

JH: That’s always the question. That was the question back with telephones. It will always be the question. “They want it, but do they want to pay for it or are they willing to pay for it?” I don’t have anything substantive documentation to put towards that. But what I do see – and this is probably true in all the research they do, a handful of end users take it so seriously that they will say assign one person to the subject – let’s say quality of online samples. They actually invest in someone inside and make them a warden or a monitor to work with the research vendors on just that subject. Well, that costs money to take one staff and in effect make them the in-house expert, if you will. Most marketing companies won’t do that. You would hope they would. One of the problems getting an answer to the question you asked, Steve, today is the expanding role of the purchasing department. Those guys bring a whole new dimension to the process. They aren’t nearly as swayed by some of the subtleties that research people would recognize and appreciate. These guys operate on a cost basis as if they were buying a commodity– there is no black and white; everything is just gray to them.

SA: So thinking past the sample issue, which is obviously a bit concern.

JH: That, and professional respondents, yes.

SA: In terms of the actual information that research clients are getting – they are working with a vendor who is paying attention to sample quality and professional respondents…

JH: They tend to gravitate to them over time.

SA: Then for those people, do you get the sense that the end clients are feeling they are getting what they are paying for for online? Are they satisfied with it? Are they getting the information they need to do their jobs?

JH: If they keep buying it, you have to conclude that if not entirely satisfied, they find the deliverable at least tolerable. I think that’s the proof of the pudding. They can say anything they want to, but if they keep buying it at the rate they are going and it keeps continuing to increase the amount of usage, you have to conclude that it’s serving their purpose. Now deep in their hearts, they may find it inferior or flawed. But perhaps it gives them a speed element that makes them heroes to their marketing management; that alone makes it all worthwhile. So I would say it’s serving its purpose. But it does bring up to me a very famous saying. I don’t know if you ever took courses in economics when you were in university, but if you did you probably ran across Gresham’s Law. It said quite simply, “Bad money drives out good.” That’s an economic truth. In that sense, online research could be the bad money that is driving out good research practice. But it does turn right around and get accepted because it has this evil virtue of being so fast and relatively cheap.

SA: But is that to say then that it is inherently inferior because of the medium?

JH: If the management doesn’t appreciate the difference in quality, and doesn’t even want to listen to it, who knows?

SA: So what you are saying is there is a difference in quality.

JH: I don’t want to say that. I guess you’d have to get right down to any particular study done on any particular type of project for any particular client. Generally speaking, we do know that there is a lot of concern among people who really understand good research methodology from an academic point of view. We’re getting further from that standard of excellence. Does that mean that there is no utility in the data? I didn’t say that at all.

SA: Can you address some of those concerns? Obviously the sample is a big one. Are there other concerns besides sample?

JH: I have my own hunches, but I don’t have any numbers to look at to tell you something solid. . I’d rather stick to areas where we’ve made inquiries and have some numbers to back-up what we’re saying. I’m much more comfortable doing that.

SA: Returning to the sample issue, what do you think can be done? If you had to prescribe something to the online side of the industry what would you prescribe on the sample issue?

JH: I would say that if you want to stay in the business and be a major player through time, you should spend money on internal R&D. BASES, in my opinion came to dominate the market modeling business for a simple reason, they spent so much money on R&D and they worked closely with the clients. They told them exactly what they were doing and why they were doing it. They invested the money, which also included hiring some very high-powered academics to go off and sit in the corner and work on these problems. Over time it really paid off because the clients came to believe in what these guys were doing.. Of course, BASES is basically in the business of forecasting the acceptance of new products and there was proof in the pudding. Did BASES predictions work out well? So there was kind of – like an election. There was a result and everybody could say yes, BASES was right. But the biggest thing was that didn’t just happen. BASES made it very clear to their clients that it was an on-going process; you just don’t develop a methodology and say there it is and that’s it, see you around. They kept pumping money in and they continue to this day to pump money in to R&D. Going full circle, Steve, I think if someone wants to continue to be a major player in the area of supply and online survey research, they too, will take that posture and spend the money, bite the bullet, do the internal work, have people who specialize, concentrate on this business of quality and show to their clients what they are learning and how to apply it and why they are doing it this way versus that way – they will prevail. And in the process, the quality of their product will get better and clients will believe it’s a better product. So many research companies tend to come up with say a methodology in the case of customer satisfaction. Then they want to just mass-produce it. “This is what we do; take it or leave it.”

I think especially in the case of online, the field is just hungry for people who say, “We realize this is an ongoing process. We’re up to our eyeballs already in it, now let’s find out how we can improve or get rid of some of the problems or alleviate some concerns” and talk openly with the client about it and share with them. They’ll prosper in the long run I think.

SA: Let’s turn from the present and look to the future. Looking into your crystal ball, where do you see online research in five years or ten years out from now?

JH: When we start talking five or ten years out, it’s a fascinating subject to me because things change, especially in today’s world of technological innovation. For instance, we looked at TV and radio audience measurement. One of the largest expenditures for survey research in the United States is traditional TV and radio audience measurement. Traditionally measurements are based on diaries. Then they went to meters and now they are going to personal meters. They find miniaturization has made it possible for someone to carry a little cartridge about the size of a pack of cigarettes around on their body that will record the signals they are exposed to; whether they are in a bar or in their car or wherever they are. This is just now after years and years of development and again, R&D money and so on, it’s just now starting to be implemented in the field in various countries around the world including the United States. It was a transition project that literally started ten years ago. But science made possible the quality of equipment and the miniaturization of the equipment and eventually the lower cost of the equipment so that you can buy it in large numbers and equip panelists with the little portable gadget. You got to look out and say technology can strike from surprising cases. If I had talked to you ten years ago and said do you think some day your child at home will have a pen pal in China and send a thing called an email back and forth to them, you would have said you are crazy. But it has come to be.

SA: So you see technology really is going to drive where…

JH: What we don’t know in the research business really is what’s going to strike again. None of us does. Don’t tell me you knew – go back ten years ago and say that you will come to work every morning and find 100 so called emails on your computer. You would have laughed. But here it is. We don’t know ten years down the road what other technology – which is maybe already developed, has just gotten to the point where it’s cost effective or they can’t perfect the miniaturization part of it that it hasn’t hit yet. It could come along and change everything just as online changed survey research. So when you talk five, ten years out, you got to always say “What kind of technology is lurking in the wings?” It’s just a matter of time and we get one step closer to some sort of Buck Rogers kind of breakthrough that enables us to gather data in ways we never dreamed of. It’s possible. That’s exactly what’s going on in the world of TV and radio audience measurement. It has finally come. People are going to wearing around a little portable receiver that will record all their signals and enable to say that this is what they listen to during the day, commercials. You embed the commercials with a special signal and you got their commercial exposure and away we go. It’s very conceivable to say someday that that receiver could be the size of say a silver dollar and that millions of people could be wearing them all the time. Why not?

SA: And do you see that methodology will evolve, too?

JH: Methodology will follow the technology is what I’m trying to say. You talk about samples; the samples you learned about when you were in college; statisticians and probabilities and all that nonsense. Go to a brand manager and say there are 100,000 people in this panel – and he’s not going to think panel. “Well, how could 100,000 panel size be wrong?” There are that many people out there doing this. “But Nielson samples 2,000.” “Yes, but this is 100,000.” That’s the way their minds work. If you get that number up to there is 500,000 people wearing this little gadget, we’re apt to believe that more than he is going to believe a smaller “sample” in a minute. I think people in the research business have to face up to Gresham’s Law, again.

SA: At the IIR Market Research Event in San Francisco in 2004, I attended your speech calling for less customer insight, which you described as vague and based loosely on intuition, and for more customer intelligence from the market research industry. In your opinion, has online research contributed to more customer intelligence? If so, do you see it as something that is helping to serve the mission of market research?

JH: To me, this whole insight thing to me is a regression. You are going backwards. If you looked up “insight” in the dictionary, they use the word ‘hunch.’ Now do you really want to make big basic investment decisions – like taking a new product to market or building a new plan or something – based on hunches, like something a woman happened to say in a focus group? Or something she did in her kitchen? I mean, do you really want to base that expenditure on an “insight” or do you want hard data?

Hard data suggests that it could come from a lot of sources in addition to what you and I know as traditional marketing research. I think if marketing research people or the industry has a flaw, its being too much wedded to this procedure we call ‘research.’ There are a lot of other ways to get good information that may be much more valuable and helpful. I’ll start off with database mining. With the advent of some terrific software – now when I get to technology, let’s don’t ever forget software. Some of the software that has been developed has made it possible to dive into huge databases and access data like you’ve never been able to do before. Whole companies have been built on showing people how to take in-house databases and convert them into information sources. If you have the right kind of input, the right kind of data and the ability to access it properly with the right kind of software, you’ve got a data source that can potentially overwhelm anything you could learn through conventional marketing research. That’s where I employ the word “intelligence.” The important thing is to get the right kind of information that can be most helpful. Don’t get too hung up on where it comes from. It doesn’t have to come from a research company in order to be extraordinarily useful and relevant – and maybe much less expensive. The research industry has been provincial; it really could have been a major player in the intelligence industry today if it embraced other ways of doing things instead of focusing just on research techniques. Why shouldn’t researchers be consultants to a company on the development of data and the quality of data?

SA: So research companies could be helping companies leverage all the data they have at their disposal to understand their market.

JH: Well the research people can bring their background and their point-of-view to a database mining project and saying, “Hey, look at what’s inherent in that database or look what we can make inherent in there, and here is how we can capture it.” Now, their ability to do that has been somewhat limited by software, but the software people have closed that gap, with the software that exists today. Guess who comes and does that? Management consultants do it. Research people sit there and watch – aren’t even involved.

SA: So if you had a mantra for the industry…

JH: Think big. Instead they have thought small – and you can see why. Researchers have procedures, products, and knowledge to deliver it.. That’s where researchers make their money, but that’s not where the future is. The future is what the client needs, not what works well for the research company today. I’m afraid too many research companies have looked at what works well for us, what works well internally, what we can do easily and simply and make a lot of money doing at present. Well, the real question is serving the client’s inherent need for marketing intelligence.

I have a little case history which will help explain what I’m talking about. At one time in the automotive industry, Detroit became extraordinarily interested in the pricing of cars as it related to leasing agreements. As more and more of the upper-end cars were released, it wasn’t the sticker price that was important, it was the lease agreement. That was the real price competition. So Detroit salespeople say “Hey, we want to know what our competitor is doing in the lease end of the business.” Well, research people look at that and say “Well, interview a bunch of buyers and try to find what kind of leases they signed.” There was one small research company out in Oregon that said, “Every car that goes off the lot has an insurance policy on it as part of the lease mechanism. We can get from that insurance policy all the terms of the lease and a lot about the buyer: age, sex, location, etc. and let’s just do that and short-circuit the whole system” and supplied the data continuously to Detroit on the terms of lease agreements but were getting it from the lease agreements themselves from the insurance companies. Only two or three insurance companies specialize in insuring cars that are leased. “If we can get them to give us their records, we got it.” Research people will still be sitting out there trying to figure out humungous samples of people to be interviewed to find out about their lease arrangements when you can get the same or better information from the lease agreements from the insurance companies. That’s the kind of thinking I’m talking about. You approach the problem not in what is the best research solution; you approach the problem as what is the best data collection system to solve this particular client’s wants and needs. I hope that makes it clear because that’s what I’m trying to say. Start thinking that way instead of saying “We can talk the client into doing 10,000 interviews a month and keep our phone system busy or something.” Stop thinking that way because it’s going to be the death of you. We’re still performing an extremely valuable function, role – because somebody has to do all that; collect the data, process it and analyze it. You still have to do all that.

SA: At the end of the day, somebody has to make sense of it.

JH: But the sales guys are going to buy it. It’s the customer, the guy that is putting up the money, the sales managers. He says, “Where are you getting this?” You say, “Right off the insurance policy.” And boy he’s going to believe it. Somebody else can say, “Well, we’re interviewing people over the phone.” He’ll say, “That isn’t the same. You can get it off the insurance policy.” He’s going to believe the insurance policy approach. You don’t have to talk sample size or any of that stuff.

SA: That’s all the questions I have for you. I want to thank you for sharing your time and your thoughts on the industry and online research.

JH: Thank you.