Gongos Research speakers Katherine Ephlin and Greg Heist of Gongos Research, who presented “Is the Future of Commuities in the Palm of Your Hands?” took some time after their presentation on Monday to share what their presentation was about at TMRE 2010.
by Bill Weylock, Brand3Sixty
David Santee, former Research Director at H&R Block has some really interesting slants on a familiar precept: know your audience. And by that he means, really really know your audience.
As a preamble, David pointed to a possibly sad but certainly unavoidable fact: market research is a hard job today. Like so many other responsibility areas, research is no longer parsed into traditional discrete functions. Today’s market researchers need to be multi-function experts in gathering, analyzing, communicating, synthesizing, and recommending solutions suggested by data.
How to make any of this happen?
Part of the answer is learning how to get the attention and investment of the audience for research: the management decsion maker.
We’ve heard that… “Decisions are made emotionally and justified rationally.”
We’re used to that in designing consumer and B2B research. We know that purchase decisions are made emotionally no matter what the buyers want us to believe … David’s breakthrough thought was something like “Hey! This doesn’t apply just to consumers of our products, but consumers of our research.” Manager decisions are made emotionally as well. Managers are people too. Did he really say that?
You can get the full presentation, filled with interesting touchstones for ensuring quality research product, from the web. The most striking insights for me are that in order to get attention, research must tell a story, must have a point of view, must be presented forcefully, and must be tailored to the end user audience that needs to be influenced. Research can be tailored to the audience even in the design phase.
Make sure your research speaks to the decisions that need to be made in language and illustrations that drive the points home compellingly. Like other presenters today, David stressed the large dividends in bringing researchers closer to the meetings where priorities are set and questions are framed that will drive decisions. Getting there may be its own story.
The key point for me is that getting research understood and appreciated is our job. It is not enough to be right. You have to get read.
by Bill Weylock, Brand3Sixty -
First session of my afternoon: relative projectibility and sampling reliablity.
REI determined that the rising percentage of cell-phone-only users makes land lines an unreliable sole-source approach method. Then they run into the problems with cell phone: more expensive to source and reach, require an incentive, not location based, ratio of cell phones to landlines not known, not able to cross off a household after reaching a number (as you can with a landline).
So on to online sampling: guess what, Anne-Marie cites the GRIT 2010 study indicating a large percentage of research professionals think online sample is less reliable than the market generally appreciates.
By the way, we are about to launch the second 2010 GRIT which will focus on new technology adoption in addition to continued tracking of basic industry metrics (with comparisons going back to 2004). The findings will be a key feature of the upcoming IIR conference on technical and technological innovation to be held in April.
Anne-Marie cited a fascinating study by Washington Mutual (just before they vanished into JP Morgan Chase) finding that respondents who take 5 or more surveys annually are less likely to desire financial products than others. Survey experience alone could account for up to 7% variation in their results. This suggests applying scoring algorithms to sample – including, among other things, extent of participation in online surveys. Panel providers take note.
This year REI did a test of methodologies, collecting data in 7 markets online and 2 markets by cell-phone while they rolled out their normal annual landline fielding. Note that cell phone interviews were only n=35 in each market, yielding directional findings only.
They found striking differences between landline and online participants in brand awareness, in ad recall and percentage of respondents who shop REI. In all cases the figures were significantly higher within the online sample.
Demographic comparisons revealed higher penetration of Causcasians and college graduates in the online sample, leading to their conclusion that online panel sample seems provide a different population than a landline sample.
One of their theories is that maybe a panel rewarded by retail incentives is more aware of and engaged with retail. In any event, they made a determination that online sample is not projectable for REI research.
Again, results are directional, but comparing landline and cell samples showed little difference in awareness, ad recall and penetration of REI shoppers. Demographic analysis suggested that cell phone sampling could also provide reach into populations hard not adequately delivered by landlines.
Next year REI is going to test ABS with landline and cell phone data collection. They can match sample to cell or landline numbers at about 65% – 75% and expect that this will become their sampling method of choice going forward.
It seems that REI has rejected online sample and is harking back to a previous era in bringing ABS to the fore for 2011. It is an evolving experiment, but they expect that ABS with sampling shared by cell and landline phone outreach should provide more representative sample than previously obtainable.
Okay, online sample providers. I did ask.
First I asked Anne-Marie whether she was saying that online sample is unreliable and unprojectable and should be rejected by all market researchers in favor of phone. It didn’t come out exactly that way, but that was the gist. I wondered (and frankly was hoping) to hear that there was something peculiar to REI that made the demographics of online sample unrepresentative of their universe.
Unwilling to go as far as a universal slap at online sampling, Anne-Marie simply restated her opening thesis: that marketers should ensure that their sample is sufficiently representative of their markets. In REI’s case, online sample does not seem to hit the mark.
I also asked, why she had been unable to weight the online sample to census. The rub seems to be that the panels were unable to provide sufficient penetration in key markets to support weighting.
This topic seems to cry out for more coverage and certainly more discussion. To my ear it is as provocative as anything we have heard in the past two days. What do you think?
We got a chance to sit down with Jeff Martin, VP of sales for perks.com, and he was able to tell us a little bit about what perks.com does. Here’s a recap of that interview:
Q. Can you tell us a little bit about perks.com?
We give research companies the points based reward solution they need to increase panelist retention. We offer low start-up fees because we expect our clients to succeed through our programs, making us more of a partner than a vendor. But more than that, we offer added-value administration – our on-staff program managers ease much of the pains of administration, integration and implementation. We don’t make your company retrofit your technology and processes to us, we do whatever we can to adapt to your needs.
We offer rewards solutions in more than 70 countries and across more than 14 languages. Our program managers, our developers, while located stateside, have experience dealing with the intricacies of launching programs in any country, in any language. We provide cost-effective solutions for incentives within the panelists’ country and seek out the appropriate vendors, if needs be. From translation to implementation, Perks has a proven track record of multi-national success.
Q. Tell me a little bit about the rewards selection.
A. Through our partnerships with over 300 vendors globally, we provide massive breadth of choice which means real value to panelists when it comes time to redeem their points. Our unique system provides a rewards catalog with products valued from a dollar to thousands of dollars – that will to keep your respondents active, engaged and loyal.
Here’s a quick pic we took of Jeff Davis below. Remember to stop by the Perks.com booth if you are in attendance at TMRE!
by Bill Weylock, Brand3Sixty
Reed Cundiff of Microsoft on the challenges of the MR professional at Microsoft today…. Let’s just say that his graphic was a vice, and he invited us to figure which body part would be most appropriately inserted.
Microsoft of course is famously challenged and has come out with two major products this past week – Windows Mobile 7 (yay) and Kinect (which I gotta try).
Challenges to the MR department stem from the democratization process that Microsoft has played a big role in: for instance enabling really bad PowerPoint presentations.
Now Microsoft MR needs to run around cleaning up such messes as badly done Survey Monkey projects or a study that has contravened standard consent protocols with respondents.
One of their big missions is to help marketers across the vast organization understand the importance of doing good research rather than just research. He stresses that 5% “off” in research results can have vast implications for hundreds of millions in research supporting billions of products and revenue.
MR tries to evangelize quality and at least to set up guard rails for the inevitable ad hoc research marketers will be doing.
Business Intelligence has proliferated from telemetry, traffic data from Bing, from “phone home” calls from distributed applications. The MR department is awash in data that is not dependendent on primary custom data collection. They are also awash in options. It all calls for some intensive attitude adjustment.
They could drown in information, on the volume of research conducted outside their purview. They have a choice of “victim vs. player.” They try to take advantage or such things as last minute demands for information to build credibility with senior managers and get closer to the decision. Their mission is to drive business impact across the company.
They have to choose roles between Knower and Learner: they do not need experts stuck in amber. They need adapters who can embrace new data streams and the new and growing needs for synthesis and consulting.
They have to choose between maintaining the Branch and sustaining the Tree: focus on the health and success of the tree rather than the size of your branch.
The administrative key to helping ambitious employees forsake self-promotion? They try to make the review process more transparent and let them know that playing well with others is a major factor in advancing at Microsoft. Once they see that good team players get promoted, they relax and trust the ethos.
This is a recurring theme at this conference and wherever two researchers gather: being subject experts is not enough and can even be a problem if it comes with inflexibility. Now success is measured by impact on the business and researchers must be communicators, evangelists for quality practices, and completely invested in the priorities of the business units they are assigned to support.
Further to all of this, the emphasis is no longer on data collection and discrete studies that have their own arc and validity.
Like other industries, but particularly true of software and technology, their mission is to get closer to the customer and let that understanding influence product development and services. Now focus areas need to be keyed to the strategic intersts of the business groups they support. Now it is “how can we mine existing knowledge and build on it.” How can we exploit partial answers from existing studies and initiatives?
Getting talent is a major issue. Hiring is not always the answer. Increasingly they build researchers into consultants by helping them with communications, issue framing, and story telling. They also help them in understanding the conceptual modeling capabilities of the team members. To what extent can they jump from the spreadsheet to real-world implications? How comfortable and effective are they at managing unstructured data.
A tough question came from the floor: how does MS MR achieve closer proximity to the planning and decision process? Is there not resistance from the other side, wanting research to be kept in its place? “Give us the findings and let us maket the decisions?”
I thought the answer was as good as the question. Reed pointed out that it depends on the levels you approach. Management comfort level with research reaching into real planning teams is higher at the senior levels where turf is not threatened. Upper management wants their research resource investment maximized and understands that access is key.
So MR gains access from above and fights “border skirmishes” at the lower levels after ultimate victory is already theirs.
Upper management does put up some “guard rails,” insisting the MR be present not as strategy consultants but as research consultants (not the easiest tightrope I can imagine, but negotiable). And management wants research clearly tied to imminent business decisions.
Multi-disciplinary, multi-tasking, socially and politically poised, intellectually acute, and selflessly collaborative. This does describe you, right?
by Bill Weylock, Brand3Sixty…
After a great presentation by Chris Anderson the Explore panel got off to a bit of a late start.
During setup Matt Dusig and Beth Rounds did a pretty fair imitation of an NPR comedy special intro – vamping for time as well as I’ve seen.
Matt led with a video clip from Simon Sinek, featuring his Golden Circle of innovative thought. Thesis? Innovators like Apple, MLK, Wright brothers think in a very different pattern from most of us.
We tend to start with what we are doing or want to do, go to how we would do it, and sometimes (pretty rarely when you think about it) trouble to arrive at why we do it?
Innovators start with the why and end with what.
If Apple were like the rest of us? They would say something like “We make great computers. They’re beautifully designed, simple to use, and user friendly. Want to buy one?” That’s starting with the USP and proceeding to the pitch.
What they really do is start with their “why”: We believe in challenging the status quo and thinking differently. We do that by making all our products beautifully designed and friendly to the user. We just happen to make great computers. Want to buy one?
The finalists, as you know, are eBay, American Water, and ANZ Bank (New Zealand, which earns the “here from far away” award as soon as we get it back from the bronzer.
Later presentations this afternoon will present the nominated case studies in detail. Here we’ll just talk about the themes that emerged in the panel discussion.
What caught my ear first and wouldn’t let go for a while: American Water’s TIP (Target Identification Project) apparently delivered them new ways of opening and maintaining channels of communication between their management and the civic and business leaders who are their primary stakeholders – taking them beyond the relationship sell. Since almost everything we read stresses the importance of establishing and keeping up relationships with customers, I can’t wait to hear the details on that one.
A larger point is that innovation can be revolutionary and broad brush, but it can also be incremental – an accretion of small and continual improvements in product and process.
Part of eBay’s story (much more to follow) involved using storyboards to research product innovations by showing the consumer how the product would affect them and their experience.
Selling innovation within upper managment? It has to be done. Innovation can’t survive without buy-in and continuous support from upper management. But upper management doesn’t care about innovation in research or innovation qua innovation. They care about outcomes and advantage.
Moral for research? We care about the sizzle and internal researchers do as well. But we have to sell management the steak. What’s in it for them and what difference will it make?
And passion is important. Within conservation organizations like American Water, there is a reflexively cool reception to innovation. Selling a new approach must be justified by effects on the bottom line but must also be sold and resold by passionately committed managers. If you want them to believe, you have to believe double.
For at least two of the research providers on the panel, innovation is part of the DNA. Invoke was founded on an innovative platform and continues to involve clients in developing and refining their product and service offerings. Touchpoint, the ANZ partner, is a technology based company which eats innovation for breakfast and is founded on curiosity. As a technology resource, they interface with a wide variety of categories and are able to cross-pollinate by adapting innovative practices from one vertical for another.
Ian Lewis, from the floor, asked the panel to identify the most disruptive and innovative techniques and trends that would influence market research in coming years (hope I got that right, Ian).
I think we would have had a strong discussion except for the time constraints that squeezed the debate. Whatever the enabling technology or process, the panel agreed that the most important concerns are broadened and active and full duplex communications between the organization and the stakeholder, between senior managers and internal researchers, and between research buying clients and research providing suppliers.
The winner will be announced this afternoon. Stay tuned.
by Bill Weylock – Brand3Sixty
IFC Marketing wanted to understand who their viewers really are. Conventional wisdom was strong and insisted they were mostly media savvy young males. They were so convinced of this that Marketing could not get the resources to do the research they knew they needed – until recently…
Partnering with Sachs Insights they conducted quant first and arrived at segments, which they then illuminated through in-person, in-home, in-haunt videography.
They found Authentic Influencers who tended to be wannabe musicians, artists, and think of themselves as the artists they figure they really are inside even if they work for banks….
They influence Responsible Rebels who insist on edgy writing, irreverent humor, uncensored programming that includes sex, drugs, and rockandroll (where artistially justified?). They want thinking people’s programming that makes them feel in the know and apart from the hoi polloi. They talk about the IFC brand the way they talk about themselves.
They learned that viewers accept them as authentic and trust their recommendations. They learned that their target valued quality over obscurity: Mad Men, even though it’s on a network. But they did not identify IFC as providing high quality product. They thought IFC was smart but only about independent film that requires as certain mood and commitment before viewing.
IFC wanted to shift thinking from “what film is on IFC?” to “what is on IFC?”
They threw all of their rich ethnography and segmentation at creative shops asking for a synthesis around a new identity.
One shop came up with Always On, Slightly Off.
Sachs went out into the field to research the line and came back with associations for independent, fun, and (most important) “like me.”
The research changed everything – from advertising messaging and media to content acquisition and development. An ongoing Vision Critical panel maintains communication between management and viewers.
IFC feels they have taken their perception of viewers far beyong the traditional numbers channels have normally used as metrics.
The presentation from IFC Research, IFC Marketing, and Sachs Insights showed us a very elegant and effective marriage of segmentation and ethnography.
We broke for lunch instead of questions but I did get a chance to ask what opened the spigots and got senior management to fund an elaborate and sustained research program where before they had stonewalled. With no chance for full details (hope to report more later), they all three agreed that it took a relentless onslaught, including bringing in Sachs to present and “being a pest.”
Great presentation with video I hope we can link to.
By Marc Dresner, IIR USA
For those of you who arrived late yesterday or this morning welcome to beautiful San Diego, where the sun is shining on market research’
We’ve had an outstanding program so far today, with more to come, but I must admit I’m a bit skeptical that it will top yesterday’s symposia tracks. A happy problem to be sure!
I’ll first doff my hat to our terrific team of internal and guest bloggers, who have done a masterful job of capturing some key, granular details from a whirlwind of presentations and wrapping them in truly expert analysis.
Rather than reinvent the wheel, I’ll take this opportunity to recap yesterday from a more general perspective.
First of all, as I weaved in and out of sessions yesterday, some pretty strong themes emerged: 1) change is an accelerating constant, 2) instant isn’t fast enough and 3) complexity and simplicity are the new ying-yang research imperatives.
If you weren’t here or didn’t notice, the social media & community research track was far and away the big draw of the day. I would say most of these sessions were standing room only, but that’s not entirely accurate: people were even sitting on the floor.
As noted elsewhere, Dawn Lacallade from ComBlu opened the track with a primer on how to correctly deploy social media both for marketing/branding and market research purposes.
Two key takeaways for me: 1) the biggest mistake companies make when they attempt to create social media communities for research purposes, in particular, is that they forget to think about the audience. It’s great that your internal clients have a laundry list of questions they want you to answer, but you had better give your audience a reason to care and want to be there.
2) Not unrelated, consumers reach ‘community saturation’ after belonging to about three communities: 1. Recreational (e.g., Facebook), 2. Professional (e.g., LinkedIn or in our case one of the many online communities for researchers out there) and 3. A ‘life-life’ or personal interest group (e.g., something hobby related). So if you’re contemplating starting a community, you had better pay close attention to how you’ll fit into one of those three categories, because after that the consumer is maxed out on community memberships.
Lallacade was followed by Taco Bell’s senior consumer insights manager, Linda Ashbrook. For me, this was definitely the most eye-opening and controversial presentation of the day.
Ashbrook discussed how Taco Bell has developed a robust qual program using Facebook for small, temporary ‘mini-panels’ and labs designed around specific project needs.
They’ve conducted about 20 of these and have two dedicated internal moderators who specialize in these types of groups on their research team. The groups have become so popular among internal clients that TB has in some cases reluctantly had to bring in external Facebook-trained moderators to handle overflow.
According to Ashbrook, these Facebook panels yield incredibly deep, rich information; they’re more flexible and may be used for a much broader portfolio of research activities than traditional focus groups; and the medium not only nets their target respondents, but recruitment costs about half of what a terrestrial field recruiter costs (they’re trying to winnow that cost down to about 20% of traditional).
Moreover, compared to ‘traditional’ online communities ‘ that people are referring to online communities as ‘traditional’ these days speaks volumes ‘ TB’s Facebook mini-panels are significantly less expensive and less labor intensive. They’re short-term, so no need to refresh your panelists, and no need to struggle for ways to keep people engaged once you have what you need.
I know some of you reading this are probably chomping at the bit with counter arguments, but keep in mind: this is a big client, the initiative is being driven by the research team ‘ not by marketers as one might expect ‘ it’s providing the insight they want from the right demos, the process can be managed entirely internally, and it’s cheaper than some of the ‘traditional’ alternatives.
This is no time to bury your head in the sand and wish it away.
Oh, and one last thing: Taco Bell is eager to find a way to introduce a quantitative dimension to these Facebook mini-panels, which, if accomplished, could have implications for their use of ‘traditional’ online panels.
At this point, it was clear that the social media track was going to be well covered, so I shuffled over to the decidedly less sexy segmentation track. You know what? The folks in the social media track missed out on learning about some serious research innovation.
Christina Liao and Mike Mabey of CMI provided a fascinating window into how they’re segmenting consumers by decision pathways. So here we’re taking an approach that ‘traditionally’ focuses on demographics, psychographics and/or behavior, and flipping it to trace the decision-making process in order to get at that critical ‘why’ behind the choices we make. Heady stuff.
I had the opportunity to interview Christina and Mike later (the video will be available on this blog, so stay tuned) and here’s the kicker: I asked them what’s next as they refine this technique and they said they’re planning to weave in social media. I suppose we can’t escape this topic!
Next in the segmentation track, we were treated to the tale of how Ameriprise with its research partner Chadwick Martin Bailey undertook the daunting task of shifting the former’s segmentation focus from exclusively financial advisors to looking at consumers, too. Sounds simple enough, but this was a serious challenge.
The surprise here was that the presentation turned out to be less about the technical aspects of the segmentation ‘ although ample insights were provided ‘ and more about the process by which the research team was able to effect a major change in the way the organization thought about its business. Outstanding blueprint for how research can drive strategic thinking.
I capped the morning with a one-two punch from none other than PepsiCo’s Stan Turek and Target’s Mark Johnson on how the research and insights functions in both organizations have transformed and where they’re headed.
Both speakers made it abundantly clear that the background, skill set and responsibilities of tomorrow’s researcher will look radically different from those of today’s researcher. In fact, at both organizations, the journey is well underway and research is evolving from a consumer focus to a meld of shopper/consumer ‘ from user to chooser ‘ that is infinitely more complex, with enough moving parts to make your head spin.
In the afternoon, I relied on the mobile track to revive me from my midday food coma, and the first presentation by Katherine Ephlin and Greg Heist of Gongos definitely delivered. Via a series of case studies with clients like VISA, GM and Blue Bunny Ice Cream, the speakers shared smart phone research-on-research they’re conducting. Btw, smart phones will probably be ubiquitous sooner than most people realize.
Among other things, Gongos is finding that smart phone research respondents are actually more engaged than their counterparts in ‘traditional’ research communities and that, surprisingly, their text responses to open-ended questions are so rich that they’re almost indistinguishable from those of non-mobile respondents.
What really excited me was to see that some respondents didn’t even bother answering questions via keypad; they just turned the device on themselves, answered the question and uploaded the video response. It’s easier for the respondent, and it doesn’t take a lot of imagination to envision what a researcher could do with AV responses to a text question.
Next, I dropped in on the media measurement track to hear Unilever’s Tom Emmers, who talked about the challenges of conducting research in the digital age amidst continuous, rapid channel proliferation and also provided a good deal of information about how Unilever is coping.
A key research strategy has been to lead off with insights into the consumer’s mindset according to time/place (e.g., receptivity to advertising depending upon the context) and feed that knowledge into channel insights (e.g., creative testing, cross-channel media impact, etc.).
Emmers explained Unilever’s approach to marketing in this evolving mediascape also entails several layers, starting with immersing the organization in the target market, followed by campaign development and finally undertaking ‘dynamic optimization’ activities during the first few weeks of campaign launch. Unilever and its partners have become so proficient at this last step that banners, video and other web properties and online campaign collateral can now be optimized within a week.
While the research function has adapted remarkably to this constantly changing environment, their success is all the more astonishing when one considers that budget and headcount for digital research have not increased despite the fact that Unilever now executes a new digital campaign roughly every two days!
Emmers admitted he wonders whether or not this pace can be sustained. What will it take to measure an almost infinite number of touchpoints to achieve a holistic understanding (i.e., reality)? ‘How much more complex can it get before we just throw up our hands and say we simply can’t manage this’? Emmers asked. ‘Nobody has the answer, because there isn’t one’I will say this much, research has innovated more in the last two years than in the five to ten years prior, and I’ve had more fun in the last two years than ever before.’
The final symposium of the day proved to be one of the most practical and compelling sessions for me. I’ve heard Dr. Randall Brandt of Maritz speak several times over the years, and he never disappoints. The topic on this occasion was how to integrate and analyze multiple, disparate VOC data sources.
Brandt laid out a marvelous blueprint for creating a consistent, uniform set of customer experience categories that can ultimately facilitate convergence mapping. So for example, this approach could tell us whether or not inbound feedback ‘ say, complaints ‘ from a website jibes with issues identified via survey data.
Once these data are standardized, the sources and what they say may be plotted on a graph in four quadrants, the top right in this case indicating points where the data agree/converge; the bottom left quadrant indicating contradictions/divergence. A case can then be made that those items that converge constitute key drivers of customer satisfaction/loyalty and should be prioritized by the organization.
But what about those items that diverge? Well, Brandt said these cases should command the same attention because they also can be a valuable insight source. As an example, a complaint that surfaced in call centers but was not echoed in survey responses could well indicate an opportunity to adjust and prioritize how certain complaints are handled by the call center.
A fringe benefit to this approach, Brandt quipped, is that when a researcher is confronted with a skeptical client, they can’t exactly dismiss the results because they don’t ‘believe’ in surveys, since the data come from multiple sources.
Day One of TMRE concluded with a keynote to kick off the general conference program by Michael Tchong, trend analyst and founder of Ubercool, Inc.
In a spectacular presentation largely about technology, multitasking and our culture’s increasingly short attention span ‘ designed specifically for people with short attention spans who multitask ‘ Tchong flew through a litany of statistics and trends at breakneck speed with wit, acuity and exuberance.
In between fidgeting with my blackberry, I learned that we have more in common with computers than I realized: we both perform multiple functions simultaneously, we’re both prone to crashing and we both could use more memory’