By Marc Dresner, IIR USA
‘People can come up with statistics to prove anything. Fourteen percent of people know that.’
~ Homer Simpson’s response to a TV news interview question in which the local anchor cited an inconvenient statistic
With Halloween so close, I thought we might all be up for a good, scary story.
What you are about to read is confidential and may get me in trouble, so please don’t share it with anyone’not a soul! (Beware: even weaker Halloween puns ahead.)
A closed-door meeting of my condo association’s board of directors took an amusing and slightly alarming detour the other night into the realm of statistics.
We five had gathered to finalize the 2011 budget and ‘ more importantly ‘ to review bids from several cable and satellite TV providers.
(I stress that the latter is more important because while I do not enjoy budgeting, I do enjoy what little television I have time to watch and I am dissatisfied with our current bulk services provider.)
The budget discussion was relatively uneventful. But the cable conversation was a bit more’spirited. (I warned you.) Apparently I am not the only person with strong emotional ties to the tube.
First, we needed to be able to justify the switch, because any option other than staying with our current provider would nearly triple everyone’s monthly cable bill. With more than 260 units, that’s a lot of rotten produce being thrown our way at the next homeowner’s assoc meeting if our call turned out to be dead wrong.
Now, we’re confident that most of our residents are unhappy enough with our current provider to accept the increase; however, beyond anecdotal evidence and a two-year-old survey with a lukewarm response rate, we really didn’t have much else upon which to stake our decision.
So you can imagine my reaction when one of my board colleagues ‘ who is much smarter and more accomplished than I, and by whom I will likely be sued if he reads this ‘ suggested that the vast majority of residents are probably open to accepting a new cable/satellite services provider at triple the price in a recession BECAUSE ‘ drum roll ‘ conventional wisdom in political polling dictates that for every person who bothers to make their opinion known, there are one or two others who feel the same way, but do not say so.
So, one vote equals two, maybe three? Based on our two-year-old survey results, that would constitute a sizable majority of residents (most of whom did not bother to respond to the survey).
Interesting stat, no? And, by the way, let’s hear it for self-selection error!
It gets better’
In response, another of my board colleagues ‘ again someone much smarter than I, with outstanding credentials ‘ leaned in and said, ”Joe,’ this is not a political poll.’
Re-read that last line, just for laughs.
So, board member number two’s issue was not with the accuracy of board member number one’s math, but with the nature of the topic being polled. Apparently, the 1=2-or-maybe-3 rule indeed applies to political polls, but not to non-probability sampling in matters regarding cable television services.
Now, I am by no means a statistician and I did not dispute this line of reasoning (the meeting was already running long, and it was a harmless point well taken), but I believe both of my colleagues were slightly mistaken.
Isn’t the correct rule of thumb that for every one person who voices a complaint, there are 27 others who feel the same way but don’t complain? Or should we defer to the ole adage: ‘A happy customer tells a friend; an unhappy customer tells everyone’? It’s all so confusing’
I relate this tale all in good fun, and I mean no disrespect to my wonderful condo board colleagues, none of whom ‘ I trust ‘ will call for my resignation if they happen to read this.
At the end of the day, my point is that the blas?? tossing about of questionable stats is frighteningly commonplace ‘ and educated, intelligent people routinely and willingly accept them without question.
Which reminds me: we have an election coming up!
I’ll keep my politics to myself, but I’ve already seen enough push polls and suspicious numbers to sustain me through the next four election cycles. And while it’s easy to point the finger at politicians (I have a special finger for that), the worst offenders are the journalists who lack the background and/or inclination to verify the validity and reliability of the numbers they so eagerly push out into the echo chamber as fact.
This sentiment was echoed in IIR’s recent podcast interview with NBC’s SVP strategic insights and innovation, Horst Stipp, who had some pretty strong things to say about the irresponsible use of research, and who noted that journalists and politicians are not the only offenders: Researchers, too, occasionally lapse.
Bottom line: I think we can all agree that the misuse and manipulation of statistics ‘ in journalism, politics, commercially and even by condo boards’for whatever end ‘ is totally out of control, with grave implications for research credibility.
So what’s your scary stats story? We each have one, so I’m interested in hearing all three of yours.