Beware Public Polls

Whenever the results of a public opinion poll are splashed across the pages of a newspaper, I immediately seek out large grains of salt.

The fact is, all such polls should be greeted with a healthy dose of skepticism.

Now don’t get me wrong.

It’s not that I think the numbers in public domain polls are inaccurate or that their methodology is wrong.

My problem is that while public polls are good at telling us what Canadians are saying about issues or candidates, they don’t necessarily tell us what Canadians are actually thinking.

And that’s a crucial difference.

To paraphrase the old line which likened statistics to bikinis, what public polls reveal is interesting. But what they hide is often vital.

What am I talking about?

Well, let’s imagine a dialogue between a pollster and Mr. Smith:

Pollster: Tell me Mr. Smith do TV ads have any influence on your buying behavior?

Mr. Smith: Absolutely not. TV ads do not influence me in anyway. I don’t even pay attention to them.

Pollster: Alright. Next question. What brand of toothpaste do you buy?

Mr. Smith: I always buy Crest toothpaste.

Pollster: Why?

Mr. Smith: Because everybody knows people who use Crest have 23 percent fewer cavities.

Clearly, there is a difference between what Mr. Smith says about TV ads and what he actually thinks.

And just like Mr. Smith, people will sometimes give pollsters answers that don’t necessarily reflect reality; they will instead give answers which they deem socially or politically correct.

I am sure, for example, there are many Canadians who would tell a pollster that “negative ads” don’t work, but tell the same pollster they voted against former Liberal leader Michael Ignatieff because he was “just visiting.”

That’s why the only sure way to find meaning in a poll is through a rigorous crosstab analysis in which you cross reference answers.

Otherwise, a poll can be deceiving.

Case in point, are all those public opinion polls released during the last election, which consistently and wrongly predicted another Conservative minority.

It’s not that the data in those polls was faulty, it’s more likely they just publicized the wrong answers.

Imagine the pollster from our fictional example with Mr. Smith, releasing a poll under the headline: “Survey shows TV ads don’t work.”

And public polls released by advocacy groups with an agenda to promote, should be especially treated with caution.

For instance, the Manning Centre for Building Democracy recently released a poll which supposedly indicates ideological differences across partisan lines “have virtually disappeared” in Canada.

As Allan Gregg, one of the pollsters behind the survey told the media, “What I think you are seeing here is something almost the equivalent of end-of-ideology epoch. There is no major differences left among the electorate in terms of directionally where we should be going.”

The Manning poll also purportedly reveals that a “unique strain of conservatism, combining free market principles, moderation, incrementalism and social justice” has become the new Canadian “mainstream.”

But does this poll reflect reality?

I doubt it because the Manning poll simply tells us how respondents answered a series of value questions.

It found wide support for notions such as government “learning from the past" and "governments should be concentrating on the problems of today rather than tomorrow” and government should move with “caution rather than boldness.”

Plus it found support for other traditional conservative traits such as “self-reliance.”

Maybe this is evidence of a new mainstream orthodoxy, or maybe it’s just Canadians answering questions which seemed designed to elicit a certain answer.

Is anybody really against self-reliance? Who would oppose governments learning from the past?

If you asked different value questions, you might find completely different ideological answers.

Again, the Manning poll is telling us what Canadians are saying, but not necessarily what they are thinking.

And it’s what people actually think that really matters.