In Defence of (internal) Polls

Canada’s political pollsters got it spectacularly wrong in British Columbia and in Alberta.

And so now many pundits and media types are questioning the credibility of the polling industry.

But before anybody relegates the science of polling to the same category as astrology, it should be pointed out that there’s a huge difference between the free public domain polls the media likes to cite and internal, private polling.

The fact is, the public polls we read about in newspapers usually only tell a superficial and partial story; they reflect what people are saying, but not necessarily what they are thinking.

I know that sounds odd, so to illustrate my point consider this imaginary dialogue between a pollster and a Mr. Smith:

Pollster: Do TV ads have any influence on your buying behavior?

Mr. Smith: No. TV ads do not influence me in anyway.

Pollster: What brand of toothpaste do you buy?

Mr. Smith: I always buy Colgate toothpaste.

Pollster: Why?

Mr. Smith: Because everybody knows it’s the number one toothpaste recommended by dentists.

So Mr. Smith says ads don’t influence him, but clearly they do.

Yes, this is a made up example, but it demonstrates the problem pollsters face: people often hold contradictory or confusing attitudes, especially when it comes to politics.

This is because the vast majority of voters don’t follow the political scene all that closely, hence their political views are often tentative and subject to change.

For instance, back in 1988 when I was working for the National Citizens Coalition, we commissioned a poll which showed that a significant number of Canadians supported then NDP leader Ed Broadbent, enough support that he could actually get elected Prime Minister.

Voters, the poll told us, liked Broadbent because they saw him as more “honest” than the other leaders.

To us -- the NCC is a pro-free market group -- this was bad news.

Of course, this is the kind of information you get in a public poll.

However, our internal poll also revealed Broadbent’s potential Achilles heel: many of the respondents who said they supported Broadbent, also opposed the NDP’s socialist policies.

In other words there was a disconnect; voters liked Broadbent, but they didn’t like his platform; they didn’t even know his platform.

Thanks to our poll, we were able to craft a strategy to undermine Broadbent’s support.

We simply pointed out to Canadians that while Broadbent might be a nice guy, he’s also promoting a dangerous and “scary” left-wing agenda.

By the way, that’s exactly the same strategy the BC Liberals used to successfully degrade the BC NDP, which had been riding high in the polls.

And I’m sure, like us, the BC Liberals adopted this strategy based on internal polling data.

My point is, understanding and analyzing a political poll is a complicated business. It’s more than just asking Canadians who they think will make the best Prime Minister.

To adequately study a single poll means investigating how respondents answered 30 questions or more, which means going over hundreds of pages of cross tabs.

And this is where pollsters earn their money; they wade through a morass of data to find that issue or attitude their clients can successfully exploit.

In short, despite its bad rap, the statistical science which underpins opinion polling works, which is why political parties will continue to rely on their own internal polls.

Public opinion polls, on the other hand, should be taken with a grain of salt.

That’s the true lesson of British Columbia and Alberta.