How accurate are ACT’s poll ratings? Part 2

Note: this post originally appeared on ‘Douglas to Dancing’, a blog I maintained from 2007-9 on the ACT New Zealand political party. The blog was an extension of the thesis I wrote about the Act Party in 2007, From Douglas to Dancing: explaining the lack of success of ACT New Zealand and evaluating its future prospects (PDF).

ACT is polling at 2% or less in opinion polls – 1% according to Saturday’s Fairfax poll. Three weeks out from election day, this must be discouraging for ACT supporters. Is this picture an accurate reflection of what the party will gain on election day? Or is it underestimating support for ACT? In part 2 of my report, I examine whether a reverse “Bradley effect” could be underestimating support for ACT in opinion polls.

Because I’m interested in historical comparisons, the best resource would be a database averaging the various polls taken in election years. Lacking this, I decided to look at two polls with data easily available. An interactive graphic is available for the Herald-Digipoll, which shows polling results for ACT going back to before the 1999 election, while Roy Morgan has published polls available from just before the 2005 election. A disclaimer: I’m not a statistician and this is an informal experiment. Here is what I found:

• In 1999, the final Herald-Digipoll put ACT at around 8.5% in the final poll before election day. This was a downwards trend: the party had over 10% support in the previous two polls. On election day, ACT received 7% of the party vote.
• In 2002, the final Digipoll put ACT at 8% support. On election day, this turned out to be 7.1% of the party vote. Previous polls had put ACT lower, however, at around 6%.
• In 2005, the final Digipoll put ACT at around 1.4%. Previous polls had put ACT higher, however, at around 2%. The final Roy Morgan poll before the 2005 election placed ACT on 3% support; the penultimate poll recorded 2.5%. On election day 2005, ACT received a 1.5% share of the party vote.

Results? Remember, I don’t claim this to be a scientific exercise. Yet on looking at these numbers, it seems that if anything, opinion polls close to election days seem to overestimate support for ACT, not underestimate it! The only time a final poll before election day put ACT below what it actually received was in 2005 – and the Digipoll then only put ACT a tiny fraction lower. This could be ominous for ACT indeed: with the party only reaching 1-2% in most polls, the election day result could be lacklustre to say the least. If history repeats itself, the latest 2008 Roy Morgan poll, which put at ACT at 3.5%, could be as much a “false hope” as the 2.5-3% ratings the company had for ACT last time around.

Why could this be? I can think of two plausible reasons off the top of my head:

• The stigma effect no longer applies to ACT as strongly as it once did – if it ever did. Memories of the 1980s have faded. Moreover, ACT has shed any racist overtones it had by playing down its “One Law for All” message. Is anyone really afraid of saying he or she supports ACT? Is it any “worse” than supporting National?
• ACT supporters are perhaps actually overrepresented in opinion polls, due to the fact that they are likely to be older and thus more likely to be at home when rung by polling companies. Younger voters are more likely to use mobile phones only and therefore not be picked up in polls. Yet despite a small but vocal ACT on Campus group, they are the voters less likely to be ACT supporters. (By the same token, we could expect younger Green voters to be underestimated in opinion polls).

My conclusion is this: if the polls are “wrong”, this is not likely to be in ACT’s favour. If polls put ACT on 1-2%, it’s most likely because ACT is at 1-2%. Or worse.