Public Polling Misses

While I was running Nikki Haley’s Presidential Super PAC and now again throughout this cycle I have been consistently shocked at how seriously people take public polling. After the latest Q POLL release that once again sets no real quotas and still uses f****** random digit dialing, I decided to look at how these average work out.

45 races in the Trump era (2016-) that were polled somewhat extensively and somewhat competitive

The average polling miss was was 4.3 points from the FINAL RCP and 5.36 from the early October average. That's AVERAGE! 

The bias is also one directional with the average early October polling underestimating Republicans by 3.9 points. If we look at the current RCP averages that would mean just an ‘average’ Republican polling era (not even a high one) would get you a pickup of

OHIO, MONTANA, PENNSYLVANIA, MICHIGAN, WISCONSIN, and WEST VIRGINIA with the GOP losing no seats. So just an average polling error in GOP favor nets you six seats and a 55 seat GOP majority.

In fact the averages missed by at least 3 points 63% of the time, 5 points 56% of the time, and 7 points or more 24% of the time! 

In all these races the public polls over estimated Democratic support 69% of the time. 

What’s also interesting is to look at it by cycle. The 2022 cycle was the most accurate especially if you look at the Early October average before a bunch of junky (largely pro-Republican) polls flooded the zone. 2020 stands out as particularly bad with missing by 6.5 points on average.

Does the bias mostly go one way? Yes, except for late October 2022.

Now look the margin of error on a survey varies but anywhere from 3-6 depending on sample size in most political surveys so its not shocking that there is error, but you would expect that the averages would cluster around the truth.

So today if you’re writing off Larry Hogan at D+7 you might consider that 24% of the races in the last 4 cycles has an error of 7 or higher.