Trump's Allegations of "Skewed" Poll Numbers Simply Don't Add Up

Oct 25, 2016

Complaints about skewed public polls are nothing new. Recent election cycles have included many such accusations from candidates – especially when they’re running behind. In 2012, many Republicans held onto the idea that the polls were skewed against Mitt Romney, right up until Barack Obama won reelection.

This week, the issue of skewed poll has been supercharged by a conspiracy theory article featured on the Drudge report, and an accompanying tweet from Republican presidential nominee Donald Trump.

Trump later said at a Florida rally “When the polls are even, when they leave them alone and do them properly, I’m leading.”

But let’s be clear: There is nothing to these suggestions. The article spins an innocent and common polling practice called “oversampling” into supposed evidence of media polls intentionally skewed to chill the Trump vote. The accusation is based on an apparent Clinton campaign document from The Atlas Project, newly revealed by WikiLeaks. The memo examines some potential voter dynamics in the 2008 election, and offers suggestions for specific geographies and groups of voters the campaign should target.

It provides an interesting inside look at some parts of the Clinton campaign, and really nothing else. It is not evidence of a conspiracy to rig media polls.

Let’s look at the several issues the article covers. 

  1. What is the main allegation here?

The title of the piece featured on Drudge is New Podesta Email Exposes Playbook For Rigging Polls Through "Oversamples". The writer begins with standard complaints about a recent public poll (from ABC) that does not conform with what he feels the party identification breakdown of poll respondents should be. But he finds a new culprit: “oversampling,” a common polling practice, especially for campaign pollsters. A few key excerpts from his article summarize the thrust of his theory:

  • "The email starts out with a request for recommendations on 'oversamples for polling" in order to "maximize what we get out of our media polling.'" (emphasis his)
  • "The email even includes a handy, 37-page guide with the following poll-rigging recommendations.  In Arizona, over sampling of Hispanics and Native Americans is highly recommended."
  • "Meanwhile, it's suggested that national polls over sample "key districts / regions" and "ethnic" groups 'as needed.' "
  • "And that's how you manufacture a 12-point lead for your chosen candidate and effectively chill the vote of your opposition."
  1. Why do campaign pollsters sometimes oversample?

Campaign polls, like those described in the Atlas memo, are only partly about finding out which candidate is ahead. Campaign polls also help campaigns strategize where and how to spend resources on media, and what to say and to whom.

In other words, the goal for the campaign is (in the words of the email to which the report was attached) to maximize what it gets from its polling.

Some of the voter subgroups campaigns want to target are small. If a target group (say, Latino voters) is only 5 percent of a given state, a poll of 1,000 votes will only yield 50 Latino respondents. Instead of basing multi-million campaign advertising decisions on 50 voters (with a margin of error +/- 14 points), the campaign may instead decide to interview an extra 200 or 300 Latinos.

Once data collection is complete, these oversamples can be viewed separately from the rest of the poll as a representation of Latinos in the state. They can also be used in the statewide sample, after they are weighted down to their proper proportion of the statewide population. This gives the campaign results that accurately reflect the entire state, as well as a more reliable look at Latino voters in the state.

  1. Does the memo have anything to do with rigging public polling?

No. The memo describes a common campaign polling technique used to analyze small subgroups, and says nothing about the media polling that is the ostensible basis of the article. The document is an internal campaign document talking about campaign strategy, not about public polling. The writer ignores the distinction between media polling and campaign polling and uses the existence of a campaign polling practice to tar media polls with accusations of bias.

  1. Do media firms ever oversample?

It’s fairly unusual, but is done on occasion.

One recent example was the WBUR polling on the proposed Boston Olympics. The WBUR poll (which I oversee) was tracking opinion both statewide and in Boston, given the importance of public opinion in each geographic area. Because voters in Boston make up a fraction of the state’s total voter rolls, a statewide poll would likely yield fewer than 50 Boston respondents. So the poll oversampled Boston voters.

The poll found, for example, that 44 percent of Boston voters and 42 percent statewide supported the Olympics. But again, before looking at statewide numbers, Boston residents were weighted down to their proper proportion of the state’s total residents.

  1. Is there anything even remotely sinister or untoward going on here?

No.

No mainstream media outlet, and I mean none at all, has any interest in producing skewed poll results. They would also likely be unable to intentionally skew results even if they wanted to, since most media polling is conducted by private polling firms or university departments – not the media outlets themselves. These polling outfits live and die on their reputations of accuracy and reliability, and have no interest in such a ham fisted plot. 

If Trump and his allies don’t like the results of the latest polls, their complaint is not with oversampling.

Steve Koczela is president of The MassINC Polling Group. He writes for NHPR about polling, voter demographics and other topics related to New Hampshire politics and elections. Follow Steve on Twitter.