A Tale of Two Polls

Two presidential horserace polls went into the field at about the same time and the results reported within a day of each other. One was by Fox News, the other by CBS/NYTimes. They have significantly different results.

The Fox News poll has Trump ahead by 3 percentage points (within the margin of error). The CBS/NYT poll has Clinton up by 6 percentage points (beyond the margin of error). That's a 9 percentage point difference, enough to make you ask, WTF?

As Ron Faucheux notes in his lunchtime email blast, the difference may very well lie with how the two polls measure "independent" as a category. While the results of these two polls among Republicans and Democrats are similar, it's the Independents where we see differences. In the Fox poll, Trump beats Clinton 46-30 among Independents. In the CBS/NYT poll, it's a 40-40 tie.

What's going on?

Look hard at the number of "Independents" in each poll. For Fox News, only 16 percent of respondents classify themselves as such. In the CBS/NYT poll, 36 percent are classified as "Independents." How can there be such a huge difference in the number of Independents floating out there? One possibility is question wording.

Here's the Fox News question:

When you think about politics, do you think of yourself as a Democrat or a Republican?

Now, here's the CBS/NYT question:

Generally speaking, do you usually consider yourself a Republican, a Democrat, an Independent, or what?

See the difference?  The Fox question doesn't offer "independent" as a choice. You, the respondent, must volunteer it, making this more of a forced-choice question. Built into the CBS/NYT question is the opportunity to choose Independent, "or what." If the data from these questions are used in weighting the sample, it can have small effects, or even big effects, on the results.

Which is the better method? Hard to say. And keep in mind these are surveys of "registered voters," not "likely voters," so they're predictive power so early in the campaign are, honestly, meaningless -- especially when you consider the election is not nationwide but state-by-state. Still, there's much to learn in two polls, out at the same time, with different results.