Mixing Modes

So there's this survey of Georgians (where I live) about candidates running for various offices. The lede is fairly straightforward, that David Perdue is ahead in the race for the GOP nomination for U.S. Senate.

I entitled this post Mixing Modes for a reason. At the bottom of the survey is this bit of methodological detail:

Cell-phone respondents and home-phone respondents included in this research: SurveyUSA interviewed 2,300 state of GA adults 03/16/14 through 03/18/14. Of the adults, 1,985 were registered to vote. Of the registered, 508 were determined by SurveyUSA to be likely to vote in the 05/20/14 Republican Primary, 443 were determined by SurveyUSA to be likely to vote in the 05/20/14 Democratic Primary. This research was conducted using blended sample, mixed mode. Respondents reachable on a home telephone (78% of registered voters) were interviewed on their home telephone in the recorded voice of a professional announcer. Respondents not reachable on a home telephone (22% of registered voters) were shown a questionnaire on their smartphone, tablet or other electronic device. 

I know, kinda long. I boldfaced the important part. When we talk about mixed modes we mean exactly that -- some people here got a phone survey, others got a questionnaire delivered to their mobile device. That's an interesting way to (cheaply) include non-landline phones in an automated poll, as you can't use a robo-poll on a cell phone, at least not legally.

But it's also two very different kinds of surveys. In one, a robot voice reads off questions and you push a number to answer. In the other, a text questionnaire appears. People respond in different ways, based on a survey mode. But let's stop with that and get to more interesting stuff for you political junkies.

If you go to the bottom of the linked page you'll see lots of breakdowns of the results by various socio-demographic and political factors. Useful stuff, especially here if you slide across to the column labeled Cell Phone/ Lan (as in landline). We don't see a lot of differences. There's a hint, though, of more likely Republican voters in the landline sample (79 percent) and more likely Democratic voters in the cell phone sample (73 percent). We'd expect that, and to be honest that difference may very well be within the margin of error.

Look at Paul Broun, my kooky congressman and U.S. Senate candidate. He gets 13 percent of the landline vote, 4 percent of the cell phone vote. Whenver you see a poll that does not include cell phones, beware. More conservative, older, more Republican respondents will be found on those robo-poll landline surveys.

The lesson here? Methodology matters, and the devil is not only always in the details, he often uses them to his advantage. So know them yourself.