Of Polls and Punditry

So there's this column in my local daily paper by Matt Towery, a Georgia pundit and pollster, a good enough guy -- so I'm told -- who writes the following in his warning to "most people" who may not fully understand what politicos and especially pollsters are up to.

And consider the fact that most of these D.C. bandits continue to carry out polls with 50 or 60 questions attached. In this day and time, almost no one would take the time to answer a 60-question survey. And these pollsters now claim that a good percentage of their surveys are answered by folks with cellphones. That’s a farce. When have you noticed a cellphone user intently listening to their phone and carefully answering 60 or so questions? It just doesn’t happen.

I added the bold face above because, frankly, it's bullshit. It's not unlike the argument when you hear a poll result that disagrees with your own position and you mutter: "Well, they never called me." How the hell does he know what people are talking about on their phones -- unless he's NSA. And why is he staring at people on their phones? And more to the point, how the hell does he know "it just doesn't happen?" Sorry dude, it happens all the time in serious, professional shops.

Towery's shop tends to use robo-polls, those computer-dialed, automatic phone calls that are quite popular because they're fast, they're cheap, and most importantly they're fast and they're cheap. That's okay as long as you label the method and note its weaknesses, in this case, skewing older in part because robo-calls cannot, usually, call cell phones. Talk about leaving out a chunk of humanity, especially young voters. I don't hate robo-polls, and with some sophisticated weighting of the data you can offset some of their drawbacks. That's if you know math and you know stats and even then, statistical weighting is part art and part science.

But look again at the boldfaced bullshit above. Given the pro shops that do cellphone surveys, such as the Pew Research Center, and given such polls tend to be more accurate than traditional landline live interviews and especially those done by robo-polling, I can only suspect there's just a little bit of professional jealousy going on here. My local paper should know better.

Again, robo-polls are fine. In many ways they're more nimble because you can put them in the field faster, get answers faster. But they are seriously flawed in terms of their sample, enough so that they traditionally underperform compared to talking to live people and, especially, including cell phones in the same.