Friday, May 13, 2016

Of Polls and False Precision

There's a new AJC poll out today, via this story, that reports a 4.26 percent margin of error. In the biz we call that false precision because, honestly, 4.3 will do just fine. Also, best I can tell, the story includes little or no survey methodology. How many were surveyed? When? By phone? Robo-dial? Were cell phones included?

About all we get is a hint in the last graf:
Dissatisfaction also varied geographically. It was highest in North Georgia, where only 9 percent of the 115 people polled said they approved of Congress. People in metro Atlanta, meanwhile, were overall happiest with Congress, with 30 percent of the 208 people surveyed approving of the state of Capitol Hill.
Which tells us ... not much at all. Can't really reverse engineer this because I assume the survey included respondents outside metro Atlanta and north Georgia.

I don't blame the polling firm -- Abt SRBI -- because it's an upright group participating in AAPOR's transparency initiative and works with the Pew Research Center, a really good shop. My hunch is the writer simply left out the info dump on poll methodology, despite such stats being strongly recommended by certain bloggers who teach public opinion and most folks who report and write polls for national news media. At the very least, we need to know the sample size, the time the survey was conducted, and how it was done. Be fair to your audience.

And if nothing else, it'll make The R&B happy that I'm complaining about someone's poll other than theirs.

No comments: