Survey Story

Updated Below

There's a good Red & Black story out today on a UGA survey I've critiqued before (see here, here, and here). I don't want to repeat my earlier critiques other than make a few comments about the poll and the story itself. First, there's this graph (bold face by me):

The results were released in July, with an overall response rate of 23 percent, or just more than 10,000 people, according to the executive summary. This was below the target goal of 30 percent, which Cook said was needed for the survey results to be generalized for the entire university population.

I've looked hard to find any justification for that 30 percent in a non-probability survey (nice way of saying a self-selected survey) being generalizable to the population. I can't find it in the survey literature, I can't find it on major survey sites like AAPOR, I can't find it anywhere. I admit I could be missing it, and this 30 percent comes not from the university spokeswoman but the consultant. It's in the report, but with no citation, no support, no justification. Given the 23 percent response rate loaded with far too many admin/staff types, I honestly don't think much of this survey, but that's a different issue.

And then there's this quote just above the 30 percent graph. Here I take off my public opinion hat and put on my journalism professor hat.

“The survey was really an attempt on the part of the University of Georgia to hear the voices and to learn more about the experiences of faculty, staff and students here at the University of Georgia,” said Michelle Cook, associate provost for Institutional Diversity, who chaired the 11-member committee that oversaw the survey.

I could be wrong, but this stinks of an emailed answer to a question provided in advance. If I'm wrong, I apologize. If I'm right, then dammit you should say in the attribution that it was in response to an emailed question. Be up front with the reader, especially given UGA flacks and their crutch use of requiring emailed questions and emailed responses -- which no news org should ever accept.

[I was wrong. Nate Harris, news editor at The R&B, told me via Twitter that his reporter got the quote in a face-to-face interview.]

Again, maybe she actually said it over the phone in exactly that way, but it reads like a written response to me. Tell me, the reader, if it's so.

And now something to praise. That's a helluva good graphic in the story comparing UGA's actual gender and race breakdowns with the survey's breakdowns. And other than my quibbles above, it's a damn good story with lots of comments from campus folks, especially students. Good work on that.