Friday, December 4, 2015

That Magical 30 Percent

There's been a long-running survey on the UGA campus. You can read the most recent story here. Here's my interest, in the next graf pulled from the story:
The survey ran from Oct. 20 to Nov. 20 with hopes of collecting anonymous response data from 30 percent of the university population, including students, faculty and staff. And while just 48 hours from the survey’s conclusion the survey was 1,000 respondents short of its goal, Michelle Cook, associate provost for Institutional Diversity, said she feels that the goal was met.

OK, let's unpack that graf a bit. Why 30 percent as a goal? For what reason? I know a thing or two about survey research and public opinion and for the life of me I can't find anything significant about 30 percent versus 25 percent, or 50 percent. Now 100 percent, that's a census.

Then again, it's a non-random sample, so the results are questionable anyway. Plus, if you know how, you can fill it out again and again. I know. I did. Three times. Data duh.

Finally, I love how we have this goal, but we're gonna be short, but someone "feels that the goal was met." That's not fluffy PRspeak, to be fair. She suspects there are outstanding surveys still to come in that will reach the magical, inexplicable goal of 30 percent.

WARNING UGA -- A 30 percent SLOP (self-selected poll) is meaningless. You cannot argue the sample is representative and therefore generalizable to the campus population you're trying to describe. Which is the purpose of the survey. You might try statistical weighting of the results, but I'm not sure you want to go there. Doable, but tricky. You don't know how many repeat surveys you have, but given the nature of the questions I'm fairly certain you have a number of them as folks insert their favorite social and political agendas into the data.

When the survey is "released" I may have to request the raw data just to check on their "interpretations" of the analyses. No doubt that'll piss off some suit somewhere up the hill.

That's my job.

Does this make me the anti-diversity asshat? No, and I'm not. What I dislike, though, are data-based decisions based on bad, skewed, questionable data just to fit someone's preconceived notions of policy changes they want to implement. It's called political cover. And it's bullshit. Just woman up and say these are the changes we want to make, we feel are in the university's best interest. Hell, it's how they do everything else at UGA, a place where they've even centralized when classes can be taught -- day and time -- and in what room.

I'm not the asshat here.

1 comment:

Shiv Patel said...

What I find more interesting is that the University paid an outside consultant to do this. Surely a company that does this full time would know better.