Debates and CNN's Squiggly Lines

Mesmerized by those little squiggly lines at the bottom of CNN during the presidential debates? Me too. Love 'em. Can't get enough of 'em. Might dress as one at Halloween.

But are the measures valid?

An article suggests not. Says Cliff Zukin of Rutgers and former AAPOR head:

"It has no scientific validity -- it's not a sample of anything that has generalized validity," he says. What's more, he argues, it introduces inaccurate numbers that assume a power of their own. "The problem with bad numbers is that people tend to believe their eyes."

The lines go down at interesting times, such as when candidates sharply question one another, or pull out some anecdote. I love the gender breakdowns. Obama and women, what's up with that? Now to be fair, the people turning these little dials, at least last night, were uncommitted Ohio voters. But exactly what does it mean when you're told to dial up if you get a favorable feeling or down if you get an unfavorable feeling? Is that specific enough, or are such general guidelines good, letting people decide for themselves what "favorable" and "unfavorable" mean.

Social scientists like more info than that. We like to triangulate, we like to measure the same concept in multiple ways. We like to be able to run Cronbach's Alpha on stuff. It makes us feel special and it gives these comforting stats that suggest we're seeing a reliable and valid measure.

So the dial-a-thon is really a question of validity (is it measuring what we think it's measuring -- some doubt this) and reliability (if we measured it again, or with some similar measure, would we get the same results -- I'm thinking probably not).

Let's get down to it then: the dial-a-thon is theatre, plain and simple.

Kinda like presidential debates.