I'm working with a large data set now that includes several measures of political knowledge. One set of questions gets at what people know of the two candidates (their religion, state of residence, demographic stuff). The Cronbach's Alpha for this index is not great, about .55. An index of standard political knowledge items such as how long is a U.S. senator's term, how many times can a president be elected, standard textbook civics class stuff, that is only .60 or so. Not good.
But questions about the candidates stands on issues, that alpha is awful, somewhere between .40 and .50 depending on which items I use. These are way too low and, to be honest, unexplainable.
Oh it gets better. These indices, created quick and dirty to peek under the hood of my data, don't line up at all with my key independent variable. And dammit they should. This is a great theory with the data getting in the way (I'll discuss more fully at another time, closer to subbing this thing to a journal). Basically, there should some relationship here.
I've blown two long afternoons on massive data recoding and analysis, triple checking all my recodes.
I shoulda been a plumber. Named Joe.