You Cheaters You

I'm scouting out the data available in an early release of the ANES 2016 surveys and in particular I've looked to see what questions may have a mode effect -- that is, when the survey is done one way versus another, in this case a random national sample of folks who were interviewed face-to-face (F2F) and a random sample that completed the questionnaire online.

Interesting differences do occur. I wrote about some the other day.

Today let's look at how well people answered political knowledge questions, F2F versus online. The four below ask respondents to identify the holders of four offices. I think we have some cheaters here, folks who went and looked up the answers when completing the online questionnaire. My evidence? You be the judge. In the table below I provide the percent correct to the questions using what's called scheme 1 (some folks were randomly assigned to scheme 1, others scheme 2. The results were similar and I didn't want to take the time to combine the groups).

	Percent Correct
	F2F	Online
Speaker of House	43.3	57.7
German Chancellor	26.1	48.3
Russian President	79.9	84.7
U.S. Chief Justice	7.4	33.6

Clearly the online folks were either much smarter, which is unlikely as people were randomly assigned to either the F2F or online group, or some of them took the time to go look up the answer. Tsk tsk tsk. Cheaters cheaters cheaters. How else can we explain 7.4 percent of the F2F getting a question right and 33.6 percent getting it right in the online group?

Which makes for an interesting methodology question. How do you handle this if you're studying political knowledge? Instead, there might be an interesting study here in who cheats, if only we knew whether they did or not, but with some massaging of the data we might get a clue. Will look harder at this over summer.