People Lie

I've written about this extensively, research into how people lie (or fib, or fudge, or exaggerate) in surveys.  The most recent issue of Public Opinion Quarterly has a study that examines viewing of U.S. presidential debates. 

The finding? 

People lie.  Or, at least, they exaggerate about their viewing, not unlike previous studies that find people say they voted when they didn't, or that they inflate how often they consume the news and so on.

In the POQ study, Marcus Prior compared survey questions about debate exposure with, as a benchmark, Nielsen ratings of the vice presidential and presidential debates from 2000 through 2008.  Between 47 and 63 percent of voting-age residents said they watched the presidential debates, about double what the Nielsen ratings say is the more likely number.

But is Nielsen a good benchmark?  Probably so, and Prior addresses this by examining data in different ways to make clear his point -- that surveys make for a lousy way to evaluate exposure to a major political event.  It may be all we have, but one filled with error and inflation, so much so that it may affect studies that seek to understand the consequences of viewing on opinions and voting preferences.


 

News with a Partisan Slant?

I wrote a couple of days ago about a new Public Opinion Quarterly study that finds people who consume "likeminded news" (news that agrees with their partisan point of view) actually increases certain aspects of political participation.  In that piece I promised to review the study author's categorization scheme for which television news programs qualified as being slanted toward Republicans, Democrats, or neutral.  Here's my take.

First, let's recreate the list from the study's appendix (Table A1, p. 312, if you have access to POQ).  Remember these are 2008 programs, so some have disappeared.  I comment after each category, only briefly, then I'll get into details after the list on how the author assigned programs or networks to the categories.

Programs with a Republican Slant

The Beltway Boys
Fox & Friends
Fox News
The Fox Report with Shepard Smith
Geraldo at Large
etc etc., all of them Fox News programs...

Okay, I stopped here listing them because they're all Fox News shows, and I think most of us might agree with the slant (except Studio B with Shepard Smith, which to me is one of the more "straight" presentations on Fox.  So I quibble with that one).


Programs with a Democratic Slant

ABC Nightline
Anderson Cooper 360
BET News
CNN Headline News
The Colbert Report (really, it's in there)
Countdown with Keith Olbermann
The Daily Show (really)
Good Morning America
Hardball with Chris Matthews
Late Edition with Wolf Blitzer
MSNBC Live
Out in the Open
Situation Room
The View
This Week with George Stephanopoulous

As you can tell, a bunch of major networks and their shows.  Again I have quibbles, especially with Situation Room, which strikes me as a journalism guy with some small experience as fairly straightforward in its presentation. And it's amusing to see Steward and Colbert in there as news programs, but I think we can all buy into that as well.


And finally, Neutral Programs

ABC World News
CBS Evening News
America This Morning
CBS Morning News
etc etc. all three major networks and morning shows
Frontline
Larry King Live
Lou Dobbs
60 Minutes
McLaughlin Group
20/20
NewsHour
Reliable Sources
Meet the Press

Okay, we can quibble with the list above too.  Lou Dobbs?  Neutral?  McLaughlin Group?  All in all, the list above seems mostly right to me, but of course your first methodological question should be -- how the heck do you systematically decide what program or network goes in which category?  Good question.  The explanation, also found in the article's appendix, is complicated.  Let's walk through it.

• What candidate does a program favor?  This was a follow-up question to respondents who reported certain media exposure.
• Mix and Match.  "I took into account respondents' perceptions of partisan slant for both the program and its parent network," says the author, trying to assign a category to unmentioned programs or to further justify the categorization.
• A statistical threshold of 25 percent was used to make sure the categories were firm.  More on this in the appendix.
• A Lexis-Nexis search was used for rarely mentioned programs, using various terms like liberal or Republican to assign a slant to a program or its host.

This is some creative work, and I've only skimmed above the detailed effort to assign programs to what any reasonable, informed person might (or might not) agree with.  But it's important some systematic approach be used, not just gut feeling, when doing this kind of work. So I applaud the effort while quibbling with a few of the assignments.  After all, in large part we're talking above about general respondent impressions of a news organization or program and their slant, not the reality of its journalistic approach.  Given the constraints, this is an excellent stab at categorizing programs and networks.

Like-Minded News and Political Participation

A fascinating study published in the latest Public Opinion Quarterly asks a very simple question -- does consuming news you agree with make you more likely, or less likely, to participate in politics?

The answer?  More likely to participate.  But with limitations.

There is a growing body of work on the migration of partisans to news that generally agrees with their political point of view (as in, conservatives to Fox, liberals to MSNBC).  This study by Susanna Dilliplane uses national panel data and some sophisticated modeling to explore whether these trends lead to greater or less political participation.  It's terrific work (and yes, it cites two of my published studies, another big plus.  End shameless plug).

There are competing theories and findings as to whether partisan media have mobilizing or demobilizing effects.  People generally agree that exposure to competing view points is a good thing (see the work of Mutz, for example).  But does such exposure actually help, or hurt, when it comes to participation?
 
Using the 2008 presidential election as a test, the research finds "substantial support for the proposition that exposure to partisan news affects political participation, particularly behavior during the campaign."  For example, consuming likeminded partisan news increases campaign acitivity and encourages people to make an early decision about the election.  Consuming news that conflicts with your partisan point of view -- that can have the opposite effect.  So what is generally considered good, hearing lots of viewpoints, demobilizes you, makes you less likely to participate.


But, and this is a major but and therefore deserves special boldface and italic treatment, such partisan exposure is unrelated to actual voting.  This may be due in part to the use of within subjects rather than between subjects analysis.  After all, this is panel data, so you look at change within the same people over time.  That doesn't leave a lot of room, statistically speaking, to find effects like voting.  One hopes it's not a methodological artifact and partisan viewing/reading actually does not influence likelihood to vote.  One hopes.

On a methodological note -- how do you decide whether a television news network is likeminded or not? Yeah, we can all agree Fox or MSNBC is likeminded for conservatives or liberals.  Here the author does some nifty combining of data to categorize the media.  Nifty, though some might quibble in relying on public perceptions of a news organization's partisan leanings.  The appendix breaks these down for you.  Republican slant?  Fox News and nearly all of its specific programs (O'Reilly, for example).  Dem slant?  Colbert and Stewart show up here, as do MSNBC and its various programs, and CNN's Situation Room (I'd quibble with that one, a lot).  Neutral?  A bunch of stuff.  I'm going to blog about this categorization scheme at another time.

What's the takeaway here?  While we applaud the exposure to various viewpoints, it may actually not be good when it comes to people participating in politics.  The theoretical underpinnings of this finding are in the article so I won't go into detail here.  But it is interesting to note that exposure to more neutral new coverage shows hints here of also mobilizing folks.  That's good news for those of us who prefer a world with multiple viewpoints presented in news stories rather than a single, preached narrative by certain (ahem) cable news networks.

Everybody Lies -- Especially about Church Attendance

As Dr. House famously says: "Everybody lies."

As my father used to less-than-famously say, people lie about three things: voting, going to church, and their gas mileage.

This is about the middle one, the lying about church. No preaching.  Instead I'm pointing at a new Public Opinion Quarterly piece on the overreporting of church attendance in the U.S.  We've known for some time that people exaggerate their attendance at religious services, but this study does something interesting -- it looks across 14 countries to compare how Americans stack against others. 

In general, surveys suggest a "yawning divide" in religious attendance between Americans and Europeans and Canadians.  Call it American religious exceptionalism.  But is this really so?  Using diary data rather than cross-sectional surveys, Philip Brenner finds Americans to "in line with a number of European countries."  Yes, American religious attendance is high compared to other similar countries, but so high as to qualify as an "outlier" and, according to these data, looks a lot like a few other countries.

What I find interesting is why this gap exists between what we say we do, in terms of religious behavior, and what we really do.  Maybe House is right.  Maybe everybody lies.  Or maybe this has more to do with self-presentation and social desirability, a sense of how we should answer versus what we really do.

What People Know isn'tas Important as We Thought

An article in the Spring 2011 Public Opinion Quarterly includes an article with a title sure to grab my attention: Rethinking the Role of Political Information (pdf of full article here).

Why did it grab my attention?  First off, I don't want to have to rethink anything.  How dare the journal and the author suggest I do so.  And yet here it is, POQ, saying I should.  Of course anything with political information dovetails nicely with what I scribble about here, so I skim academic journals for blogging material (because, let's face it, no real person with a life is going to bother doing this). 

Reasons enough, and yet here's the kicker for me -- it's a study that challenges the conventional wisdom.  Not challenge in an earth-shattering, paradigm-shifting kind of way, but the author (Matthew Levendusky at Penn) does suggest, based on his analyses, we give too much credence to the power of what people know.  He does a nice job of discussing how much political information matters, or we think it matters, while managing to appropriately not cite me at all (grrr, c'mon guy, I need the social sciences citation hits).  In an analysis that gets a wee bit more complicated than I want to wrestle with here, he finds support for his notion that previous studies overstate the relationship between political information and political behaviors such as voting and participation.  Or, as he put it, "the effects of general political information are much smaller than previous estimates suggest, thereby opening up the possibility that other factors and other kinds of information ... matter to voters."   

In a large part, this matters mostly to those who do academic research in which political knowledge is a control variable in predicting political behaviors.  In a small part it suggests something more and something deeper, that perhaps what people know -- always thought to be vital in a democracy -- perhaps doesn't matter as much as we thought, and that the costs of improving political knowledge in the electorate will provide us few gained benefits in other behaviors, like voting and participation. That's a little depressing.

Sober Media II

I blogged a couple of days ago about this new, interesting study in Public Opinion Quarterly that created a dichotomy between sensationalist media and sober media.  That earlier posting focused on the methodological validity of that dichotomy, particularly as measured in the POQ study.  Today, the results.

The dependent variables centered on attitudes about state and federal courts and a major aspect of this is not only the sensationalist/sober dichotomy but how it interacted with political sophistication to predict attitudes about the courts.  Sophistication here was measured by a set of interest and knowledge questions pertaining to the courts.  To review, sober media were considered newspapers and network television news, while sensationalist media were considered talk radio and cable news. The authors subtracted one from the other to create a scale.

Figure 2 in the study is fascinating.  Imagine the x-axis as a measure of sophistication, the further right you go, the higher it is.  The y-axis, the dependent variable, is diffuse support of the courts.  For those who prefer high sensational media, greater sophistication leads to a decrease in support.  For those who prefer high sober media, greater sophistication leads to greater support.  It's a classic X interaction.

Essentially, the use of sensational media moderates the impact of political sophistication, which by itself leads to greater support of the courts.

In other words, media bad.  Only certain media ... cable news and talk radio, at least when used more than other more sober media, like newspapers and network TV news.  And this holds even when statistically controlling for party identification, age, sex, race, education, and trust in government.

We're starting to see a picture now of television news -- when it's good for folks, such as those low in interest or education, and when it's not so good.  Also becoming clearer is our understanding of how a fragmenting media marketplace with non-mainstream news sources are impacting what people think or know about their political world.  Unfortunately, the picture -- as it becomes clearer -- also becomes less attractive.

See the first link above, to the previous blog post, for details about the study.

Sensationalist and Sober Media

Sober media?

The latest issue of Public Opinion Quarterly includes a fascinating study of how the media influences attitudes about the American court system.  All well and good, but the fascinating part comes not from the relationship between news and attitudes about the courts, but the methodological approach taken by the authors in discriminating between sensationalist and sober media.

Stay with me.  It's worth the ride.

The authors write: "Our theoretical claim is where an individual receives information about the courts will shape the way they think about the courts."

Yup, can't argue with that.  The fragmenting of the news media has led to a number of potential differences between folks who, for example, get their news from Glenn Beck versus those who get their news from The Economist.  According to the authors, sensationalist media deconstruct court rulings in such a fashion as to result in less "diffuse" and "specific" support (these are terms described elswhere in the paper). 

Sober media, according to the authors, "focus on the legalistic nature of court processes" and will reinforce views "above the political fray."

This is important and I couldn't agree more, other than to be bitter about not thinking of it first.  Where it all begins to unravel comes from -- as is often the case -- in the methodological details.

In measuring "differential media exposure," as they call it, they distinguish between sensationalist media (political talk radio and cable news) and sober media (newspapers and network news).  They measure exposure to each and use a simple formula, shown below:

Differential Media Exposure =
(talk radio + cable news) - (newspaper + network news)

While I buy that the majority of talk radio tends to fit here, I'm not certain cable news is so easily collapsed into a sensationalist category.  And there's little evidence provided by the authors, or in any content analyses I'm aware of, to immediately support such a conclusion.  It feels right in my atheoretical gut, but so does a good dark beer.

But what they've done here is broken some interesting new ground, a fancy way of saying the rest of us can now use this formula to justify our own analyses.  And one study will build upon another, but what we don't really know -- other than in our own respective atheoretical guts -- is whether this sober to sensational media dichotomy is sound and sober, or a wee bit tipsy.

Tomorrow, a bit more about the results.

Article Details
Christopher D. Johnston and Brandon L. Bartels (2010), "Sensationalism and Sobriety: Differential Media Exposure and Attitudes Toward American Courts, Public Opinion Quarterly, 74, 260-285. 

If We Just Ask More Questions . . .

A study in the latest issue of Public Opinion Quarterly examines the nature of political ideology, throwing at the reader a mix of cool advanced statistical methods to come to a simple conclusion -- for a lot of folks the standard single-item ideology question on a traditional 7-point scale, from liberal to conservative, just doesn't work.

Yes, the political elite have become more partisan and ideological.  And yes, they've dragged a lot of the chattering class and even everyday people farther to the left or the right.  What this study finds is a whole lot of people still answer liberal on some policy questions and conservative on other policy questions, so for them the single-item ideological placement question fails to capture who they really are, what they really think.

Or, as the authors note in their conclusions:

Our results show that failing to account for the multidimensional nature of ideological preferences can produce inaccurate predictions of voting behavior for the plurality of Americans who do not call themselves liberal or conservative.

Their recommendation?  Ask more questions.  Don't rely only on that good old fashioned liberal-to-conservative question to classify people, because for those in the middle, it fails to capture what people really think and believe.  Add policy questions since people can sometimes be fiscal conservatives and social liberals, or some other mix.

What's the media angle here?  I think the fragmentation of the news media (see Fox vs MSNBC) is dragging a lot of people, kicking and screaming, into the same partisan divide elites began sharing many years ago.  Listen to Sean Hannity, for example, and you'll hear him push for a whole slate of consistent ideological beliefs across the spectrum.  I'd love to know if "talk radio conservatives" are more consistent in their ideological beliefs than "non-talk radio conservatives."  Ooooh, future study. 

Study specs: Clarke, H.D. & McCutcheon, A.L. (2009).  The nature of political ideology in the contemporary electorate.  Public Opinion Quarterly, 73, 679-703.

Our Personal Networks: A Methodology Moment

We get our political information in lots of ways -- the news media, obviously, but also from the people we talk to every day, our social network.

It's the latter that is the focus of an article in the most recent issue of Public Opinion Quarterly. The authors look at two different ways of generating survey data about people's social networks.  As the authors write:

"One method asks respondents to provide information about the individuals with whom they discuss 'important matters,' while the other asks respondents to provide information about the individuals with whom they discuss politics.  We find that respondents provide more or less the same data on their political discussion partners regardless of which name generator procedure is used."

My first thought was -- wow, cool, you can get published with a null finding.

My second thought was -- there's leeway in how we study social networks, allowing us to confidently compare studies with slightly different methods.  Useful.

The authors conclude that our "core and political discussion networks comprise the same individuals."  In other words, we talk to the same people ... politics, or not.

That's good to know from a methodological standpoint.  I assumed people carve their larger social networks into smaller chunks.  This chunk is made up of people you talk about politics with, that chunk is people you talk music with, that other chunk is for religion, etc.  Maybe some overlap, but I always assumed there were differences.  This research suggests not.

I'm a bit skeptical. Maybe I'm generalizing from an N of 1, but maybe the lack of a result here has more to do with a methodological quirk in the wording ("important matters" versus "politics") than in how we consciously or unconsciously create our interpersonal networks.  I don't know.  My gut says the method here is too mild to truly capture differences in our interpersonal networks.  The lack of results are useful, especially if you want to combine data from different approaches in a larger analysis, but I don't think we can say quite yet that we have the same interpersonal network for everything.

Helping People Decide

The latest Public Opinion Quarterly includes a study with direct bearing on what people know -- in this case, the idea that informing them before surveying them will influence how they answer questions.

The study is Helping Citizens Decide in Referendums (with, of course, a colon and long sentence to create titular colinicity).  The dependent variables were likelihood of voting, preference, strength of opinion, and vote consistency.  What's that last one mean?  It harkens back to some traditional  research that examines how people organize their political positions and whether there's a logical (consistent) arrangement of attitudes.  In other words, you tend to be liberal on a host of issues, or conservative.  Therefore, consistent.

Independent variables?  Various flavors of political sophistication or knowledge through something called ICQ -- which stands for Information and Choice Questionnaire.  Basically, providing people with info about a specific problem before asking them what they think.


A careful reader will note this sounds like deliberative democracy (see my post here on it).  Not quite.  No deliberation taking place, at least the way it's meant, but yeah -- kinda sorta the same idea, that informed opinion is different than public opinion.

Informing people in advance before polling them did lead to more consistent opinions, but more so for people with less education and interest.  This follows theory, to a point.  As the authors note, cognitive processing models such as ELM suggest that people high in political sophistication will be more able and motivated to carefully consider an issue, but here it's the people with lower political sophistication who "profit more." 

I suspect some of this has to do with the artificial nature of a lab experiment, but even so it raises some interesting theoretical and methodological questions.  We've always known informed opinion differs from generic public opinion.  This study takes another step closer to understanding not only how it differs, but the consequences of those differences.

The Inflated News Audience

Interesting article in the just-off-the-press Public Opinion Quarterly by Markus Prior in which he finds significant overreporting of news exposure by the public. Yes, there's a duh moment here since we know people overestimate how often they vote, how often they attend religious services, and how good their gas mileage is (correlates with gas prices, the higher the gas prices, the better they lie about their mileage). They always overestimate their viewing of "highbrow" content too, like PBS, as compared to actual viewing of stuff like reruns of Gilligan's Island or Sponge Bob.

So the notion that people overestimate their news exposure, no real surprise. The degree of that overestimation, that's a surprise -- at least to me. Prior reports a range of a factor of three to a factor of eight, depending on demographics, on overreporting of news exposure. He also fusses about some survey wording that mass comm types have been bitching about for decades, especially those of us with a history in the actual news business, but that's beside the point here.

Who overreports the most? Young people. By far -- if you buy into his comparison of Nielsen people meters and data from the National Annenberg Election Survey. There's a bit of apples and oranges here, but not enough to challenge his basic finding that people overestimate their consumption of news on television. Families with kids and households with higher incomes, they also overestimate as compared to all viewers.

If you buy into this analysis it raises an interesting methodological question: should we correct statistically for estimates of news exposure? Should be reduce to some degree the news exposure estimates of younger versus older respondents? Prior suggests scholars "would do well to assess media effects with research designs that do not rely on self-reported exposure at all." Damned unlikely, especially if you rely on secondary analysis of archival data, and I'm not sure exactly how to get at this unless we preface our media exposure items in much the same way we do political knowledge questions in which we tell respondents it's okay, really it's okay, if you don't know this (or watch or read the news). Someone needs to test that approach and see if it really does deflate the overinflated numbers.

Study Specs
Markus Prior, "The immensely inflated news audience: Assessing bias in self-reported news exposure," Public Opinion Quarterly 73 (Spring 2009), 130-143.

What People See -- in Surveys

An interesting Public Opinion Quarterly article examines how people "see" a questionnaire. They use eye-tracking hardware and software to examine, among other things, how respondents view a survey question and response alternatives (our fancy way of saying the various choices you're given online when asked to answer a survey question).

According to the authors, when all choices are visible, "respondents looked at the options near the top of the list longer than they looked at those at the bottom. In addition, they were more likely to choose the options at the top."

This neat study doesn't look specifically at political knowledge, so I'm going to extrapolate and ponder here just what this tendency might mean for respondents facing questions designed to tap their knowledge of politics and public affairs.

Could this introduce random or even systematic error into responses? Yes. Could this lower our estimation of people's actual knowledge? Probably. Should be put the "right" answer near the top so they can find it? I dunno, but my hunch is no.

I'd love to see this approach used specifically on political knowledge questions and put the "correct" answer in different locations to see whether that matters. All students know that on multiple choice tests, if all else fails, choose "C" as your answer, so I wonder whether in survey items in which there is a "correct" response (versus just your attitude about some policy or issue) the respondents would be more likely to seek out that right answer. I think so, but it needs testing.

Unfortunately I don't own some eye-tracking gear. Sigh.

What People . . . See

Fascinating study in the latest Public Opinion Quarterly (abstract here) that finds facial similarity between a candidate and a voter can influence evaluations of that candidate. In other words, if someone looks like me, I'm gonna like 'em.

Now this rests on familiar theoretical groundwork of similarity, but it's a very cool take. The researchers morphed a candidate's face with an experimental respondent's face (or someone else's). Yup, they liked the morphed face that looks like them most. This works best on unfamiliar candidates, only a little on familiar candidates, but it still works once you set aside party ID and policy preferences.

Thus, the evidence across the three studies suggests that even in high-profile elections, voters prefer candidates high in facial similarity, but most strongly with unfamilar candidates.

Very very neat. It's hard to make practical use of this, obviously, since a politician can't very well morph a jillion different pix of them self with all their different potential voters. But I think there are some good mass comm studies here. I'd like to run an experiment with a blogger's web site, with the blogger's photo, but morphed either to match or not match an experimental subject's own face. Can we get them to like a blogger more even if that blogger writes about stuff they really don't like, or only moderately like? This has some great PR research possibilities.

Okay you budding PhDweeb types, get to work.

Helluva POQ

A great new issue of Public Opinion Quarterly touches on many of my favorite topics: political knowledge, false consensus, innumeracy, trust in government. I've blogged about several of these already and so I'll top off my week with false consensus.

I love false consensus, always have. It's the idea that people project their own opinions and behaviors on the public at large. In other words, if I enjoy police procedural novels, I am more likely to overestimate how many people out there also enjoy police procedurals. Magdalena Wojcieszak takes the false consensus concept on the road and online to examine neo-Nazis and environmental extremists (what lovely groups) to see if the concept holds up.

It does.

For example, those participating in neo-Nazi online discussion boards overestimated how much people dislike progress in equal rights.

Part of this is due to the fragmenting media environment in which crazy people, or mildly nutso people, can group together in common cause and this selective exposure to similar others makes participants more likely to overestimate how much people agree with them. If I hang out on a discussion board of police procedural fans, my sense of the climate of opinion out there will be skewed and I'll think more people like them than really do.
(as an aside, read Ian Rankin's John Rebus novels. Bloody brilliant)

So it's a good issue of POQ for me. Good blogging material, and stuff I like to read and think about. If they'd only included a research article on the best Christmas gifts to get your wife I'd be really really happy.

Discouraging "Don't know" Responses

A Methodological Moment

An interesting Public Opinion Quarterly piece follows a line of research about the measurement of political knowledge and whether we should encourage or discourage "don't know" responses. Using an online survey, the authors argue that eliminating the "DK" response works fine and it removes self confidence and other psychological factors that influence results.

It's a fine analysis, excellent discussion. Given the holiday season and grades to be recorded, I'm gonna talk only about a few key points.

"With the DK-omitted strategy, confident risk takers are no longer privileged relative to those who lack confidence or are risk averse."

First, the writer in me absolutely hates the use of privileged in this instance. Setting that aside, basically people willing to guess often get items right even when they don't know. Removing DK takes this out of the equation.

"While estimates of political knowledge have been consistently low, it is clear that question design may be a confounding factor. As web surveys become more ubiquitous, the DK omitted strategy is an increasingly viable option for measuring political knowledge."

I agree, but worry how this applies to traditional telephone or F2F (face to face) surveys. "I dunno" is an easy out, true, but in those people-based interactions not giving them an out might create survey difficulties. Hard feelings. Frustration. Guessing. All bad, all adding random error or perhaps even systematic error. In most surveys, the "don't knows" are often collapsed with incorrect answers to be scored as a "0" and correct responses getting a "1" in some kind of summed index. In other words, "don't know" often is the same as being wrong.

(As an aside, I prefer a -1 for getting it wrong, 0 for dunno, 1 for being right, but I've not used it very often because the results look just about the same as the traditional approach.)

But as we move more and more to online use of surveys, the arguments here are well taken and important for those who research what people know.

Homophobic Innumeracy

Not only does the general public do a lousy job of answering political knowledge questions, it also suffers from statistical innumeracy. In other words, we suck at estimating risk, at how much the government spends on foreign aid, and -- it seems -- how many gay people there are out there.

A new Public Opinion Quarterly study by three people (two of who I took for class as a grad student!) looks at homophobic innumeracy and finds that, indeed, the public is bad at this. Then again, the public is bad at estimating the size of other minority populations. As the authors note, people "aim high" in these estimates.

People in the certain demographic groups aim the highest, or "mis-estimate" the most: woman, less educated, blacks. Not coincidentally these folks also tend to score lowest on political knowledge tests. Interestingly, they find no real relationship between religious beliefs, even a literal interpretation of the Bible, and overestimation.

Following politics was associated with less overestimation. Or, I suppose, more accurate estimation. Unfortunately that's as close as the authors get to a media variable. I'd love to know whether exposure to the news made one more or less likely to overestimate the number of gays and lesbians in society. My guess is that media exposure would lead to greater overestimation by segments of society. Why? Because you see gays marching or protesting or dealing with some legislation on the news, you're more likely to think there are a lot of them, or at least more of them than exist in the population.

The funny part is, there are people who argue about the numbers here. It's one of those great debates, like what's the actual number of people who attend religious services. It all depends on your methodology.

Don't Know

The latest issue of Public Opinion Quarterly includes an article asking a simple question: should survey questions about political knowledge include a "don't know" response, or should respondents be forced to guess if they do not know the answer.

A series of articles by Jeffery Mondak and his colleagues suggest that perhaps it is best to create a forced choice. Encouraging "don't knows" "invites a guessing response set" perhaps linked to personality differences, they argue, and thus is biased.

Most measures of political knowledge include a "don't know" or "no answer" alternative. Is there a systemmatic bias? Can this screw up the results in some underlying way? A POQ piece by Patrick Sturgis, Nick Allum, and Patten Smith that showed up in my mailbox today suggests that "don't know" is, at least from what can tell today, a perfectly fine approach to questions of knowledge. Their experiment finds no obvious distortions, although they recommend further work to tease out potential issues.

My own gut feeling is that "don't know" is a good alternative, that we can't really tell what are guesses, even good guesses, versus bad ones that by sheer dumb luck turn out to be correct. I've also wondered whether the 1 = correct, 0 = otherwise is a good coding scheme for knowledge items. Shouldn't an incorrect response be coded as -1, a 0 reserved for no answer or don't know, and then a 1 for correct responses? An incorrect response seems to me to be qualitatively different than no answer at all. And yet like many others, I use the 1 if correct, 0 otherwise approach.

Old habits, even old methodological habits, die hard.

Media and Nonresponse

I know that's a vague title above. Humor me.

There is a piece in the latest Public Opinion Quarterly that does a nifty job of demonstrating the effect of media coverage on nonresponse -- basically the tendency to say "don't know" or "unsure" in a survey. The more coverage by the news media, the less likely people are in a survey to say they don't know or cannot provide an answer.

The authors, Stroud and Kenski, place this in an agenda-setting framework. For those who missed mass comm grad school, agenda setting is the notion that the media do a lousy job of telling people what to think but a helluva job of telling people what to think about. Agenda setting is not a theory, more of a serious borrowing from priming and accessibility theories out of social psychology. So yeah, agenda setting suffers from a "so what" problem. This paper provides one of those "so whats" and it's a nice job of doing so.

What's the link to what people know?

Think of the present presidential campaign. Yes, I know, it hurts to do so, what with Clinton's campaign using kindergarten papers written by Obama to push polling against McCain, possibily by Huckabee's folks. It's a sad, tired, bunch of people running for office. Power attracts the corruptible, as someone once said.

What the media covers influences the likelihood to answer a policy question. That influences in a subtle way the findings on that policy question. This influences the candidates. This also colors coverage by journalists, who for the most part are baffled about polls in the first place. If hearing or reading about something in the news makes you more likely to provide an answer, this also affects political knowledge, the ability to correctly spit back to a pollster some response to a survey question.

Nonresponse can matter in polls. The influences are not huge, but they are enough to matter. And apparently the media can influence the likelihood to give an answer, or to pass.