Accepted for Publication

Hollander, Barry A. (forthcoming), The role of media use in the recall versus recognition of political knowledge. Journal of Broadcasting and Electronic Media.

The only thing missing in the title above is a colon. Without one, you kinda doubt it's actual scholarly research.
 
What's it about (you ask breathlessly)?

Glad you asked.

I used a question-wording experiment in a national survey to see whether recognition-type political knowledge questions (multiple choice) get different results than recall-type questions (open-ended). First, of course they do. Everyone knows multiple guess is easier than short answer questions. I hypothesize something more -- that people who rely on TV news will do better on recognition questions but people who rely on print will do better on recall.

It worked. Mostly. For TV news exposure, it predicts recognition (but not recall) knowledge. Print newspapers and radio do nothing. But using Internet for news predicts recall (but not recognition) knowledge. And yes, this was a multivariate analysis, meaning I controlled statistically for all the usual suspects (age, income, education, political interest, etc.).

And to top it all off, among the less educated, TV plays a bigger role in recognition knowledge than it does for those with greater education.

There's a bunch of theory stuff in the paper, but basically TV news is all about recognition, is all about being an information leveler for less educated (or interested). Recall requires deeper attention. What's interesting is exposure to paper newspapers does nothing and it seems to have been supplanted by Internet news as a predictor of knowledge. That alone is kinda fascinating, in a PhDweeby kind of way.

This will all appear in a journal near you, sometime in 2014. 

 

News with a Partisan Slant?

I wrote a couple of days ago about a new Public Opinion Quarterly study that finds people who consume "likeminded news" (news that agrees with their partisan point of view) actually increases certain aspects of political participation.  In that piece I promised to review the study author's categorization scheme for which television news programs qualified as being slanted toward Republicans, Democrats, or neutral.  Here's my take.

First, let's recreate the list from the study's appendix (Table A1, p. 312, if you have access to POQ).  Remember these are 2008 programs, so some have disappeared.  I comment after each category, only briefly, then I'll get into details after the list on how the author assigned programs or networks to the categories.

Programs with a Republican Slant

The Beltway Boys
Fox & Friends
Fox News
The Fox Report with Shepard Smith
Geraldo at Large
etc etc., all of them Fox News programs...

Okay, I stopped here listing them because they're all Fox News shows, and I think most of us might agree with the slant (except Studio B with Shepard Smith, which to me is one of the more "straight" presentations on Fox.  So I quibble with that one).


Programs with a Democratic Slant

ABC Nightline
Anderson Cooper 360
BET News
CNN Headline News
The Colbert Report (really, it's in there)
Countdown with Keith Olbermann
The Daily Show (really)
Good Morning America
Hardball with Chris Matthews
Late Edition with Wolf Blitzer
MSNBC Live
Out in the Open
Situation Room
The View
This Week with George Stephanopoulous

As you can tell, a bunch of major networks and their shows.  Again I have quibbles, especially with Situation Room, which strikes me as a journalism guy with some small experience as fairly straightforward in its presentation. And it's amusing to see Steward and Colbert in there as news programs, but I think we can all buy into that as well.


And finally, Neutral Programs

ABC World News
CBS Evening News
America This Morning
CBS Morning News
etc etc. all three major networks and morning shows
Frontline
Larry King Live
Lou Dobbs
60 Minutes
McLaughlin Group
20/20
NewsHour
Reliable Sources
Meet the Press

Okay, we can quibble with the list above too.  Lou Dobbs?  Neutral?  McLaughlin Group?  All in all, the list above seems mostly right to me, but of course your first methodological question should be -- how the heck do you systematically decide what program or network goes in which category?  Good question.  The explanation, also found in the article's appendix, is complicated.  Let's walk through it.

• What candidate does a program favor?  This was a follow-up question to respondents who reported certain media exposure.
• Mix and Match.  "I took into account respondents' perceptions of partisan slant for both the program and its parent network," says the author, trying to assign a category to unmentioned programs or to further justify the categorization.
• A statistical threshold of 25 percent was used to make sure the categories were firm.  More on this in the appendix.
• A Lexis-Nexis search was used for rarely mentioned programs, using various terms like liberal or Republican to assign a slant to a program or its host.

This is some creative work, and I've only skimmed above the detailed effort to assign programs to what any reasonable, informed person might (or might not) agree with.  But it's important some systematic approach be used, not just gut feeling, when doing this kind of work. So I applaud the effort while quibbling with a few of the assignments.  After all, in large part we're talking above about general respondent impressions of a news organization or program and their slant, not the reality of its journalistic approach.  Given the constraints, this is an excellent stab at categorizing programs and networks.

Knowledge about Global Warming

Rather than measure and examine plain old knowledge, some scholars take that extra step.  This study published this year in Science Communication of what people know about global warming is one example, where the authors constructed a "knowledge complexity" dependent variable as well as that old standard -- accuracy -- and then, and this is kinda neat, an interaction term of the two (complexity x accuracy).

Okay, so what's knowledge complexity?   Here's a bit from the methods section:

That is, a schema may be considered complex if it contains a variety of domain-specific elements that share conceptual linkages; thus, a measurement of cognitive complexity should seek to measure the number of elements and the degree to which they are associated.

 There are a lot of results.  Go through them yourself if you want specifics, but a bunch of variables in the multiple regression significantly predict knowledge complexity.  Education, obviously.  A variable called Understanding (how much you understood info about it).  Information seeking styles, yes, the effort one puts into it, no surprise, and a variable called media sources (the number of sources used).  All those, significant.  Not significant?  Individual media exposure items like newspapers, TV, and the web.  Not sure why they are significant but not the overall measure of number of sources, except perhaps for some multicollinearity issues.


In a final table the authors break down the complexity variable into parts (elements and connections). You don't learn an awful lot from this, but helpful nonetheless if one wants to explore this approach to knowledge. 

It is interesting to see media sources is a positive predictor of the "elements" dependent variable but not the "connections" dv.  In other words, the more sources you use, the more "things" you remember about global warming but you don't see any more connections than someone with fewer sources.  That's interesting and deserves deeper thought.

Teen Cellphone Use

Teens are closing the gap on adults when it comes to cell (mobile) phone use, according to a new Pew study. See the graphic to the right, or click on it to see a bigger image.

Yes, my teenagers finally got cell phones last month -- they'd been borrowing ours for years -- so they'll show up in the data soon enough.

The demographics of teen cell users are fairly consistent. Boys and girls, about the same. Race, not really a factor. Income, some effect but not as much as you'd think. Scroll down to see those tables if you're into data crunching.

What's this to do with what people know? A lot of these phones are truly mobile media devices, so you can get news and weather and sports -- if you choose. Or think of it in this way: every minute spent staring at that tiny screen is one less minute on the computer or watching TV or reading, thus cutting into the time in which political learning might take place.

The media pie doesn't usually get bigger, the slices among the different media tend to become smaller and smaller for older media, bigger and bigger for the newer media. In other words, radio never disappeared, but its slice of the pie (our time) got smaller, and became more specialized such as music in our cars or at work. The "slice" of mobile phones continues to grow and as a consequence, time spent with other media will likely suffer. Since its hard to get a lot of news via phone (not impossible, but less likely), you'd have to hypothesize that political knowledge will also be reduced.

When a Good Idea Unravels

Another good theory -- shot down by the data.

I'm looking at how much people trust the government and are satisfied with democracy after an election. There's been a lot of work that shows, much of the time, losers are -- not surprisingly -- more unhappy with government and democracy after they become, well, losers. It occurred to me that when you expect to win, but lose, you'd be even less satisfied with democracy or the government.

Not so much.

Playing with data from the 2000, 2004, and 2008 presidential elections, I really found no significant effect. I was sure people who predicted their candidate would win but ended up on the losing side would be really pissed, for all kinds of solid theoretical reasons. But noooo, the data says I need to find something else to think and write about, at least for publication in a respected academic, peer-reviewed journal. So instead I write about it here, and a not-so-respected, no-peer-review blog

The winner-loser thing doesn't always work, either. In fact, winners in 2000 (Republicans) actually score lower on trust than the winners. This gets all mired in partisan and ideological feelings about big bad federal government that goes beyond mere winning and losing. A lot of the research fails to take this into account, by the way. A serious failing.

Anyway, I'm gonna set this one aside as one of those great ideas that didn't quite pan out, one I may revisit at some point if inspiration strikes.

But while I'm in prediction mode for elections, I am going to do some work in a favorite topics of mine, wishful thinking, which is basically the notion that we tend to see our own candidates (or sports team, or whatever) as the likely winner. This effect is robust and even exposure to polls that tell us our guy/gal is certain to lose does not moderate this preference-expectation link.

Basically, who we prefer is who we think will win, and all other information be damned. Gonna play with that and examine the effects media consumption might play, because dammit it ought to moderate the effect. But ... and here's an angle worth exploring, selective exposure/attention to bias-confirming information (conservatives watching Fox News, for example) may actually enhance wishful thinking. Here we have two competing hypotheses -- that means if I present 'em both, maybe I'll nab support for at least one of 'em.

Time to get the old SPSS rolling ...

Where Ya Get Your H1N1Info

H1N1, or swine flu. Where are people getting their info on this much-discussed version of influenza? I give you two different lists, both courtesy of the great folks at the Pew Center in this new report.

Most Useful Sources (in order)

1. Internet
2. Cable news
3. Local TV news

Source of Information

1. Local TV news
2. Cable news
3. Network evening news

That second, disturbing list, deserves some explanation. Respondents were asked where they learned something about H1N1 and they could rattle off any number of sources, so 69 percent somehow managed to find local TV news as a source of swine flu info and 63 percent identified cable news. Both have covered this frothing at the mouth, but that's a different matter.

Indeed, the Pew folks found H1N1 to be the big story in terms of interest, even besting that perennial favorite the economy.

Of course H1N1 so far isn't as bad as TV went on and on and on about, and there are even reports of people thinking about swine flu parties to get infected. Why? Apparently there is anecdotal evidence that back in the big one, the 1918 epidemic, people who caught an easier Spring version were better able to handle the next Winter version that killed so many people. The thinking goes -- and I do NOT recommend this because it's dumb -- that catching it now will help you next Winter when we get a nasty, mutated, angry version of the swine flue sweeping across the globe.

The Internet is the growing source of info, and compared to TV I have to say -- thank god. Then again, it's hard to know where on the Net people are getting their info, so lemme take back that thanks until we know more.

Knowing More -- Added Stuff

A Pew Center person pointed me to a report that breaks down where on the Net people are going for H1N1 info. Thanks!

The good news? The CDC is #1. The bad news? Wikipedia is #2. Then there's Google and Yahoo, basically search engines, and then oddly enough ... MySpace? The land of awful music and even awfuler web pages, a source of swine flu info? CNN shows up on the list at #6 and sadly is the only news site mentioned. There's also another government web site, and ... Facebook?

The only thing missing from this frightening scenario is Twitter. Probably comes in at #11.

Summing Up

After years of marinating in the topic, this week I sum up what we know about the relationships between media use and political knowledge. Sometimes I'll even talk about one without the other, but mostly how the two fit.

Topic Today: TV News.

Let's get the obvious out of the way first. TV news kinda sucks. You see it all the time -- propping up two partisan hacks against one another and think this somehow magically informs viewers. And local tv news is laughable. That's all easy and out of the way so now we can move on to how watching TV news is related to what people know.

For years the answer was simple: TV news does little or nothing how people learn about politics and public affairs.

Hell, we'd even find negative relationships. In other words, the more you watch TV, the less you knew. For those of us who study cognition, we called it cog-sucking. TV sucks out your brains.

And now we know better.

For the least educated, the least interested, TV news actually aids in learning. TV news makes public affairs accessible. It tells stories in ways people understand, often with narrative hooks or focus on a single person to get us into the story. For those who don't know much, or don't care much, TV news can make a big difference in what they know about the world.

What else do we know so far? TV news is fragmenting. Spin around the dial to the cable guys and watch for a while. MSNBC to the kooky left, Fox to the kooky right, and CNN just kooky. Forget about local tv news. Delusions of adequacy is going too far and no serious person takes local tv news, well, seriously (there are exceptions, but not gonna get into them here).

How people learn from TV is an interesting question, given the flashing video and lights and pretty people and smarmy chatter and talking heads and music and all the rest. People glean bits and pieces from the news through, often, inadvertent exposure. The highly informed, they can get a lot out of breaking news, the less informed get a bit here and there. Given the ceiling effect for the highly informed they can't improve much, but for the least interested, it makes a big difference. For that reason alone, TV news plays a vital role in a democracy. Think of TV as necessary but far from sufficient for a democracy to succeed.

Tomorrow, either more on TV or perhaps a different medium.

Media Variables

A coupla days ago I discussed the ANES 2008-2009 panel data and its political knowledge variables. I tried yesterday to discuss the media variables but twice Blogger decided not to save or publish anything I wrote. So I'm trying again today.

While the political knowledge items are full of promise for analysis, the media variables ... not so much.

There are four questions, all following this format:

During a typical week, how many days do you watch news on TV, not including sports?

So the response alternatives range from 0 to 7 days. The four questions cover news on TV, radio, the Internet, and "a printed newspaper." There are a few things to like here. Using "printed" is smart. Helps respondents differentiate between the dead-tree and online version. And the "not including sports" is smart too.

Now time for the but. You knew it was coming.

TV news is too general in a fragmented media market. So is radio. Respondents may confuse NPR and Rush Limbaugh and a fragment of news caught every day between Led Zeppelin songs. And mere exposure? Attention is more important, especially when studying the effects of TV news.

ANES has a tradition of mediocre media items, though there have been individual years where they surprised me, especially when a list of entertainment programming that allowed for some interesting tests to be conducted. From a media variable perspective, these are disappointing.

In a perfect world, ANES and Pew get together and marry the ANES policy and knowledge items with Pew's brilliant media questions.

Offered With No Comment

WSJ, Pew, and Knowledge

Interesting column online by the Wall Street Journal's "Numbers Guy" that looks at the Pew research on what media people say they use and their political knowledge. It's more about what media people say they consume than discussion of political knowledge, but still kinda interesting, especially the stuff on social desirability when asked what news or television we watch.

We know social desirability is a problem in survey responses. For example, we underestimate how much TV we watch and when reporting our TV viewing, we overestimate how much "quality" or "highbrow" stuff we watch (PBS) and underestimate how much crap we watch (Dancing with American Idol's Survivor).

It's an interesting problem because overestimation of news viewing could scram the numbers, create artificial relationships with measures of political knowledge. Higher or lower, I'm not sure. Gotta think it through, and to do this you'd have to consider what kind of people are more likely to overestimate news consumption.

There are measures of this, by the way. Sometimes in surveys we'll try to get a sense of someones likelihood to answer questions in a socially desirable way and then control, statistically, for this possibility. But you don't see this done very often in surveys. That's probably too bad.

Where People Get Their News

New Pew numbers, no surprises. Traditional media use aiming for the x-axis, use of the Internet going up toward whatever the hell is parallel to the x-axis but higher.

The report does a nice job of typing people: integrators, new newsers, traditionalists. The bad news? The decline in print outpaces the gains in online news consumption.

Does Knowledge Stand Alone?

Pardon my PhDweeb moment as I go theoretical and methodological. I want to discuss (briefly) whether political knowledge is a worthy concept standing alone or whether it should be folded into some overall concept.

Candidates include expertise and sophistication and awareness, all which have some history. Let's pluck out sophistication, just for the sake of argument. What makes someone sophisticated in a realm such as, say, politics? Obviously knowing stuff about the topic, thus we get knowledge, but also we need to get more theoretical. We need motivation and ability, for example.

Political interest probably deserves mention. It suggests motivation. We usually measure this with a single item that asks if a respondent cares about or is following politics or a campaign or whatever it is we're studying. High in this, people have the motivation to keep up with politics. Doesn't mean they have the ability to comprehend, so with that we might add education, which both gives one the motivation to keep up (you see the stakes and how they affect you) and the ability to do so (literacy, background info, etc.).

Okay, then there's knowledge. This suggests the ability to translate what one has been exposed to into long-term memory, and it suggests one can access that information on that domain. Knowing stuff matters.

And finally we have media use. This is the channel that delivers the information, and exposure to it says something about motivation (I keep up, therefore I must care) and potential ability (I hear/read about politics, therefore do I actually process the info?). Knowledge is the end result of media use, then.

So do we gain anything by studying sophistication rather than the individual factors that might be thought of as factors in a multidimensional variable? I dunno. I believe we lose something by lumping it together in a single variable, in part because not all media exposure is the same (CNN vs Fox vs Print, etc.), in part because not all knowledge is the same. Textbook versus civics versus current versus recall versus recognition, all these matter when studying what people know, and how they know it.

My gut feeling is that sophistication is a lot of work to fold in a lot of variables and losing some rich data in the process. This might have worked in a more homogeneous media environment, but I'm not sure it works today in our fragmented marketplace.

Virtuous and Viscious Circles

The news media promote both a virtuous and vicious circle when it comes to learning about public affairs, according to a new study in the British Journal of Sociology.

How's that for a news lede?

"The stronger finding is that use of media and, especially, a positive engagement with the news, seems to sustain both voting and an interest in politics," the authors write. That's the good news, the virtuous stuff. The "already-engaged" become more interested and engaged; however, the opposite, 'vicious circle' is also indicated, with the unengaged becoming less interested or engaged."

This sounds remarkably like the old knowledge gap thesis, though it's not cited. That's being picky. The point is that news consumption does contribute to factors of civil participation, but not so much for those already "unengaged" in voting, in discussion, in being involved in the stuff of democracy.

Good news? The media "enhances more than undermines participation." Bad news? It can also have some dampening effect in actual political action by some consumers.

Question Order Effects

The order you ask questions can influence how people answer survey questions. That's why you often see the general all-purpose "how would you evaluate the president's job" first, not after a long series of questions that ask about all the stuff that is wrong with the country.

Order matters.

One of my earliest academic efforts was a question-order paper. Hell, it even won first place in the theory and methodology division at AEJMC (must have been a weak year because I didn't even mention agenda setting, and I don't think T&M allows a paper that doesn't mention agenda setting. It's a law).

So how about political knowledge? Well, according to one study, asking difficult political knowledge questions early in a survey will dampen how much respondents say they pay attention to the media and are interested in politics. That's kinda interesting. Maybe they get these tough questions and become more self aware of their actual knowledge or interest or media use, or perhaps it artificially dampens their true sense of how much they care and keep up with public affairs. Give people a "buffer" item that allows them to think it's okay to not know so much and the effect is less.

The upshot is such questions, like sensitive items people hate to answer, should go later in the survey. We often preface questions today to make it okay for people to say they don't vote (always inflated) and now you will see respondents told that some people don't keep up with the news, then they are asked "how about you?" This makes good sense. We want accurate measures, not socially desirable ones.