Sunday, July 31, 2011
Wait, Wait, Lemme Tell You ...
On the road but thought I'd pass on this link from the New York Times about the show Wait Wait ... Don't Tell Me from NPR, which is of course a current events quiz show. Otherwise, I don't expect to be posting much this week.
Thursday, July 28, 2011
Female Journalists and HPV Coverage
We all know journalists help set the public's agenda. What reporters cover, people find more important. And when people see something as more important they act in very different ways -- from how they attend to the news to how they behave or support political action.
But does the gender of journalists make a difference?
Yes, at least in a recent Journalism and Mass Communication Quarterly study that found "the presence of females as reporters and news executives in the newsroom affected news content related to women, such as coverage of the HPV vaccine." Examining data at the organizational and individual level, the authors found the more "gender-balanced" the newspaper, the more pominent the coverage of the HPV vaccine. Male-dominated newspapers relied more on official sources and less on people (teens, parents, teachers) as compared to more gender-diverse newspapers.
There are limitations to the study. There always are. But it's interesting that the gender makeup of a newsroom could so clearly influence the quantity of stories and how they were reported.
What we don't know, at least from this study, is whether the differences in coverage make a difference in the readers of these newspapers. It would be interesting to know whether, in different communities, there were fewer or more teenage girls getting vaccinated depending on such differences in coverage. When you extend data that far, things tend to unravel, but I'm guessing you might be able to tease out some significant behavioral effects.
Full cite: Teresa Correa and Dustin Harp (2011). Women matter in newsrooms: How power and critical mass relate to coverage of the HPV vaccine. Journalism and Mass Communication Quarterly, 88 (2), 801-319.
But does the gender of journalists make a difference?
Yes, at least in a recent Journalism and Mass Communication Quarterly study that found "the presence of females as reporters and news executives in the newsroom affected news content related to women, such as coverage of the HPV vaccine." Examining data at the organizational and individual level, the authors found the more "gender-balanced" the newspaper, the more pominent the coverage of the HPV vaccine. Male-dominated newspapers relied more on official sources and less on people (teens, parents, teachers) as compared to more gender-diverse newspapers.
There are limitations to the study. There always are. But it's interesting that the gender makeup of a newsroom could so clearly influence the quantity of stories and how they were reported.
What we don't know, at least from this study, is whether the differences in coverage make a difference in the readers of these newspapers. It would be interesting to know whether, in different communities, there were fewer or more teenage girls getting vaccinated depending on such differences in coverage. When you extend data that far, things tend to unravel, but I'm guessing you might be able to tease out some significant behavioral effects.
Full cite: Teresa Correa and Dustin Harp (2011). Women matter in newsrooms: How power and critical mass relate to coverage of the HPV vaccine. Journalism and Mass Communication Quarterly, 88 (2), 801-319.
Trick Questions
I caught a bit of video this morning of Newt Gingrich holding up a Newt 2012 t-shirt. He was set up, and nicely. The ABC reporter got him to hold up the shirt, which he gladly did for the camera, probably figuring it for a harmless puff pic. Gingrich is all about American jobs. Turns out, though, the shirt was made not in the U.S. -- but in El Salvador.
Newt, you've been punk'd.
This is nothing new. The traditional journalistic punking involves milk.
You may not know the routine, but reporters used to follow presidential candidates around as they try to portray themselves as caring about the common man. And then you ask him or her: "So, what's a gallon of milk cost?"
They struggle for an answer. They've been punk'd.
There more here than just catching a high rolling politico looking like a fool. They can do this by themselves quite well (remember Michael Dukakis doing the bobble-head doll routine in a tank, or John Kerry in hunting gear?). And then there was President George H. W. Bush being "amazed" by a grocery store scanner, something Americans see every day. Turns out that story is false, but it fit the narrative of a well-heeled Bush who didn't really understand Americans in a time of economic downturn.
My point?
In what people know, perception is powerful. We tend to see what we want to see and believe what we want to believe, but a powerful narrative that paints a candidate in a certain way, such as hypocritical -- as in the Gingrich t-shirt -- or as historically challenged -- as in Michelle Bachmann or Sarah Palin, has lasting impact on the public mind. And they are hard to reverse. Chevy Chase's biting imitation of Gerald Ford on Saturday Night Live may not have lost Ford the 1976 presidential race, but it certainly didn't help.
Research shows we tend to vote for presidential candidates based on two major themes: competence, and character. Other factors in how a race is framed, such as war or the economy, also play a huge role. But being perceived as a hypocrite or fool, those also can influence how people think about a candidate, and making every subsequent slip-up, even the most minor, seem huge by comparison.
Newt, you've been punk'd.
"I'll have to ask the folks who ordered this," Gingrich responded. "I don't order it and I don't do it." Campaign spokesperson Michelle Selesky said "That was a rush order made by some of the volunteers." Selesky noted the print work on the shirts was done in Atlanta.Okay, so Atlanta is kinda like America. So that's something.
This is nothing new. The traditional journalistic punking involves milk.
You may not know the routine, but reporters used to follow presidential candidates around as they try to portray themselves as caring about the common man. And then you ask him or her: "So, what's a gallon of milk cost?"
They struggle for an answer. They've been punk'd.
There more here than just catching a high rolling politico looking like a fool. They can do this by themselves quite well (remember Michael Dukakis doing the bobble-head doll routine in a tank, or John Kerry in hunting gear?). And then there was President George H. W. Bush being "amazed" by a grocery store scanner, something Americans see every day. Turns out that story is false, but it fit the narrative of a well-heeled Bush who didn't really understand Americans in a time of economic downturn.
My point?
In what people know, perception is powerful. We tend to see what we want to see and believe what we want to believe, but a powerful narrative that paints a candidate in a certain way, such as hypocritical -- as in the Gingrich t-shirt -- or as historically challenged -- as in Michelle Bachmann or Sarah Palin, has lasting impact on the public mind. And they are hard to reverse. Chevy Chase's biting imitation of Gerald Ford on Saturday Night Live may not have lost Ford the 1976 presidential race, but it certainly didn't help.
Research shows we tend to vote for presidential candidates based on two major themes: competence, and character. Other factors in how a race is framed, such as war or the economy, also play a huge role. But being perceived as a hypocrite or fool, those also can influence how people think about a candidate, and making every subsequent slip-up, even the most minor, seem huge by comparison.
Wednesday, July 27, 2011
Social Media as Public Opinion
I wrote Monday and Tuesday about the promise and problems of using Twitter as a measure of public opinion. See those posts for a semi-detailed discussion of why it works and why it doesn't work, especially when it comes to the dangers for journalists tempted to use social media as a gauge of what people think.
But the politicians, they're already there.
According to a new study (full pdf here), nearly 64 percent of congressional offices consider Facebook and 42 percent consider Twitter as "somewhat important" or "very important" measures of public opinion. That's the soundbite, but like all soundbites it's misleading. Let's dig a bit deeper.
So congressional staffers, and journalists, need to take care in how they interpret such data, even as we become more sophisticated in our methods of analyzing thousands and even millions of tweets.
But the politicians, they're already there.
According to a new study (full pdf here), nearly 64 percent of congressional offices consider Facebook and 42 percent consider Twitter as "somewhat important" or "very important" measures of public opinion. That's the soundbite, but like all soundbites it's misleading. Let's dig a bit deeper.
- While 64 percent think Facebook is "somewhat" or "very" important, only 8 percent call it "very important" as a tool for "understanding constituents' views and opinions." Only 8 percent. The "somewhat" is a catch-all category, one people fall into almost by default. Also, I cannot easily find what the other response alternatives were, though I suspect they were "somewhat unimportant" and "not at all important." That would be standard.
- And while 42 percent think Twitter is "somewhat" or "very" important, only 4 percent think it is "very important" as a gauge.
- By comparison, 13 percent of respondents thought "paper surveys/polls" were very important and another 55 percent thought traditional polls were "somewhat important."
- Even so questionable a measure of public opinion as online polls received higher marks than social media. Seven percent said online polls (slops) were very important, and 47 percent said they were somewhat important.
So congressional staffers, and journalists, need to take care in how they interpret such data, even as we become more sophisticated in our methods of analyzing thousands and even millions of tweets.
Tuesday, July 26, 2011
Twitter II -- Is it Public Opinion?
I wrote yesterday on whether Twitter qualifies as public opinion. I opened with an attack on the notion, one based on the methodological challenges of using the micro-blog as a measure of opinion. I then ended by suggesting that the earlier me was wrong, that Twitter may better resemble our classic understanding of the concept of public opinion.
Let me take a stab at following up on my defense of Twitter. At the end, I'll try to extend this from a philosophical concern to a more practical concern for journalists.
Before the existence of sophisticated polling, early thinkers had a very different understanding of public opinion. Its roots can be found in the coffee houses of England and the salons of France. In an 1820 letter, Sir Robert Peel complained about "that great compound of folly, weakness, prejudice, wrong feeling, right feeling, obstinacy and newspaper paragraphs, which is called public opinion."
Sounds like Twitter to me.
By this, I mean that early concepts of public opinion were based on the idea of communication. While I'm a poll guy and use survey data extensively, public opinion is more than merely what public opinion polls measure. Public opinion is an 18th Century invention, and at its heart is public discussion. It is, to borrow a phrase, an "organic sociological process" that takes part in the "public sphere."
As John Durham Peters writes in an excellent chapter that sums up much of this early thought, "in reading the newspapers, the public reads about itself, and thus finds ways to come into existence."
Take out newspapers, put in Twitter. It's a nice fit. After all, in Twitter people can talk back and forth, at "influentials" and others who fill the twittersphere. From this, theoretically at least, public opinion is a changing, ephemeral, difficult to grasp. And hard to measure. Polls are much easier. And thus, public opinion becomes, not what it was intended to be, but what became easier to measure.
Polls matter. Surveys are useful. Uncanny in predictive accuracy for elections, excellent as snapshots of what people think about a question asked of them at a specific time, they are filled with usefulness in their generalizability. I love 'em. Can't live without 'em. But as measures of the classical sense of public opinion, they fall desperately short.
Can Twitter be any better?
Snapshot counting of positive versus negative tweets falls into some of the same traps that polls do. All the million-tweet analyses cranked out by computer scientists do little to address what we really mean by public opinion. It's a difficult concept. As V. O. Key wrote: "to speak with precision about public opinion is a task not unlike coming to grips with the Holy Ghost."
To work, this will require significantly more sophisticated analysis techniques to tease out not the snapshot, but the tendrils of communication and change that occur as a topic or topics bounce across the twittersphere.
It's a methodological nightmare. But it's probably a better sense of what we truly mean by public opinion.
So, what's this mean for those in journalism?
Very soon we (the royal journalistic we) will face folks trying to sell us Twitter counting as an alternative to polls. Surveys are expensive, counting tweets -- less so. I suspect by 2012 we'll see some of the major players, the CNNs and New York Times of the world -- messing around with tweet counting either done in-house or, more likely, through consultants. As I discussed in detail in the previous day's post, doing so is full of problems. Yes, cheaper. And yes, above I argue it more closely resembles our classical understanding of what public opinion means. But from a practical standpoint, we simply are not there yet. My warning is this: journalists, be very very careful in using such data, if you use it at all, except in tandem with traditional polling data. Do NOT rely merely on tweets. We aren't there. Yet.
Let me take a stab at following up on my defense of Twitter. At the end, I'll try to extend this from a philosophical concern to a more practical concern for journalists.
Before the existence of sophisticated polling, early thinkers had a very different understanding of public opinion. Its roots can be found in the coffee houses of England and the salons of France. In an 1820 letter, Sir Robert Peel complained about "that great compound of folly, weakness, prejudice, wrong feeling, right feeling, obstinacy and newspaper paragraphs, which is called public opinion."
Sounds like Twitter to me.
By this, I mean that early concepts of public opinion were based on the idea of communication. While I'm a poll guy and use survey data extensively, public opinion is more than merely what public opinion polls measure. Public opinion is an 18th Century invention, and at its heart is public discussion. It is, to borrow a phrase, an "organic sociological process" that takes part in the "public sphere."
As John Durham Peters writes in an excellent chapter that sums up much of this early thought, "in reading the newspapers, the public reads about itself, and thus finds ways to come into existence."
Take out newspapers, put in Twitter. It's a nice fit. After all, in Twitter people can talk back and forth, at "influentials" and others who fill the twittersphere. From this, theoretically at least, public opinion is a changing, ephemeral, difficult to grasp. And hard to measure. Polls are much easier. And thus, public opinion becomes, not what it was intended to be, but what became easier to measure.
Polls matter. Surveys are useful. Uncanny in predictive accuracy for elections, excellent as snapshots of what people think about a question asked of them at a specific time, they are filled with usefulness in their generalizability. I love 'em. Can't live without 'em. But as measures of the classical sense of public opinion, they fall desperately short.
Can Twitter be any better?
Snapshot counting of positive versus negative tweets falls into some of the same traps that polls do. All the million-tweet analyses cranked out by computer scientists do little to address what we really mean by public opinion. It's a difficult concept. As V. O. Key wrote: "to speak with precision about public opinion is a task not unlike coming to grips with the Holy Ghost."
To work, this will require significantly more sophisticated analysis techniques to tease out not the snapshot, but the tendrils of communication and change that occur as a topic or topics bounce across the twittersphere.
It's a methodological nightmare. But it's probably a better sense of what we truly mean by public opinion.
So, what's this mean for those in journalism?
Very soon we (the royal journalistic we) will face folks trying to sell us Twitter counting as an alternative to polls. Surveys are expensive, counting tweets -- less so. I suspect by 2012 we'll see some of the major players, the CNNs and New York Times of the world -- messing around with tweet counting either done in-house or, more likely, through consultants. As I discussed in detail in the previous day's post, doing so is full of problems. Yes, cheaper. And yes, above I argue it more closely resembles our classical understanding of what public opinion means. But from a practical standpoint, we simply are not there yet. My warning is this: journalists, be very very careful in using such data, if you use it at all, except in tandem with traditional polling data. Do NOT rely merely on tweets. We aren't there. Yet.
Monday, July 25, 2011
Can Twitter Measure Public Opinion?
There's been a lot of interest lately in whether Twitter can be used successfully as a measure of public opinion. Certainly it's cheaper than a real survey, assuming you have the expertise to gather and analyze thousands, or hundreds of thousands, and yes even millions of tweets, as some scholars have done.
This matters not only for scholars, but journalists as well.
So far the results on Twitter are mixed. A German study found an analysis of tweets to be as good as polls in predicting a multi-party election. A million-tweet analysis found Twitter to be both good and not so good, depending on what you were analyzing, in a comparison with traditional U.S. polling data.
There are enormous difficulties in arriving at a suitable, valid, and reliable automated measure of sentiment from tweets. So often what we post is tongue-in-cheek, negative when we mean positive (or vice versa).
In other words, a million tweets may seem to have enormous predictive power. But size isn't everything.
Setting aside the problems with measurement error in any program that analyzes tweets as either positive or negative, let's not forget that while a sample of 100,000 or even a million seems impressive compared to traditional surveys of 1,000 respondents, we've seen big mistakes arise when we confuse the size of our sample with the quality of our sample. In other words, size doesn't always matter.
And for this, I remind you of the infamous 1936 Literary Digest poll.
You can read about it here, or here or a host of other places. The magazine was the Time of its period. The short version: this survey of over a million people predicted Alf Landon, a Republican, would win the 1936 presidential election. None of us remember studying the Landon administration in U.S. history class because, as you may have guessed, there never was a President Landon. Roosevelt won.
The magazine, using a method that had worked for it in the past, sent out over 10 million ballots, got over a million back, and called the election so very wrong. Why? Because a big sample is not the same as a good sample. This was 1936. It was the Depression. The magazine relied on its own subscription rolls, on those who owned cars, who had telephones, and host of other sources that all skewed toward folks during those terrible economic times who could afford such things -- people who tended to vote Republican. Thus, the mag predicted a Landon victory.
Size, then, doesn't always matter.
Twitter is like this. While I like Twitter and use it often, relatively few Americans make use of it, fewer still actively post to the micro-blog, and even if you can glean a million tweets on an election or public issue, the resulting sample is deeply skewed toward the geeky and those who like technology or just enjoy sharing with the world their daily wisdom. That's a lot of sampling error. And we're not even getting into the difficulty of having a computer program decide what's positive or negative in a 140-character-or-less posting.
There are a lot of interesting uses for Twitter -- how people respond in real-time to a television program or sporting event or even breaking news. As a measure, by itself, of public opinion? No. Unless, of course, you learned nothing from the Literary Digest. Journalists and scholars alike need to keep this in mind when using Twitter as the source of all knowledge, at least when it comes to evaluating what people think.
And now for my defense of Twitter -- as a measure of public opinion.
Yes, you read that right. While this deserves a more in-depth analysis than I can do here, Twitter in many ways resembles what we classically think of as public opinion. Since the 1940s or so, as polling technology grew more sophisticated, we've tended to define public opinion as that which public opinion polls measure. Circular, to be sure, a definition driven by polling methodology and not by sound theoretical reasoning.
The classical understanding of public opinion is more nuanced. It includes aspects of communication, something missing in our modern snapshot definition. In other words, people in the coffee houses of the 1700s discussing the issues of the day, a fluid understanding of opinion as it moved and changed due to not only what people thought but what they said, and how it moved and shifted. Thus -- Twitter. It fits this classical, versus the modern, understanding of public opinion.
As I said, my argument above deserves more time and all the usual academic citations we love to layer on our work, but the thesis is a simple one -- Twitter is an imperfect measure of public opinion, as we define it in modern times, but it may very well be the perfect measure of public opinion as we classically understand the concept: messy, fluid, and full of communication.
As journalists become more sophisticated in evaluating the Twitterverse for more than mere anecdotal evidence, they need to keep in mind the limitations of even a million tweets.
This matters not only for scholars, but journalists as well.
So far the results on Twitter are mixed. A German study found an analysis of tweets to be as good as polls in predicting a multi-party election. A million-tweet analysis found Twitter to be both good and not so good, depending on what you were analyzing, in a comparison with traditional U.S. polling data.
There are enormous difficulties in arriving at a suitable, valid, and reliable automated measure of sentiment from tweets. So often what we post is tongue-in-cheek, negative when we mean positive (or vice versa).
In other words, a million tweets may seem to have enormous predictive power. But size isn't everything.
Setting aside the problems with measurement error in any program that analyzes tweets as either positive or negative, let's not forget that while a sample of 100,000 or even a million seems impressive compared to traditional surveys of 1,000 respondents, we've seen big mistakes arise when we confuse the size of our sample with the quality of our sample. In other words, size doesn't always matter.
And for this, I remind you of the infamous 1936 Literary Digest poll.
You can read about it here, or here or a host of other places. The magazine was the Time of its period. The short version: this survey of over a million people predicted Alf Landon, a Republican, would win the 1936 presidential election. None of us remember studying the Landon administration in U.S. history class because, as you may have guessed, there never was a President Landon. Roosevelt won.
The magazine, using a method that had worked for it in the past, sent out over 10 million ballots, got over a million back, and called the election so very wrong. Why? Because a big sample is not the same as a good sample. This was 1936. It was the Depression. The magazine relied on its own subscription rolls, on those who owned cars, who had telephones, and host of other sources that all skewed toward folks during those terrible economic times who could afford such things -- people who tended to vote Republican. Thus, the mag predicted a Landon victory.
Size, then, doesn't always matter.
Twitter is like this. While I like Twitter and use it often, relatively few Americans make use of it, fewer still actively post to the micro-blog, and even if you can glean a million tweets on an election or public issue, the resulting sample is deeply skewed toward the geeky and those who like technology or just enjoy sharing with the world their daily wisdom. That's a lot of sampling error. And we're not even getting into the difficulty of having a computer program decide what's positive or negative in a 140-character-or-less posting.
There are a lot of interesting uses for Twitter -- how people respond in real-time to a television program or sporting event or even breaking news. As a measure, by itself, of public opinion? No. Unless, of course, you learned nothing from the Literary Digest. Journalists and scholars alike need to keep this in mind when using Twitter as the source of all knowledge, at least when it comes to evaluating what people think.
And now for my defense of Twitter -- as a measure of public opinion.
Yes, you read that right. While this deserves a more in-depth analysis than I can do here, Twitter in many ways resembles what we classically think of as public opinion. Since the 1940s or so, as polling technology grew more sophisticated, we've tended to define public opinion as that which public opinion polls measure. Circular, to be sure, a definition driven by polling methodology and not by sound theoretical reasoning.
The classical understanding of public opinion is more nuanced. It includes aspects of communication, something missing in our modern snapshot definition. In other words, people in the coffee houses of the 1700s discussing the issues of the day, a fluid understanding of opinion as it moved and changed due to not only what people thought but what they said, and how it moved and shifted. Thus -- Twitter. It fits this classical, versus the modern, understanding of public opinion.
As I said, my argument above deserves more time and all the usual academic citations we love to layer on our work, but the thesis is a simple one -- Twitter is an imperfect measure of public opinion, as we define it in modern times, but it may very well be the perfect measure of public opinion as we classically understand the concept: messy, fluid, and full of communication.
As journalists become more sophisticated in evaluating the Twitterverse for more than mere anecdotal evidence, they need to keep in mind the limitations of even a million tweets.
Friday, July 22, 2011
God's Approval Ratings
Only 52 percent in a recent poll approve of the job God is doing. I feel a heckuva lot better about myself today, for some strange reason.
I heard about the PPP poll via The Atlantic Wire. If you download the actual report, you'll find that while 52 percent approve of God's job, only 9 percent disapprove and 40 percent -- well, those folks aren't sure. What's fun is not only the results of questions immediately below this but the results later in my post which break down the crosstabs, such as who gives God the highest marks -- men versus women, by age, by political partisanship. Read all the way through to get it. First, some of the other questions:
Further down in the report you can have a little fun with the crosstabs. Looking only at God's approval ratings, we find:
I heard about the PPP poll via The Atlantic Wire. If you download the actual report, you'll find that while 52 percent approve of God's job, only 9 percent disapprove and 40 percent -- well, those folks aren't sure. What's fun is not only the results of questions immediately below this but the results later in my post which break down the crosstabs, such as who gives God the highest marks -- men versus women, by age, by political partisanship. Read all the way through to get it. First, some of the other questions:
- If God exists, do you approve or disapprove of
its handling of natural disasters? 50 percent approve, 13 percent disapprove, 37 percent unsure.
- If God exists, do you approve or disapprove of
its handling of animals? 56 percent approve, 11 percent disapprove, 33 percent unsure.
- If God exists, do you approve or disapprove of
its handling of creating the universe? 71 percent approve, 5 percent disapprove, 24 percent unsure.
Further down in the report you can have a little fun with the crosstabs. Looking only at God's approval ratings, we find:
- Among the "very liberal," 54 percent give God good job ratings. Among the "very conservative," 61 percent do. Not a huge difference.
- Women gave God higher marks than men, 55 percent to 48 percent. Again, kinda close.
- Republicans gave higher marks than Democrats, 55 percent to 50 percent. Ditto on close.
- Blacks gave higher approval (72 percent) than Hispanics (53) and whites (47). That's a big difference, but the number of blacks in the survey may be so small as to raise methodological issues. Still, I'm not surprised by the difference.
- And finally, to bury a lede, the older you were, the lower God's approval ratings. Among those 18-29, God got 67 percent aproval. The other age categories and approval ratings in parentheses were: 30-45 (61), 46-65 (50), older than 65 (40). I suppose as you get older, you have a little more to bitch about when it comes to God and the job, um, "its" doing.
Subscribe to:
Posts (Atom)