Random blog posts about research in political communication, how people learn or don't learn from the media, why it all matters -- plus other stuff that interests me. It's my blog, after all. I can do what I want.
Tuesday, February 5, 2013
Public Opinion about Public Opinion
Public opinion is nothing more than this,
what people think that other people think.
Okay, Victorian playwright Alfred Austin actually put these words in the mouth of his title character, Prince Lucifer, in his 1891 play.
(yes I've mentioned this before, back in 2008)
What I'm discussing here is part of a larger work I'm fiddling with, the idea that we don't spend enough time studying people's opinions about public opinion -- or, for lack of a better word, meta-opinion. There are theoretical traditions where this is more common (Spiral of Silence comes immediately to mind). These traditions tend to view public opinion from a social control perspective. Two factors come into play here: (1) we're always sensing the climate of opinion around us, and (2) a fear of isolation leads us to go with the majority, or at least not express our minority viewpoints as readily as we might.
Psychology has examined this in detail at the micro, individual level, or small-group level (my favorite, false consensus, but see also pluralistic ignorance for a more macro and contradictory approach).
My point is related to, but separate from, what people think about public opinion polls. In a sense, people view such polls negatively when the results disagree with their own positions, an example of the hostile media effect. It's vital we tease out the difference between how people respond to a poll -- and who published it -- and how they think about public opinion in a broader sense. In the latter, I'm talking about their acceptance (or rejection) of the idea of public opinion, their acceptance that other opinions have merit, their acceptance of whether such opinions should guide policy. How do people define public opinion? And has this definition changed in a world that has moved from mainstream few-to-many media to social media in which we have an almost instantaneous access to, as our friend Lucifer said above, "what other people think?" I'm guessing social media, From Facebook to Twitter, have changed our ideas of what truly is public opinion. And I'm guessing it's become something more narrow, more in tune with our "friends" and "followers."
And it's entirely possible I'm completely wrong.
This is me thinking aloud as I work on a longer, more thorough examination of the topic, hopefully with actual data gathered (or found elsewhere) to fully explore whether people think of public opinion today in a very different way than they used to -- and what that means for public policy, for elections, and ultimately for a healthy democracy.
Wednesday, July 27, 2011
Social Media as Public Opinion
But the politicians, they're already there.
According to a new study (full pdf here), nearly 64 percent of congressional offices consider Facebook and 42 percent consider Twitter as "somewhat important" or "very important" measures of public opinion. That's the soundbite, but like all soundbites it's misleading. Let's dig a bit deeper.
- While 64 percent think Facebook is "somewhat" or "very" important, only 8 percent call it "very important" as a tool for "understanding constituents' views and opinions." Only 8 percent. The "somewhat" is a catch-all category, one people fall into almost by default. Also, I cannot easily find what the other response alternatives were, though I suspect they were "somewhat unimportant" and "not at all important." That would be standard.
- And while 42 percent think Twitter is "somewhat" or "very" important, only 4 percent think it is "very important" as a gauge.
- By comparison, 13 percent of respondents thought "paper surveys/polls" were very important and another 55 percent thought traditional polls were "somewhat important."
- Even so questionable a measure of public opinion as online polls received higher marks than social media. Seven percent said online polls (slops) were very important, and 47 percent said they were somewhat important.
So congressional staffers, and journalists, need to take care in how they interpret such data, even as we become more sophisticated in our methods of analyzing thousands and even millions of tweets.
Monday, July 25, 2011
Can Twitter Measure Public Opinion?
This matters not only for scholars, but journalists as well.
So far the results on Twitter are mixed. A German study found an analysis of tweets to be as good as polls in predicting a multi-party election. A million-tweet analysis found Twitter to be both good and not so good, depending on what you were analyzing, in a comparison with traditional U.S. polling data.
There are enormous difficulties in arriving at a suitable, valid, and reliable automated measure of sentiment from tweets. So often what we post is tongue-in-cheek, negative when we mean positive (or vice versa).
In other words, a million tweets may seem to have enormous predictive power. But size isn't everything.
Setting aside the problems with measurement error in any program that analyzes tweets as either positive or negative, let's not forget that while a sample of 100,000 or even a million seems impressive compared to traditional surveys of 1,000 respondents, we've seen big mistakes arise when we confuse the size of our sample with the quality of our sample. In other words, size doesn't always matter.
And for this, I remind you of the infamous 1936 Literary Digest poll.
You can read about it here, or here or a host of other places. The magazine was the Time of its period. The short version: this survey of over a million people predicted Alf Landon, a Republican, would win the 1936 presidential election. None of us remember studying the Landon administration in U.S. history class because, as you may have guessed, there never was a President Landon. Roosevelt won.
The magazine, using a method that had worked for it in the past, sent out over 10 million ballots, got over a million back, and called the election so very wrong. Why? Because a big sample is not the same as a good sample. This was 1936. It was the Depression. The magazine relied on its own subscription rolls, on those who owned cars, who had telephones, and host of other sources that all skewed toward folks during those terrible economic times who could afford such things -- people who tended to vote Republican. Thus, the mag predicted a Landon victory.
Size, then, doesn't always matter.
Twitter is like this. While I like Twitter and use it often, relatively few Americans make use of it, fewer still actively post to the micro-blog, and even if you can glean a million tweets on an election or public issue, the resulting sample is deeply skewed toward the geeky and those who like technology or just enjoy sharing with the world their daily wisdom. That's a lot of sampling error. And we're not even getting into the difficulty of having a computer program decide what's positive or negative in a 140-character-or-less posting.
There are a lot of interesting uses for Twitter -- how people respond in real-time to a television program or sporting event or even breaking news. As a measure, by itself, of public opinion? No. Unless, of course, you learned nothing from the Literary Digest. Journalists and scholars alike need to keep this in mind when using Twitter as the source of all knowledge, at least when it comes to evaluating what people think.
And now for my defense of Twitter -- as a measure of public opinion.
Yes, you read that right. While this deserves a more in-depth analysis than I can do here, Twitter in many ways resembles what we classically think of as public opinion. Since the 1940s or so, as polling technology grew more sophisticated, we've tended to define public opinion as that which public opinion polls measure. Circular, to be sure, a definition driven by polling methodology and not by sound theoretical reasoning.
The classical understanding of public opinion is more nuanced. It includes aspects of communication, something missing in our modern snapshot definition. In other words, people in the coffee houses of the 1700s discussing the issues of the day, a fluid understanding of opinion as it moved and changed due to not only what people thought but what they said, and how it moved and shifted. Thus -- Twitter. It fits this classical, versus the modern, understanding of public opinion.
As I said, my argument above deserves more time and all the usual academic citations we love to layer on our work, but the thesis is a simple one -- Twitter is an imperfect measure of public opinion, as we define it in modern times, but it may very well be the perfect measure of public opinion as we classically understand the concept: messy, fluid, and full of communication.
As journalists become more sophisticated in evaluating the Twitterverse for more than mere anecdotal evidence, they need to keep in mind the limitations of even a million tweets.
Thursday, May 13, 2010
Who Needs a Survey?
Twitter as Public Opinion
Pollsters aren’t trembling yet, but a study of Twitter posts finds that for some key political and economic questions, tracking the content of microblogs on the Internet is nearly as good as doing a traditional telephone survey.It's a scary proposition and probably flawed, but this story touches on something I've discussed before, how social media do a pretty good job of capturing a snapshot of opinion. We'll need a new word for this. Twitnion? Twitian? Opintiwitin? Okay, they all suck, but not only traditional surveys are in trouble.
By tracking the frequency of positive or negative sentiments about people’s financial well-being—filtering for posts about saving and spending—the Twitter data also reproduced trends in some classic economic indicators such as consumer confidence.It's only a matter of time before someone does this professionally at a level in which they, with fewer costs, position themselves as a viable alternative to serious public opinion research. The bottom line matters, and if they can model this to demonstrate it is as accurate as well-done polls, then we're going to see some interesting changes.
Do I think its a viable alternative? No, not by itself. The lack of generalizability bothers me, but I think social media captures some data polls might miss and will serve a complementary function in helping us understand what people think and know. And here's a different take on the predictive power of social media.
Thursday, October 15, 2009
Polls and Policy
An excellent piece by Andrew Kohut, president of the Pew Research Center, about the role of polls in making policy. Strongly recommended for those interested in public opinion, what people think, and whether it matters. Shamelessly hotlinked to the right, one of the graphics from the article that shows the growth of mentions of polls in news stories.
Monday, September 21, 2009
Public Opinion and Terrorism
Thursday, September 10, 2009
Helping People Decide
The study is Helping Citizens Decide in Referendums (with, of course, a colon and long sentence to create titular colinicity). The dependent variables were likelihood of voting, preference, strength of opinion, and vote consistency. What's that last one mean? It harkens back to some traditional research that examines how people organize their political positions and whether there's a logical (consistent) arrangement of attitudes. In other words, you tend to be liberal on a host of issues, or conservative. Therefore, consistent.
Independent variables? Various flavors of political sophistication or knowledge through something called ICQ -- which stands for Information and Choice Questionnaire. Basically, providing people with info about a specific problem before asking them what they think.
A careful reader will note this sounds like deliberative democracy (see my post here on it). Not quite. No deliberation taking place, at least the way it's meant, but yeah -- kinda sorta the same idea, that informed opinion is different than public opinion.
Informing people in advance before polling them did lead to more consistent opinions, but more so for people with less education and interest. This follows theory, to a point. As the authors note, cognitive processing models such as ELM suggest that people high in political sophistication will be more able and motivated to carefully consider an issue, but here it's the people with lower political sophistication who "profit more."
I suspect some of this has to do with the artificial nature of a lab experiment, but even so it raises some interesting theoretical and methodological questions. We've always known informed opinion differs from generic public opinion. This study takes another step closer to understanding not only how it differs, but the consequences of those differences.
Wednesday, May 27, 2009
Rejection
In other off-topic news, working on my Public Opinion seminar syllabus. Scouring the net for good stuff and trying to plot out my day-to-day discussion topics. It's a summer seminar that meets too many days in a week for too many hours in a day.
In yet more off-topic news, in about an hour it's Manchester United vs. Barcelona. Go Barca!
And in a final bit of off-topic, off-blog news, my data analysis on a new manuscript absolutely sucks. Not sure what to do -- toss the zillion hours of work I've put into it, or try to salvage something? It's on a topic I love (wishful thinking and predictive accuracy), but the data simply are not cooperating. Not sure what to do.
Tuesday, May 5, 2009
Hypo Time
I've always been interested in people's perception of opinion. We tend to project our own opinions on the public at large. In other words, we tend to think others think the same way we do, what the social scientists sometimes call projection but based also on the fact we tend to hang out with people much like ourselves, thus skewing our perception of opinion for a broader public.
I've been wondering whether social networking, at being tied in to the mundane and profound thoughts of so many friends and quasi-friends, can influence our perceptions of opinion. Thus, I'm gonna toss out a few possible hypotheses over the next day or two.
Hypothesis 1: The greater the use of social networks (Twitter, Facebook, etc.), the greater the likelihood one will perceive other people agreeing with your own opinion.
Huh? Wouldn't more information make you more accurate, not less so? I don't think so. First off, we tend to link to people like ourselves, even if they're only barely friends, so that should increase the likelihood we'll think the world agrees with us because that's all the opinions we see expressed. Our narrow little worlds make us a bit more likely to overestimate how many people agree with us, and I think social networking only accelerates, not moderates, this effect. I'd love to see this tested.
Hypothesis 2: Social networking posts include only poll information that favors a poster's point of view, not poll results that run counter to a poster's ideological or partisan position.This one kinda makes more sense, or at least is not counter-intuitive. There are exceptions, the people who put on a Facebook news feed some poll that they think shows how dumb people are, but overall I expect to see confirmatory information, which in turn would lead to Hypothesis 1 being supported.
Hypothesis 3: Attempts to shift public opinion via social networking will ultimately fail.Yeah, a crappily-worded hypothesis, but I'm short of time and didn't want to get all PhDweeb on it, but the above hypo in its various forms has significant meaning for people in politics, public relations, and even journalism. Basically I'm arguing, and need to explore this further, that social networking, even at its most viral, does little to nudge public opinion. It might move perceptions of opinion.
To me, Facebook is the big guy in this test. Twitter, less so. Too ephemeral, too silly, too confined. But Facebook with its updates and news feeds, that's a place ripe for examination by people interested in what moves public opinion, or moves perceptions of that opinion (to me a more interesting theoretical question).
Sunday, May 3, 2009
The Borg and Social Networking
I've mentioned it before but there's this great definition of public opinion:Public opinion is no more than this,
what people think other people think.
Monday, October 13, 2008
Who Will Win
Respondents were asked who is going to win. I then combined that with who they supported. Looking at presidential elections from 1968 to 2004, you see how much preference guides prediction.
Republicans: Out of 6,960 surveyed, 81 percent predicted their preferred candidate would win.
Democrats: Out of 6,664 surveyed, 64 percent predicted their preferred candidate would win.
The amount of wishful thinking from year to year is fairly stable, but predictive accuracy (getting it right) ranges wildly from election to election. This is important. When elections are close (like 2000), personal preference and wishful thinking gets in the way and people are less accurate in predicting the outcome.
When it's a runaway election, like 1972, then accuracy improves. You'd think media exposure would improve accuracy, if for no other reason than reading or watching the news would expose people to public opinion polls and the viewpoints of others, but it does damn little to dampen the preference -> prediction bias.
Friday, October 10, 2008
Addendum to Below
A 61 percent majority of voters believes Obama is going to win the election — more than three times as many as believe McCain will (18 percent). A month ago it was evenly divided: 41 percent Obama and 40 percent McCain (Sept. 8-9). This summer, voters were more likely to say Obama would win: 51 percent Obama and 27 percent McCain (July 22-23).
I wish I could remember where I once read that perception of who is going to win is a better predictor of an election outcome than merely seeing what a poll tells you on who people prefer. Let me say that another way. Asking who you are for ends with Obama getting 45 percent of the vote to McCain's 39 percent (according to this very poll). But asking who is going to win gets Obama with 61 percent.
I vaguely recall reading once, as a grad student perhaps, that asking who you think is going to win is a better predictor than just counting up those for or against the candidates. This question, neat as it is, suggests a sense of inevitability among even McCain supporters. That gets right at the power of perception.
Some other day I'll blog about a related matter -- wishful thinking. Again, this is about perception and I've done some work in the area, but basically we tend to wishfully think and see our own preferred candidate as likely to win. Fun stuff, as of course all mass comm research is -- if you're into the whole research thing.
Public Opinion
Public opinion is no more than this,
what people think other people think.
- Alfred Austin, 1891
in Prince Lucifer
I've always been partial to the above definition of public opinion -- and to Victorian-era plays named after the devil, but that's a different story. The play itself sucks. I read it in grad school when I came across this definition. Kept the definition, forgot the play.
Scholars in various fields have nibbled around the idea that the perception of opinion matters. Read spiral of silence, or false consensus, or pluralistic ignorance, or bandwagon effect, or third-person effects and you'll find the roots lie in what people think other people think. The consequences depend on your theoretical approach. In spiral of silence, for example, the perception that one is in a distinct minority on some issue creates a fear of isolation and results in people being a little less likely to speak out on their position. The "spiral" in this case is the slow disappearance of minority viewpoints.
Okay, we've had our dose of Theory for the Day. So what?
Perception matters. If people sense a shift in the opinion climate, some will follow and some will simply grow more quiet, some will jump on the bandwagon and some will root louder for the underdog. It's interesting that as the presidential campaign gets closer to ending (thank God!) and Obama eases away from McCain, we've seen some ugliness on both sides. The perception of opinion going against your way can lead to people staring at a TV screen in disbelief. The poster child for this is the guy at the McCain rally the other day who begged McCain to go negative, that he can't believe America is following Obama. It's fascinating stuff.
I should also mention polls here and the discrediting of polls when they go against our side. "They never asked me or anyone I know" is often heard from people who don't like what the polls tell us about the opinion climate. Since we tend to hang out with people like ourselves, we create these artificial bubbles of like-minded opinion. When polls say otherwise, they just don't make sense. My guess is people for McCain right now are scratching their heads, wondering how this is happening (it's the economy, stupid). Supporters of Kerry and Gore did the same thing in 2004 and 2000.
It's human nature. Our perception of opinion is biased by those we hang out with, those we talk to, those we see in our neighborhoods or at work. When a broader measure of opinion disagrees with that, we're stumped. And that screws with our perceptions of opinion. Some people correct their own estimations of opinion, some argue with the source of the disagreement.
Again, cool stuff.
Tuesday, August 5, 2008
The Power of Names
Others ask the names. So, does it make a difference? Let's take a look at two polls, asked more or less at the same time.
The Pew Center asked:
"Now, suppose the 2008 presidential election were being held TODAY. If you had to choose between [see below], who would you vote for?" If other/unsure: "As of TODAY, do you LEAN more to [see below]?"
Only two names were provided in random rotation: McCain and Obama. The number of unsure/other is 12%.
The Research 2000 for news media client poll asked:
"If the election for president were held today, who would you vote for if the choices were between Barack Obama, the Democrat, John McCain, the Republican, Bob Barr, the Libertarian, or Ralph Nader, an independent?"
In this case we're presented the names of Barr and Nader (but not Cynthia McKinney, a Georgia girl running for the Greens). The other/unsure? 5%.
My point? Offering more alternatives may influence the numbers, especially if you are a minor , i.e., third-party, candidate. Can't get no public opinion love if you ain't even mentioned. Plus, if you don't mention other candidates, you artificially inflate the unsure votes.
Monday, July 9, 2007
Cult of the Amateur and other Stuff
I admit some sympathy for this position, despite the books flaws, which have been discussed in detail elsewhere. This is where I should insert lots of links to the pros and cons of the book. Google 'em yourself. I'm busy.
So, what's this to do with What People Know? With political knowledge?
Everything.
We are coming to the tension between an elite versus mass approach to news, knowledge, entertainment, and politics. It's Lippmann vs. Dewey, or Plato versus Aristotle. From a What People Know perspective, it comes down to where are people best informed: bloggers and YouTube, or the mainstream professional media? As the audience for the MSM erodes, are people better informed? Does it even matter that people are generally uninformed, or at least not factually informed?
This gets all tied up in the Wisdom of the Crowd as compared to elites as leaders of opinion, and the question of emotion versus factual knowledge. The end result? I imagine we are becoming more "affective" in our political orientations, less willing to spend time learning what's going on, more distracted by the thousand new voices or easier ways to spend our time, more likely to follow the more emotional appeal, and generally less capable of participating effectively in a democracy. And that's bad news.
Friday, June 22, 2007
One Way to Get Better Knowledge
- Give 'em more time to answer the question
- Pay 'em for their trouble.
No, really. They work.
The resounding doh! you hear out there comes from any real people who read this blog is understandable, but this actually matters. It tells us something about the role of motivation, that people often can come up with correct answers about political knowledge if properly motivated. A random telephone call from a stranger asking if you can name your congresspersonthing is hardly a motivating moment.
The lesson? People are smarter about political knowledge than we think. That sounds good, but we (i.e., scholars) think very little of the public's knowledge, so there's plenty of room for improvement (in perception, that is).
