At Least He Called It Unscienific

Is your smartphone making you dumber? A question asked and answered by magazine articles a few years ago, but one raised by a local TV news reporter's story as if it's fresh.

Check out the questions, which seem more or less reasonable. Here's the real issue -- what the hell do you compare the answers to? Logic would say the same questions asked before the advent of smartphones. After all, if your point is they make us dumber, then you need a comparison group. I know, I know. Logic and TV news, not the same thing. Another way to compare this would be between people who do and do not have smartphones, but that would be so confounded by age, and maybe income, as to be useless without sophisticated statistical manipulations.

By the way, there are several here I can't answer. Clearly my smartphone has made me dumber. Then again, I doubt I could ever ask them, even before I had a smartphone -- so toss that hypothesis out the window, TV guy.

At least he called it an "unscientific survey." Thank you for that.

Two Kinds of Knowledge

Here's an interesting article about two kinds of knowledge, that of science and that of indigenous peoples.  "Traditional knowledge" should partner with science, it's argued -- and let's face it, it is difficult to argue with that as long as we don't go too far afield into the wacky.  But here's a really good point:

“Indigenous peoples know what kind of information they need to make the right decisions, but it is difficult for them to access the information. Scientists, on the other hand, have a lot of information, but do not know what the indigenous peoples need. So we need you to tell us.”

I am writing less this summer.  Why?  Because it's summer, dammit.

Lesson Time: Knowledge

I blog here about recent research or new angles to what people know about politics and public affairs or how they learn from the media.  Today it's lesson time, a quick primer on knowledge.  Cognitive scientists break knowledge into a number of categories.  I know, just typing cognitive scientists makes me want to take a nap, but hang in there and let's look at the different types of memory:

Episodic Memory -- events or personal information about ourselves, such as our birthday or hometown.  When we talk about memory in everyday life, this is the one we most often mean.
Semantic Memory -- how we store words and facts.   The how here is important.
Declarative Memory -- the system for retrieving the "whats" of our knowledge, such as who is our congressional representative.  Often what we measure in political knowledge tests.
Procedural Memory -- the system for retrieving the "hows" of our memory, such as how a bill becomes law.  It's really hard to study this system.

So why do these matter?  Because there is a long (in milliseconds) process involved here in retrieving stored information, and a lot can get in the way.  Imagine the typical text news story in inverted pyramid in which the most important, or more recent, information is presented first.  On the Louisiana oil spill, the lede today is about BP trying to divert some oil with another dome, but if you haven't been following the oil spill at all, the lede won't make a lot of sense to you.  There are no semantic details to draw from to make sense of a "most important or recent first" approach to newstelling.  This is where we -- the royal journalistic we -- fail to engage some readers, especially young ones with little background info.  And this is why television news often works best for people who don't know a lot about the news, because in part the way its structured, in part the way video is used to "ease" folks into the news.

Print, in other words, is for those who already have a clue.

This raises all kinds of interesting problems for those who provide print news.  But that's another post for another day. 

Hits of the Day

A few tidbits from the Net on what people know (or think they know).

• "If more Americans knew what was included in the healthcare reform law, more may support it, a U.S. survey indicates." Story here.
• "Middle-schoolers who are forbidden to watch R-rated movies are less likely to start drinking than peers whose parents are more lenient about such films, new research on 2,406 children shows." Story here.
• "According to a survey from Drug and Therapeutic Bulletin, many doctors are not very well informed on the subject of herbal medicines, and many don't want to be." Story here. 

I am the aggregator.

Okay, not a complete aggregator, and I have a system that scours the web for stuff like this, most of which I toss because I either don't understand it or it doesn't fit.  Mostly because I don't understand it.  On the three above, a few words.  The first, about health care (why one word?), this is not surprising.  People react to a phrase or a global term and Obamacare and all its scary alternatives are worse than when you break the law down to its basics.  Then people nod and say, "yeah, that's okay."  On the second, this one is kinda cool.  Who the hell lets a middle-schooler watch an R movie in the first place?  Idiots, who also don't teach their kids about drinking.  The final one I'm torn about, because honestly I don't want my doctor thinking about herbal medicines.  I want the good stuff.  The strong stuff.  Stuff from major pharmaceutical companies that feed the profit engine of the U.S.  Or at least were tested.

All are good examples of either knowledge, perceived knowledge, or a lack of knowledge.  Or in the case of R-rated movies, bad parenting. 

Changing Journalism -- and Knowledge

Leonard Witt, who does cool stuff down the road from me on sustainable journalism, is conducting a series of interviews about the future of journalism.  So there's this interview with Jay Rosen, esteemed NYU j-prof and Master of the Universe when it comes to changing media.  The text is here or you can listen to the interview here.

Below, the part that touches on this blog's theme:

Witt:  So do you think the people will be better informed than they are now? The public and the public’s fear,  or less informed or you just don’t have a clue?
Rosen: I think it’s going to be better.
Witt: Why is that?
Rosen: Because we don’t have to depend on  a single elite for our information.
Witt: So do you think that’s already happening? Or do you think….
Rosen: Yeah it is already happening.

My gut reaction?  This is absolute crap, the idea that people will be better informed because they no longer "depend on a single elite" for information.  But it's possible my gut reaction comes from bad sushi or my previous life as a journalist or my years of studying and doing research on how people learn from the media.

In fairness, the bulk of this interview is not about how and what people learn from the news.  Neither guy is an expert on the topic.  It's more about where journalism is going and what it all means, and neither dislike the old mainstream media, they're just pointing out the obvious changes driven in a large part by technology.  So let's give 'em both some leeway here.

And then again, let's not, otherwise I'd have nothing to write about.

First, there is some evidence that political knowledge has been decreasing, not increasing, as all these new media non-elite sources come to fruition. Cause-and-effect?  Too early to say.  A lot of political knowledge comes from incidental learning, the inadvertent exposure to information, mostly from TV for people with little interest in public affairs.  In this brave new world, with so many choices, these are folks most likely to go elsewhere for entertainment rather than consume -- even if half-heartedly and accidentally -- the news.

So our brave new media world in which a million information sources bloom, for a large chunk of society that might as well be happening on the Moon.

Now, if we're talking about the chattering class, the political junky, that sliver of Americana who find politics not only interesting but as necessary as oxygen, yeah I think you've got something here.  But this is a mistake these folks often make, of generalizing from their own interests or those they hang out with over lattes.  A metro-centric viewpoint.  My own reading of the vast literature of how people learn from the media and what they do with that knowledge suggests the more you fragment the media world, the less learning that will occur, and an awful lot of that learning will be dysfunctional and misinformed.

People will feel they are informed, even when they are not -- the empty-calorie hypothesis.  

So I'm not sure exactly where Rosen comes up with the idea that people are, or will be, better informed in this new media landscape.  You could fashion an argument that younger people might be more drawn into this fragmented news environment.  Given they tend to be the least informed, we might see some improvement there, and any improvement among young citizens will help.  But I'll be honest, other than a blip in the 2008 election, the data don't really support this conclusion.  Yet.

You might also argue that people will be become better informed about what they're truly interested in, which is a riff on the V.O. Key/Doris Graber/etc. argument about the public's knowledge being greater than what we give it credit for, if we only asked the right questions.  But such balkanized knowledge and issue publics (which may or may not exist) do not serve a democracy well, so maybe new media will increase some knowledge for some people about some things, but the outcome isn't pretty, at least according to most political thinkers.

Of course, it may mean no change at all.  The public's knowledge about public affairs remained largely unchanged for decades.  And while it seems to be inching down a bit of late, that may merely be a statistical blip and nothing to do with the changing news landscape.  But it most certainly does not mean that people will be better informed, not from what we're seeing so far.

Even Docs Don't Know

Here's some confidence-building news.  Not only are we confused about the health cares system.  So are our future doctors.  This story (scroll down a bit on the link page) gives details of a study that finds, well, below you can read for yourself:

A study published in the September issue of Academic Medicine found that nearly half of all medical students believe they have been inadequately educated about the "practice of medicine" -- especially related to medical economics.

Feel sicker?  Try this:

But fewer than 50 percent of medical students said they believed they had received appropriate training in areas related to the profession they are prepping to enter.

Feeling worse?  I know I am.

I suppose it's possible lots of other professional schools face the same problem.  Wouldn't it be good to know pharmacists are baffled while spooning out your pills?  And engineers?  And journalists?  Okay, the latter is less likely to kill ya, but I expect this is why doctors have office managers, to understand all the health care stuff they don't understand.

More Knowledge Quotations

Information is not knowledge
- Albert Einstein

Knowledge is no value unless you put it into practice
- Anton Chekhov

Take the two famous quotes above and you actually run smack into an ongoing controversy among those who study knowledge in general and political knowledge in particular. Is what people know sufficient to assume sophistication? Those who study expertise or sophistication or similar constructs would argue no, that knowledge itself is necessary but not sufficient. It's not what ya know, it's what ya know and how ya use it that matters.

This gets at how we measure knowledge. In political studies, we rely heavily on those civics class questions of who is Nancy Pelosi or what party controls the U.S. House of Representatives. We assume if you can rattle off the answers correctly, you're somehow more knowledgeable than those who cannot -- but we rarely build into our models how people use such information, if at all. Exceptions are when we see studies of how people organize their political world, often along partisan or ideological lines, and the role civics/textbook knowledge plays in all that.

Studies of sophistication, they use knowledge, but they also tend to use other factors in a multidimensional (and often muddy) construct. In other words, I'm not sure we gain all that much in our studies of "expertise" and "sophistication," which brings us back to our tired yet true measures of civics knowledge. They're simple, they hang together well (Cronbach's Alpha is usually in the .70s), and they have years of previous research to back 'em up. So we default to the conceptual safe ground.

But information, as the good professor notes above, is not knowledge.

It's an interesting problem for those of us who mess with these concepts, who try to analyze and predict based on them, who desperately need to publish in academic journals for promotion or tenure or a better raise (when there are raises and not furloughs). For me it's mostly an interesting intellectual exercise, and eventually I'd love to create to final, best, ultimate measure of sophistication that makes actual sense. Or, failing that, a knowledge measure that answers many of the criticisms I mentioned above, and the many I've not bothered here to get into (but have discussed in other posts).

Maybe it's time for a Political Wisdom construct.

Huh?

Here's my model, the old guys you'd see around the table at some diner, sipping coffee and arguing politics, and some snotty college kid sits down and can tell them all the names they get wrong, the little factoids they get confused, but when it comes to knowing -- the old guys beat the snotty kid every time (in this scenario, I was once the snotty kid...long story). It's how you use information, how you make sense of the world, that matters most. Hence my new construct -- Political Wisdom.

And no, I have absolutely no idea how to measure it, especially in a telephone survey.

Knowledge Quotes

We are drowning in information but starved for knowledge.

- John Naisbitt
Megatrends: Ten New Directions
Transforming Our Lives

I like pulling quotations and then riffing off them. This book was written in the early 1980s, before the Internet entered most people's lives. So why this quotation? With the Net, with the fragmenting of the media marketplace, we're certainly drowning in information. Compared to today, back in 1980 were were merely wading in information maybe to our ankles or knees. Today we're up to our chins. We're treading as best we can, and probably going under any minute now.

We have access to so much information, and yet we seem to know less and less.

Technology swamps us with access to facts and opinion and stuff, and then someone comes along with technology to help us organize this stuff -- at a price. An iPod does that. A smart phone too. We get better and better at finding stuff out, but we get lousier and lousier at learning. The above quotation means so much today, probably even more tomorrow. I can "google" something, but does that make me knowledgeable? Or just a good typist?

The Internet Kills Knowledge?

Much like earlier Atlantic articles on whether Google is making us smarter or dumber, a The New York Times piece -- based on this article -- gets into the question as well. Good reading, both.

In part, this has everything to do with facts. Does it matter that we can immediately recall some fact as compared to being able to quickly and efficiently find some fact? Are we raising a generation of people who know less, but who are more capable of finding stuff out? Some say it's no problem. We're creating a generation, through Google, of integrators, of people who know where to find stuff out and how to link it all together to make sense of some issue, question, or problem. Others sorry that without basic knowledge, an underpinning of common understanding, new information means very little and cannot be integrated, no matter how quickly you can Google some fact.

From the piece mentioned above:

A certain lack of general knowledge—what some might call ignorance—is thus built into the system, and will be more so in the future. My Googling undergraduates are doing something they may have been encouraged to do at school.

This may be one of the most important topics in education, and what people know, for quite some time. Is it a scary world? Not so much. Writing, too, was feared to create a forgetfulness in the public, an inability to learn and instead rely on words scratched on parchment. And we all know that turned out to be a pretty good idea. So a world in which basic facts and information are at our fingertips, so goes the reasoning, frees us to consider deeper what we are learning and to better match it with other information. To integrate. To learn wisely.

I'm wanting to buy into this. Really I am.

This article quoted above is excellent, and I strongly recommend it. So good that I'm going to lift, with due credit to Brian Cathcart, a professor of journalism at Kingston in the U.K., the end of the article:

There will always be dimwits, and their feats of stupidity will always make news. Equally, there will always be teachers and parents who shake their heads at the supposed ignorance of the young. We need to be careful before we construct trends from such things. But the internet is different, and it lifts the discussion onto a different plane. We are bound to tap into it for general knowledge, and the young will do it first. Schools are surely right to encourage them. The story of Thoth tells us that the curmudgeonly response—“This invention will produce forgetfulness in the souls of those who have learned it”—is a waste of breath.

But equally, the extraordinary popularity of the quiz in the mass-communication age suggests that general knowledge, the idea of a pool of information shared within a culture and a time, is potent enough to survive.

Far Out Answers

Sometimes when you ask straightforward questions to a survey respondent you get far out answers. As I was messing around with an advance release of the ANES 2008 election data, I came across these two questions

• As far as you know, what is the current unemployment rate in the United States, that is, of the adults in the United States who want to work, what percent of them would you guess are not unemployed and looking for a job?
• What is your best guess of the average price of regular unleaded gasoline across all of today?
  

As to the first, depending on which set of numbers you use, the unemployment rate at about the time of the survey at election time was probably between 5 and 7 percent (interviews happen over a couple of months). One fed number says the average for 2008 was 5.8 percent, so let's assume it's in the ballpark.

How did people answer? Wildly. First the good news. The mode, the most popular response, was 6 percent. Sweet, but that's by a mere 264 of over 2,000 respondents. Other estimates of the U.S. unemployment rate ranged from 0 (goofy optimistic) to 100 (sickly pessimistic, but shared by 25 people grasping for an answer). Thus the challenges of asking knowledge-based questions. The gas price is a bit tougher since it's state-specific and the costs differ. The average U.S. price at about the time of the survey (serious fudging here, sorry) was between $3.50 and $3.95 (prices peaked about July 2008). Most of the survey responses were significantly lower, though one person apparently paid $50 a gallon. Maybe in Europe.

The lesson? People not only guess and guess badly, they're just plain goofy and give the silliest answer they can give to some survey questions that requires them to seriously recall number-based facts. As an analyst I'd collapse all responses over a certain point -- say all $5 or more for gas put together at $5 -- or I might even toss out the outrageous ones. It depends on the purpose of my analysis, I suppose, but the point here is that you can never guess how people will answer a seemingly simple, straightforward question.

Turning Data Into Knowledge

Great column about a conference devoted to ways to turn statistics and data into knowledge. The conference, Seminar on Innovative Approaches to Turn Statistics into Knowledge, has a site here and even better, has some online versions of presentations here. I skimmed a couple and hope to dig deeper in a day or so, steal some ideas for my own journalism classes.

According to the site:

The seminar should contribute to the development of tools to help people transform statistics into knowledge and decisions. A first condition for statistics to be used this way is that relevant statistics become known, available, and understood by wider audiences. The seminar is held in the context of the Global Project on “Measuring the Progress of Societies”. It should contribute to one of the goals quoted in the Istanbul Declaration: "produce a broader, shared, public understanding of changing conditions, while highlighting areas of significant change or inadequate knowledge".

We use lots of tricks to help people understand what's happening, such as maps that summarize statistics, or drag-down menus to help people find, for example, how their local school matches up with other schools. The end result is helping people make sense of their world.

PR and What People Know

Remember those old models of the molecule? Protons were positive, electrons were negative, and neutrons were undecided, usually all in balance to create a stable molecule.

Now think of an attitude in the same way, except rarely is it in balance. For most of us, any attitude object -- a person, a place, a company, nearly anything at all -- comes with positive, negative, and neutral bits of information, the result rarely if ever coming out equal. If I dislike Dancing with the Stars (and I do), then I'll list more negatives than positives.

This is where attitudes meet knowledge. And this is also the realm of PR.

As PR guru Karen Russell would happily note, I'm not a PR guy, I'll never be a PR guy, and PR is all the better for it, so I'll keep my PR discussion to a minimum and instead focus on attitudes and knowledge and how difficult it can be to change an attitude through the use of information. It comes down to this -- misperception is awfully hard to change.

Let me give you a recent example.

During the recent presidential campaign there was a rumor out there that Barrack Obama was Muslim, despite all information and facts to the contrary. Who stuck to that belief? People who already had reason to believe the worst about Obama, mostly conservative Christians and hardcore Republicans. The attitude overwhelmed knowledge. Indeed, putting out "true" information will sway some people but not those who are already attitudinally predisposed to be against you -- which is why I'd make a lousy PR person. It'd drive me nuts.

There are ways to attack this, I suppose. One is parody and humor, to make fun of the misperception. That's a tricky tactic and it seems to work better for younger people than for older people. I'm sure there are differences in the kind of client you represent, both in their openness to such a dangerous strategy and whether it'd really work or not. Some companies would never accept such an approach, simply because it flies against their tradition. Pitch this campaign and you'd be out the door and looking for clients elsewhere.

Simply putting out "true" information, that doesn't work either, not for the hardcore believers, but you might add a few "protons" to their molecular attitude, soften their perception a little, but you have to be careful here too. Attacking the misperception as something held by idiots, that's gonna backfire, or so predicts most theories of persuasion. People love to counterargue, both internally and with others, and by forcing them into this counterargument you often push their attitudes to an even greater extreme. The research on attitude extremity is neat and gets right at this, but it's not something we spend a lot of time looking at in mass comm.

As an aside, we do know from the social psych literature that making someone generate arguments against their position tends to soften their attitudes -- the "walking in someone else's shoes" approach.

So it all comes down to protons, neutrons, and electrons, except in this case we're talking attitudes and not molecules. Each bit and piece of information goes into our head, gets mixed up, and we apply some personal algebraic formula to arrive at an overall attitude. Knowledge can influence attitudes, but more often it goes the other way around, and that makes it damned difficult if you're trying to generate positive attitudes about someone or some thing.

Knowing the Court

When looking at what people know, we can explore lots of areas. The usual one is public affairs, but it's also important to understand what people know about lots of other domains.

For example, the courts. In particular, the U.S. Supreme Court. The author puts it well:

Two of the findings of this research run strongly counter to existing understandings of public knowledge of law and courts. First, these respondents demonstrate relatively high levels of information about the Supreme Court. To our knowledge, few prior studies have documented this level of information about the Court. We contend that this finding is in part a function of the method by which knowledge is measured, and we are consequently critical of most earlier efforts to document what citizens know about the Supreme Court. When citizens are asked reasonable questions about what they understand about the Supreme Court, most can answer accurately.

As I've mentioned before, it's often how we measure knowledge that matters most. Previous surveys and studies done by damn smart people tend to overestimate the public's lack of knowledge because of the way we ask questions.

In conclusion, the author writes:

Certainly there is little in these data to suggest that the views of the American citizenry are too ill-informed to be worthy of any serious consideration, both from the political process and from scholars of the judiciary. It seems that the American people may know enough about law and courts to be able to perform their assigned function as constituents of the contemporary judicial system in the U.S.

I would note that the data suggest a small decline in accuracy about the Supreme Court from 2001 to 2005. About 74 percent in 2001 correctly knew justices are appointed. About 65 percent knew it in 2005. Same is true for a couple of other knowledge questions.

And the old recognition versus recall issue is seen, with recognition proving superior at tapping what people know about the Court. I won't go into the r vs r debate. See my other posts.

What's missing here, for me, is media. I'd love to see how people to watch a lot of court programs or the news differ in these results. Alas, this is not to be seen.

A summary to the study is here. Scroll down and click any option to get a pdf of the study.