Wednesday, July 22, 2009

Slate's News Experiment

Michael Kinsley of Slate poses this experiment: two journalists will spend one hour a day reading print newspapers, two will spend one hour a day getting their news online, but not shovelware or aggregators, so that leaves out Google News or Huffingtonpost but does include sites like Politico.

This goes for three days, then he gathers them back to discuss their experience. They're really doing this.

The headline for this piece reads Who's Better Informed, Newspaper Readers or Web Surfers? so I assume political learning is one of the examined outcomes, along with the overall experience, the frustrations, the good and bad, and of course the ugly, of getting news in more or less one fashion.

Let's set the record straight. This "highly unscientific experiment," as Kinsley calls it, is highly unscientific to the point of being silly. It's more of a vaguely interesting gimmick than anything else.

There's one published study that looked at the differences between people who read the New York Times online versus those who read the paper version. This controlled experiment randomly assigned people to one of two groups (print vs. online of the Times) and examined differences in learning and other factors. Print came out ahead (I can't lay my hands on the link, but I'm fairly certain it ran in Journalism and Mass Communication Quarterly).

Any differences have to do with the way we read, the way we approach online versus ink on paper. It's more than mere differences of medium, it's differences of motivation. We simply approach a screen differently, almost at an unconscious level, and subtle differences emerge.

This is a bit more PhDweeby than Kinsley wants, of course. The gimmick study will be fun for the discussion that emerges, but beyond that the results are meaningless. After all, they're looking at very different sources AND very different media. That, budding methodologists, is a confound of confounding proportions.

No comments: