Wednesday, April 30, 2014

When the Null Hypothesis IS Your Research Finding

In journalism, the notion of native advertising is seen both as controversial and as a path toward financial stability. The question, then, is whether native advertising threatens the credibility of a news site. In other words:
They worry that, at its core, native advertising is about tricking your reader into reading an ad and thinking its editorial content. 
The results in this study (via Nieman Journalism Lab) suggest no.

Read the article for yourself, or the original research here (begins on page 78 of the PDF). Lemme point out I know nothing about this journal or its peer review process.  Skimming the research itself, I like its use of a college student and human (i.e., adult) samples as a comparison. I hate the use of "mediums" (please, media), but otherwise the procedure seems straightforward. Subjects were randomly assigned to view one of two versions of a site -- one with native advertising, one with "traditional" advertising.

Okay, but here's the weird part:

H1: Ad type will have minimal effect on credibility.

All of you who suffered through and passed a graduate class in research methods will recognize this as an example of hypothesizing the null hypothesis. In other words, this is usually not done. The null hypothesis is of no effect and you hypothesize a difference, and what they've done here is run an experiment with a fairly mild, subtle manipulation and have taken the lack of an effect on credibility as a, excuse the pun, credible finding.

No. No. No.

I'm buried in grading so I can't vent more, but this result should not be taken as evidence that native advertising has no effect on a news site's credibility. Perhaps there is no effect, but this study is not conduced in a fashion to truly answer that question.

No comments: