Authenticity

The folks over at Minnesota Public Radio have put up an online form so that listeners/readers can tell them what they think are the most important issues facing the state. MPR's Public Insight Journalism chief, Andrew Haeg, climbed up on Polinaut's soapbox today to talk about the status of this project:

More than 2,000 people have responded, from across the political spectrum and from every established party, and have told us what their first and second choices are, and why . (We plucked all of the issues directly from the lists of issues that candidates said they were going to be focusing on this election season.)

We’re only now beginning to analyze the responses. At the moment, all we have is this graphic showing what issues respondents ID’d as most important, and 2nd most important.

It’s not scientific. It’s not even quasi-scientific. Think of it as a Polaroid snapshot: a little blurry, a little fuzzy but authentic and pretty interesting.

It's not scientific, but at least it's "authentic." How nice.

It's also complete garbage.

Not that MPR is alone in doing this -- the voluntary survey has been a staple of local commercial newscasts for years. In fact, the most eagerly watched poll in the country is a survey for which people volunteer to participate -- the American Idol voting.

But that doesn't change the fact that voluntary surveys give you no useful information about the true shape of public opinion. None. Nada. Zip. Why? Because without using a random sample, the results of such polls cannot be projected to the larger population. Even if you could, the lack of a random sample would also make it totally impossible to calculate a margin of error.

Media organizations know this, of course. They usually take pains to report the results as "unscientific," but then they go on to say that the results are "interesting" or "represent a lot of views on this issue." This is a bit like reporting that Britney Spears is carrying Jack Nicholson's love child. It's not true, but it sure is "interesting," isn't it?

Why, then, do so many media organizations use these polls? Because junk polls draw viewers. Stations know that those who call in to the polls are quite likely to continue to watch to the end of the broadcast to see if their "side" won. Junk polls are a marketing technique.

I'm sure that part of MPR's calculation with their whole "Public Insight Journalism" initiative is that those who feel that they are contributing to the newscasts are more likely to contribute money to MPR as well. And frankly, I don't really have a huge problemn with that if they are using the comments that they get to illustrate the issues raised by a story which is based on other evidence.

But that's not what's going on here. The voluntary responses are all the evidence that they have. And for the purposes of determining what Minnesotans think are important issues in this year's election cycle, that evidence is not worth the cost of the computer memory it takes to store it.

posted by DK @ 11:00 PM permalink