Deja vu all over again

I guess Craig Westover doesn't read this blog:

MPR has a little political toy called “Select A Candidate.” You answer a series of questions about your thinking on various issues, and you’re given a score ranking of how you stand vis a vis various candidates. I tried my hand at the governor’s race.

. . .

Considering that this is an MPR toy, it’s not surprising that cumulatively, the most compatible candidates are in reverse order they were when ranked against my positions -- Steve Kelly (37 percent), Beckey Lourey (33 percent) and Peter Hutchinson (11 percent).

But here’s the interesting part again. Remember this is MPR listenership, liberal, yes, but also significant in number. The next three candidates in order of cumulative issue compatibility were Sue Jeffers (7 percent), Mike Hatch (7 percent) and Tim Pawlenty (5 percent).

Again -- this is hardly a scientific survey, but all things considered, if MPR listenership is the typical liberal voter, if it comes down to Pawlenty vs. Hatch, Jeffers could find herself a pretty popular gal. All those Kelly, Lourey and Hutchinson voters have to go somewhere.

(emphasis added)

If it's not clear from the way I edited this, what Westover is doing is looking at the cumulative totals of all those who have used the Select-a-Candidate tool. He makes the usual reference to how the survey is non-scientific, and then goes on to draw inferences anyway.

Westover actually makes two huge leaps which are completely indefensible: (a) that the people taking part in Select-a-Candidate are representative of the entire listenership of MPR, and (b) that the listenership of MPR is representative of all liberal voters in Minnesota.

Sure, it's possible that either or both of these could be true, but if so, it would be totally coincidental. Without a random sample, you cannot generalize from a small sample to a large population. Those who write for serious media outlets like MPR and the Pioneer Press should know this.

Charlie Quimby hits the nail squarely on the head (talking about the MPR "most important issues" online survey, but the same point applies here):

Organizing the information in a graphic form — even a rather crude one — made it appear to show something meaningful, despite the disclaimers. That it might've reinforced our own perceptions of people along political spectrum made it all the more seductive. Junk data, like junk food, is tasty.

(I should note that MPR carefully makes no claims that the Select-a-Candidate totals are in any way representative of a larger population.)

posted by DK @ 12:45 PM permalink