AbsurdBeats

This is shit, isn’t it?

I mean, I’m not a social psychologist, and even fellow political scientists doubt the soc-sci cred of theorists like me, but Jonathan Haidt seems to be siphoning way too much meaning out of a poorly-designed linguistic study.

To wit:

When I was doing the research for The Righteous Mind, I read the New Atheist books carefully, and I noticed that several of them sounded angry. I also noticed that they used rhetorical structures suggesting certainty far more often than I was used to in scientific writing – words such as “always” and “never,” as well as phrases such as “there is no doubt that…” and “clearly we must…”

To check my hunch, I took the full text of the three most important New Atheist books—Richard Dawkins’ The God Delusion, Sam Harris’s The End of Faith, and Daniel Dennett’s Breaking the Spell and I ran the files through a widely used text analysis program that counts words that have been shown to indicate certainty, including “always,” “never,” “certainly,” “every,” and “undeniable.” To provide a close standard of comparison, I also analyzed three recent books by other scientists who write about religion but are not considered New Atheists: Jesse Bering’s The Belief Instinct, Ara Norenzayan’s Big Gods, and my own book The Righteous Mind.

To provide an additional standard of comparison, I also analyzed books by three right wing radio and television stars whose reasoning style is not generally regarded as scientific. I analyzed Glenn Beck’s Common Sense, Sean Hannity’s Deliver Us from Evil, and Anne Coulter’s Treason. (I chose the book for each author that had received the most comments on Amazon.) [delinked two items]

Anyone else see the problem? He’s comparing three books explicitly against religion to three general right-wing texts, i.e., not three texts explicitly in favor of religion.

That’s some shit sampling right there, providing shit comparisons. If you want to compare the effect of a variable on x across two populations, then you need to hold everything constant except that variable: You need to compare anti-religion text to pro-religion texts.

Even a political theorist knows that.

And which had the most comments on Amazon? Uh huh. (Others at that second link point out problems with linguistic analysis generally.)

I am admittedly a skeptic of Haidt’s work, precisely over the issues of definition and control. I haven’t read The Righteous Mind so can’t comment on the arguments he presents there, but I have read other, shorter pieces by him and about his work. And while I do agree that American liberals and conservatives may—may—assign different priorities to different values, I think it’s just as likely that we assign different definitions to different values.

For example, libertarians and social-welfare liberals may both agree that fairness matters, but disagree as to what fair is. To the libertarian, fairness may mean being able to profit from the fruits of one’s labors; to the liberal, fairness may mean that every person has a shot at/be guaranteed a decent life.

Again, I haven’t read Haidt’s book, so it is entirely possible that he covers the definitional issue. It’s pretty basic, after all.

Then again, controlling your sample populations is pretty basic, too.