Tag Archives: data

OKCupid, Religion and Readability

OK Cupid has been doing some interesting things recently with data mined from their users which, on the one hand, is fascinating and, on the other hand, is extremely creepy.

It all boils down to some pretty interesting data points (what do white/black/asian/Hispanic people like or have in common, what is the focus of men and women of different races, etc). The one thing I was a little shocked by was this graphic at the end.

I was a little surprised that the Protestant reading level was so low. And when I am surprised by data, I try to see if I can replicate it. (This strikes me as an eminently scientific thing to do and yet I am bemused when people think I’m “attacking” their data. Whatever.)

Note: I should mention at the outset that it does kind of irk me that OKCupid displays this data and then kind of assumes that it holds for all people across the board when it is obvious to anyone who devotes more than 5 seconds to thinking about it that the data really only holds for OKCupid users who (I’m going out on a limb here) are probably disproportionately young, tech savy and single.

Moving along. To determine reading levels they ran the Coleman-Liau Index on the profiles so I went and typed up two sample religious profile summaries, one Christian and one atheist. They’re only a couple sentences long which I figure is fine since OKCupid profile summaries aren’t exactly known for their complex narrative arcs.

Here are the profiles that I typed up, attempting to mimic what I thought would be a fair religious summary from a similar reading level.

Atheist: I am an atheist. I believe that there is no God and that most people only believe religion because they are taught to do so by society and possibly also their parents.

Christian: I am a Christian. I believe that Jesus died on the cross for my sins and that he was raised again on the third day. I think that the Bible teaches us the truth and that God loves us very much.

Running them through this Coleman Liau tester to see what popped up.

Christian: 5.47 grade level

Atheist: 10.15 grade level

I laughed.

Really? Those little blurbs are so radically different that the Christian one is 4.68 grades stupider based on nothing more than a readability analysis? Sounds like BS to me.

Let’s try adding some evolution in there:

Atheist: Same as before + “When it comes to the world around us, evolution is the most likely explanation for everything.”

Score: 12.23

Christian: Same as before + “When it comes to the world around us, I think there are probably gaps in evolutionary theory and that evolution can’t explain everything.”

Score: 8.90

Well, that closes the gap by 1.35 points, but we’re still looking at a 3.33 grade gap between two positions that are transparently written with an identical textual style.

I progressively tried to add more and more to the Christian profile to counter-act the low score I got from starting with the basics of Christian belief.

Finally I ended up with:

I am a Christian. I believe that Jesus died on the cross for my sins and that he was raised again on the third day. I think that the Bible is true and that God loves us very much. I think there are probably gaps in evolutionary theory and that evolution can’t explain everything. Additionally, the philosophical underpinnings for views that argue against Christianity frequently neglect to apply the same standard of ideological rigor to their own faith based assumptions. Consequently, they hold Christianity to a double standard assuming that their position is the default one and that there is no need to defend it.

Score: 12.33

This “Christian profile view” scores about  as well as an absurdly simple statement of atheism with a supporting line about evolutionary theory. Basically, the algorithm they used translates “6th grade atheism” at the same level of textual complexity as “basic Christian beliefs + a philosophy degree”. (I flatter myself somewhat, but the final two lines are clearly a college level writing style.)

Here is not what I’m saying: I don’t think there is any level of conspiracy theory behind any of this. No one designed the algorithm so that Christians would look stupid.

However!

It seems likely that a simple statement of Christian belief like the one entered above anchors the score at the low end. The more someone communicates their Christian belief in the language that has been familiar in churches for centuries, the less likely they are to score well regardless of the remaining textual analysis of their profile. This anchoring effect might get lost if the profile was a three page essay. But profiles tend toward being short, simple statements meant to clearly indicate basic beliefs, inclinations, or personality traits.

Note: I promise I’ll pull back on religious topics. I just get irritated when people pull “evidence” of religious people being inferior in some way shape or form. It usually strikes me as hackery that the creators or purveyors of whatever data set are perfectly happy to accept and so they neglect to do any sort of skeptical follow-up.

You can actually see the same thing with a lot of war-based data. Half the time, the people pointing to the data didn’t even get the data right and a good chunk of the remaining examples strip context out of the data. Bugs the hell out of me.