Statistical Intuition

So . . . the movie Moneyball opens today in theatres everyone, and I am rather excited to see it.  I read the book some time ago and it really struck a chord with me (you'll understand why later in the post), I like baseball movies (more than I like baseball these days) and I think it could be this year's Social Network.  Yesterday I read a column about the movie and its core message of the use of statistical information to improve decisions.  This column was co-written by Daryl Morey, the general manager of the Houston Rockets basketball team and one of the more stats-friendly executives in the NBA. I think that this column really hits the nail on the head and I'd like to elaborate on it here.

We humans have a tendency to trust our intuition more than external information.  This is not such a bad tendency; until very recently in human history, external information was scarce, unreliable, and likely biased (not that it is so perfect now, but you cannot deny that it is better. Well, you can deny that if you want, I just happen to disagree).  But this tendency also gets us into trouble sometimes.  There was an episode of the West Wing in which a character said that people are more likely to believe a probable impossibility than an possible improbability.  In other words, we'd rather believe in something that absolutlely cannot happen but that seems doable than something that can happen but seems unlikely.

Why do I bring this up?  Because this belief system leads to bad decision making.  Our intuition that something could happen (e.g. one of my students believing that if she just does perfectly on the exam she'll pass the course) often ignores that something cannot happen (same example: the student's grade is such that even a perfect score results in a failing grade overall).  An examination of the numbers would show this to be an impossibility, but we don't rely on the numbers, we rely on intuition.

So, back to Moneyball.  For most of the history of baseball (and here is a tidbit of that, a fake one, and there is swearing, oh my), decisions were made on intuition.  Criteria like "does this guy look like a baseball player" and "is he white" were used instead of criteria like "can he hit the ball" and "does he use human growth hormone - he does? - great!"  Moneyball is about how one team, out of necessity, turned to mathematics and stats to try to gain an advantage (I would tell you what happened next, but I don't want to spoil anything that you could spoil yourself by looking up widely-available recent baseball records).

What I like best about Morey's column, though, is that he points out that just as you can't only rely on intuition, you also can't only rely on stats.  When the two disagree, I would suggest believing the stats more, especially if you know how the data was collected and how reliable and valid it is.  But the numbers are simply a decision-making tool, not the decision maker.  It is the same thing that I tell my business students.  In school they will acquire a great number of theories and models (tools), but the way that they add value is by knowing how and when to use those tools to make decisions.  If you only follow the stats, you become an automaton and can be replaced by a computer.  If you weigh information, take into account context, question sources of data, and so on, you add value to the process and become more difficult to be replaced by something like this.

One last note - engaging in advantage-gaining through stats is a difficult, continuous process.  The stats may show one arbitrage opportunity, but once you use that information and it becomes known, the opportunity is no longer there (because everyone is doing it).  And then you have to go on to the next opportunity.  And so on, forever.  And my intuition says that this seems like a lot of work.