Bayes’ Theorem

A simple short (and hopefully intuitive) explanation of Bayes’s theorem.

Who is Bayes?

Given the recent new that Cass business school is renaming its institution after Thomas Bayes because of the former's links to the slave trade. I though it would be good opportunity to demystify Bayes’ often misunderstood theorem.

Thomas Bayes was a British Mathematician who is most famous piece of work was published after his death. This work, “An Essay towards solving a Problem in the Doctrine of Chances“ where he used Binomial data comprising r successes out of n attempts to learn about the underlying chance θ of each attempt succeeding. Bayes’ key contribution was to use a probability distribution to represent uncertainty about θ.

He is known to have published only two works in his lifetime, one arguing that goodness or benevolence motivated God’s actions in the world and the other defending the logical foundation of Isaac Newton’s calculus from criticism.

Bayes’ Theorem

So here it is Bayes’ theorem but what do the terms in this equation actually correspond to? The best way to explain it is with a simple example about Dave. Dave is described by the people who know him best as:

“Dave is very shy and withdrawn, invariably helpful but with very little interest in people or in the world of reality. A meek and tidy soul, he has a need for order and structure, and a passion for detail.”

Which of the following do you find more likely? That Dave is a Farmer or that Dave is a Librarian?

This question was posed in a study by Kahneman and Tversky. It was found that most people, upon hearing this description would guess that Dave is more likely to be a librarian. Dave, from his description seems to fit that of a stereotypical librarian. However, Kahneman and Tversky claim this is irrational, not because of incorrect or correct views about the stereotypical librarian but because no one thinks to incorporate information about the ratio of farmers to librarian. So what is the correct answer? Let’s find out using Bayes’ Theorem. So our objective here is to find out is what is the probability of Dave being a librarian given description is true e.g.

Finding our priors is easy — a quick google and some back of the envelope calculations puts this ratio (in the UK) at 1:6. Referencing back to the equation, the information here will form our prior (or P(A)). The ratio tells us that P(A) = 1/7 ≈ 14%. We can illustrate this split here:

We now need to obtain our likelihood term, P(B | A) or the probability that the description is true given they are a librarian. For this example we’ll go with our gut and say that 40% of librarians and only 10% of farmers would fit the description.

We need one final part to finish our equation, P(B), which is the probability of the description being true. To help let’s add to our previous illustration

That’s better, so the value of P(B) is easy to see now. Evaluating the equation visually, we would get:

Which mathematically translates to:

Which gives us a final the probability that Dave is a Librarian is 40%! So even though the description may be more fit for a librarian, it is more likely that Dave is a farmer.

Thanks for reading, hopefully you found the explanation intuitive.