Probability is a metric for determining the likelihood of an event occurring. Many things are impossible to predict with 100% certainty. Using it, you can only predict the probability of an event occurring, i.e., how likely it is to occur. In this tutorial, you will learn about Bayes Theorem, an important sub-topic in probability theory.
Bayes Theorem Terminologies
Before you dive into the world of the Bayes Theorem, you must first grasp a few concepts. Understanding Bayes Theorem requires an understanding of the following terms.
-
Experiment
When you hear the word "experiment," what is the first image that comes to mind? The majority of people envision a chemistry lab with test tubes and beakers. In probability theory, the concept of an experiment is quite similar.
An experiment is a carefully planned procedure carried out under carefully monitored conditions.
Experiments include tossing a coin, rolling a die, and drawing a card from a well-shuffled deck of cards.
-
Sample Space
An outcome is the result of an experiment. The sample space is the set of all possible outcomes of an event. For example, if you’re throwing dice and keeping track of the results, the sample space will be: {1, 2, 3, 4, 5, 6}
-
Event
An event is the outcome of a random experiment. Getting heads when you toss a coin is an event. Getting a 4 when you roll a fair die is an event.
-
Random Variable
A random variable is a variable with an unknown value or a function that assigns values to each of the outcomes of an experiment. A random variable can be discrete (meaning it has specific values) or continuous (meaning it has no specific values).
-
Exhaustive Events
Two or more events associated with a random experiment are exhaustive if their union is the sample space.
Let's say A is the event of a red card being drawn from a pack, and B is the event of a black card being drawn. Because the sample space S = {red, black}, A and B are exhaustive.
-
Independent Events
When the occurrence of one event has no bearing on the occurrence of the other, the two events are said to be independent. Two events A and B, are said to be independent in mathematics if:
P(A ∩ B) = P(AB) = P(A)*P(B)
For example, if A gets a 3 on a die roll and B gets a jack of hearts from a well-shuffled deck of cards, then A and B are independent events.
-
Conditional Probability
Let A and B be the two events associated with a random experiment. Then, the probability of A's occurrence under the condition that B has already occurred and P(B) ≠ 0 is called the Conditional Probability. It is denoted by P (A/B). Thus, you have:
What Is Bayes Theorem?
The Bayes theorem is a mathematical formula for calculating conditional probability in probability and statistics. In other words, it's used to figure out how likely an event is based on its proximity to another. Bayes law or Bayes rule are other names for the theorem.
Bayes Theorem Formula
The formula for the Bayes theorem can be written in a variety of ways. The following is the most common version:
P(A ∣ B) = P(B ∣ A)P(A) / P(B)
P(A ∣ B) is the conditional probability of event A occurring, given that B is true.
P(B ∣ A) is the conditional probability of event B occurring, given that A is true.
P(A) and P(B) are the probabilities of A and B occurring independently of one another.
Example of Bayes Theorem
Now, try to solve a problem using the Bayes theorem.
Problem 1: Three urns contain 6 red, 4 black; 4 red, 6 black, and 5 red, 5 black balls respectively. One of the urns is selected at random and a ball is drawn from it. If the ball drawn is red, find the probability that it is drawn from the first urn.
Solution: Let E1, E2, E3, and A be the events defined as follows:
E1 = urn first is chosen
E2 = urn second is chosen
E3 = urn third is chosen
A = ball drawn is red
Since there are three urns and one of the three urns is chosen at random, therefore:
P(E1) = P(E2) = P(E3) = ⅓
If E1 has already occurred, then urn first has been chosen, containing 6 red and 4 black balls. The probability of drawing a red ball from it is 6/10.
So, P(A/E1) = 6/10
Similarly, you have P(A/E2) = 4/10 and P(A/E3) = 5/10
You are required to find the P(E1/A) i.e., given that the ball drawn is red, what is the probability that it is drawn from the first urn.
By Bayes theorem, you have
P(E1/A) = P(E1) P(A/E1)P(E1) P(A/E1) + P(E2) P(A/E2) + P(E3) P(A/E3)
= 1/3 * 6/10(1/3 * 6/10) + (1/3 * 4/10) + (1/3 * 5/10)
= ⅖
Problem 2:
An insurance company insured 2000 scooter drivers, 4000 car drivers, and 6000 truck drivers. The probability of an accident involving a scooter driver, car driver, and a truck is 0.01, 0.03, and 0.015 respectively. One of the insured persons meets with an accident. What is the probability that he is a scooter driver?
Let E1, E2, E3, and A be the events defined as follows:
E1 = person chosen is a scooter driver
E2 = person chosen is a car driver
E3 = person chosen is a truck driver and
A = person meets with an accident
Since there are 12000 people, therefore:
P(E1) = 2000/12000 = ⅙
P(E2) = 4000/12000 = ⅓
P(E3) = 6000/12000 = ½
It is given that P(A / E1) = Probability that a person meets with an accident given that he is a scooter driver = 0.01
Similarly, you have P(A / E2) = 0.03 and P(A / E3) = 0.15
You are required to find P(E1 / A), i.e. given that the person meets with an accident, what is the probability that he was a scooter driver?
P(E1/A) = P(E1) P(A/E1)P(E1) P(A/E1) + P(E2) P(A/E2) + P(E3) P(A/E3)
= 1/6 * 0.01(1/6 * 0.01) + (1/3 * 0.03) + (1/2 * 0.15)
= 1/52
Conclusion
You have come to an end of this Bayes Theorem tutorial. The purpose of this tutorial was to introduce you to the Bayes theorem and conditional probability. The Bayes theorem is the foundation of Naive Bayes, one of the most widely used classification algorithms in data science.
If you are interested in statistics of data science and skills needed for such a career, Simplilearn’s Data Analyst is the right course for you.
If you have any questions regarding this ‘Bayes Theorem’ tutorial, do share them in the comment section. Our subject matter expert will respond to your queries. Happy learning!
