Photo by the author
# Entry
Entering the field data scienceyou’ve probably been told have to understand probability. While this is true, it doesn’t mean you have to understand and recall every statement in your statistics textbook. What you really need is a working understanding of probability concepts that come up all the time in real-world projects.
In this article, we’ll focus on the basics of probability that actually matter when building models, analyzing data, and forecasting. In the real world, data is messy and uncertain. Probability gives us the tools to quantify this uncertainty and make informed decisions. Now let’s describe the key probability concepts you’ll employ every day.
# 1. Random variables
A random variable is just that a variable whose value is determined randomly. Think of it as a container that can hold different values, each with a certain probability.
There are two types that you will constantly work with:
Discrete random variables take countable values. Examples include the number of customers visiting your website (0, 1, 2, 3…), the number of defective products in a batch, the results of a coin toss (heads or tails), and more.
Continuous random variables can take any value from a given range. Examples include temperature readings, time to server failure, customer lifetime value, and more.
Understanding this distinction is significant because different types of variables require different probability distributions and analysis techniques.
# 2. Probability distributions
A probability distribution describes all the possible values a random variable can take and the probability of each value. Every machine learning model makes assumptions about the underlying probability distribution of the data. If you understand these distributions, you will know when the model assumptions are correct and when they are not.
// Normal distribution
The normal distribution (or Gaussian distribution) is everywhere in data science. It is characterized by the shape of a bell curve, with most values clustering around the mean and tapering symmetrically on both sides.
Many natural phenomena are normally distributed (elevation, measurement errors, IQ scores). Many statistical tests assume normality. In linear regression, the residuals (prediction errors) are assumed to be normally distributed. Understanding this distribution helps verify the model’s assumptions and correctly interpret the results.
// Binomial distribution
The binomial distribution models the number of successes over a fixed number of independent trials, where each trial has the same probability of success. Think about flipping a coin 10 times and counting heads, or showing 100 ads and counting clicks.
You’ll employ this to model click-through rates, conversion rates, A/B test results, and customer churn (will they churn: yes/no?). Whenever you model “success” and “failure” scenarios over multiple trials, binomial distributions are your friend.
// Poisson distribution
The Poisson distribution models the number of events occurring in a fixed interval of time or space when these events occur independently at a constant average rate. The key parameter is lambda ((lambda)), which represents the average frequency of occurrence.
You can employ the Poisson distribution to model the number of customer service calls per day, the number of server errors per hour, predict occasional events, and detect anomalies. When you need to model count data at a known average rate, the Poisson distribution is.
# 3. Conditional probability
Conditional probability is the probability of an event occurring, given that another event has already occurred. We write it as (P(A|B) ), read as “the probability of A given B”.
This concept is absolutely fundamental to machine learning. When you build a classifier, you essentially calculate ( P(text{class}|text{features}) ): the probability of a class given the input features.
Consider detecting email spam. We want to know ( P(text{Spam} | text{contains “free”}) ): if an email contains the word “free”, what is the probability that it is spam? To calculate this we need:
- ( P(text{Spam}) ): Overall probability that any email is spam (base rate)
- ( P(text{contains “free”}) ): How often does the word “free” appear in emails
- ( P(text{contains “free”} | text{Spam})): How often do spam messages contain “free”
This last conditional probability is what we really care about in classification. This is the basis of Naive Bayes classifiers.
Each classifier estimates conditional probabilities. Recommendation systems employ ( P(text{item liked by user} | text{user story}) ). Medical diagnosis uses ( P(text{disease} | text{symptoms}) ). Understanding conditional probabilities helps you interpret model predictions and create better features.
# 4. Bayes’ theorem
Bayes’ Theorem is one of the most powerful tools in your data analysis toolkit. It tells us how to update our beliefs about something when we receive recent evidence.
The formula looks like this:
[
P(A|B) = fracA) cdot P(A){P(B)}
]
Let’s analyze this using the example of medical research. Imagine a diagnostic test that is 95% right (both in detecting real cases and excluding non-existent cases). If the prevalence of a disease is only 1% in the population and you test positive, what is the actual probability that you have a particular disease?
Surprisingly, it’s only about 16%. Why? Because at low prevalence, false positives outnumber true positives. This demonstrates an significant insight known as base rate error: the base rate (incidence) should be taken into account. As the prevalence increases, the likelihood that a positive test result means you are truly positive increases dramatically.
Where you’ll employ it: A/B testing analysis (updating beliefs about which version is better), spam filters (updating the probability of spam as more features are released), fraud detection (combining multiple signals), and any time you need to update predictions with recent information.
# 5. Expected value
Expected value is the average result you can expect if you repeat something many times. You calculate this by weighting each possible outcome by its probability and then summing these weighted values.
This concept is significant when making data-driven business decisions. Consider a marketing campaign costing $10,000. You estimate:
- 20% chance of great success ($50,000 profit)
- 40% chance of moderate success ($20,000 profit)
- 30% chance of penniless performance ($5,000 profit)
- 10% chance of complete failure ($0 profit)
The expected value will be:
[
(0.20 times 40000) + (0.40 times 10000) + (0.30 times -5000) + (0.10 times -10000) = 9500
]
Since this is a positive result ($9,500), it is worth starting the campaign from an expected value perspective.
This can be used to make decisions about pricing strategy, resource allocation, feature prioritization (expected value of building feature X), investment risk assessment and any business decision where multiple uncertain outcomes need to be considered.
# 6. The law of gigantic numbers
The The law of large numbers states that as more samples are collected, the sample mean approaches the expected value. That’s why data scientists always want more data.
If you flip a fair coin, early results may show 70% heads. But spin it 10,000 times and you get almost 50% heads. The more samples you collect, the more reliable your estimates will be.
Therefore, metrics from petite samples cannot be trusted. An A/B test with 50 users per variant may accidentally show that one version wins. The same test with 5,000 users per variant gives much more reliable results. This principle underlies statistical significance tests and sample size calculations.
# 7. Central limit theorem
The Central limit theorem (CLT) is probably the most significant concept in statistics. It states that if you take gigantic enough samples and calculate their means, those sample means will be normally distributed – even if the original data is not.
This is helpful because it means we can employ normal distribution tools to infer almost any type of data, as long as we have enough samples (usually ( n geq 30) is considered sufficient).
For example, if you sample from an exponential (highly skewed) distribution and calculate means from samples of size 30, those means will be approximately normally distributed. This works for uniform distributions, bimodal distributions, and almost any distribution you can think of.
This is the basis of confidence intervals, hypothesis testing, and A/B testing. Therefore, based on sample statistics, we can draw statistical conclusions about population parameters. For this reason, tiz tests work even when the data is not completely normal.
# Summary
These probability concepts are not stand-alone topics. They create a set of tools that you will employ in every data analysis project. The more you practice, the more natural this way of thinking becomes. As you work, ask yourself:
- What distribution am I assuming?
- What conditional probabilities am I modeling?
- What is the expected value of this decision?
These questions will push you towards clearer reasoning and better models. Once you feel comfortable with these basics, you’ll think more effectively about your data, models, and the decisions that influence them. Now go and build something great!
Bala Priya C is a software developer and technical writer from India. He likes working at the intersection of mathematics, programming, data analytics and content creation. Her areas of interest and specialization include DevOps, data analytics and natural language processing. She enjoys reading, writing, coding and coffee! He is currently working on learning and sharing his knowledge with the developer community by writing tutorials, guides, reviews, and more. Bala also creates captivating resource overviews and coding tutorials.