Almost every quantitative tool in risk management is, underneath, a statement about probability. A value at risk number is a probability statement about losses. A credit rating is a compressed probability of default. A hedge is a bet about how likely two prices are to move together. Before any of that can make sense, you need a firm grip on what a probability is, what rules it must obey, and how probabilities combine when events interact. That is the whole job of this chapter.
The material looks simple, and the individual pieces are. The difficulty on the exam is never a single formula. It is keeping straight a handful of ideas that sound alike but mean different things: mutually exclusive versus independent, conditional versus joint, and the direction of a conditional probability. Candidates lose marks by mixing these up, not by forgetting the algebra. So the goal here is not just to state each rule but to make the distinctions between them impossible to confuse.
We will build the ideas in order. First the vocabulary of experiments, outcomes, and events. Then the axioms that any probability must satisfy. Then the ways events relate to each other, which is where conditional and joint probability enter. We finish with the two results that carry the most weight in later readings, the total probability theorem and Bayes theorem, because credit models, scoring systems, and scenario analysis all rest on them.
Key Takeaways
- A probability is a number between 0 and 1 that measures how likely an event is, where 0 means impossible and 1 means certain.
- The sample space is the set of every possible outcome of a random experiment, and an event is any subset of that sample space.
- Two events are mutually exclusive if they cannot both happen, and independent if one tells you nothing about the other; with positive probabilities these two properties cannot hold at the same time.
- Conditional probability satisfies P(A given B) equals P(A and B) divided by P(B), and this single relationship is the parent of the multiplication rule, the total probability theorem, and Bayes theorem.
- Bayes theorem reverses a conditional probability, converting the probability of evidence given a cause into the probability of the cause given the evidence, which is how a prior belief is updated by new data.
What Probability Measures
A random experiment is any process whose outcome cannot be predicted with certainty in advance, even though the full list of things that could happen is known. Rolling a die, observing whether a borrower repays or defaults next year, and recording tomorrow’s closing price of a stock are all random experiments. The defining feature is not that the process is chaotic, but that we cannot say ahead of time which of the known possibilities will occur.
Each distinct result of a random experiment is an outcome. When we group outcomes together into something we want to measure, that group is an event. A probability is then a number attached to an event that says how likely that event is to occur, on a fixed scale from 0 to 1. A probability of 0 means the event cannot happen; a probability of 1 means it is certain. Everything between is a matter of degree, and the whole subject is about assigning and manipulating those in between numbers consistently.
So what does a probability really mean when it is, say, 0.30? It means that if the same uncertain situation could be repeated many times under identical conditions, the event would occur in roughly 30 percent of them. That intuition, the long run frequency, is the anchor to return to whenever a probability calculation stops making sense.
Sample Space and Event Space
The sample space, written as the capital Greek letter omega or simply as S, is the set of every possible outcome of a random experiment. It has to be complete, meaning no possible result is left out, and its outcomes have to be distinct, meaning no two overlap. If you roll one ordinary die, the sample space is the set of the numbers 1 through 6. If you observe a single corporate bond over one year and record only whether it defaults, the sample space has just two outcomes: default and no default.
An event is any subset of the sample space, that is, any collection of outcomes you choose to group and measure. Rolling an even number is an event containing the outcomes 2, 4, and 6. An event can be a single outcome, several outcomes, the whole sample space, or even the empty set that contains no outcomes at all. The collection of all the events we might consider is sometimes called the event space. The key mental picture is nesting: outcomes live inside the sample space, and events are simply the groupings we draw around those outcomes.
Because events are sets, the language of sets applies directly. The union of two events, written A ∪ B, is the event that A or B or both occur. The intersection, written A ∩ B, is the event that A and B both occur. The complement of A, written Ac, is the event that A does not occur. These three operations, or, and, and not, are enough to express every combination of events you will meet.
Keep the three set operations tied to plain words: union is or, intersection is and, complement is not. Most probability mistakes are really language mistakes, where a candidate computes an “and” when the question asked for an “or”. Translate the sentence into set language before reaching for a formula.
Three Ways to Interpret a Probability
A probability is always a number between 0 and 1, but there is more than one way to decide what that number should be. Three interpretations are standard, and each is appropriate in different situations.
The classical, or a priori, interpretation applies when a random experiment has a finite number of outcomes that are equally likely by the symmetry of the setup. Then the probability of an event is simply the count of outcomes favourable to it divided by the total count of outcomes. A fair die gives each face a probability of one sixth because the six faces are symmetric. Classical probability needs no data; it comes from the structure of the problem itself.
The empirical, or relative frequency, interpretation applies when outcomes are not obviously equally likely, but you have observed the experiment many times. The probability of an event is estimated by the fraction of past trials in which it occurred. If, over the last 500 years of comparable single B rated issuers, about 4 percent defaulted within a year, then 0.04 is an empirical estimate of the one year default probability. Almost all probabilities used in risk are empirical, because real financial outcomes are rarely symmetric.
The subjective interpretation applies when the experiment cannot be repeated and there is no natural symmetry, so the probability expresses a considered degree of belief. The chance that a specific central bank cuts rates at its next meeting is subjective: it happens once, and reasonable analysts may assign different numbers. A subjective probability is still required to obey all the usual rules; it is personal, but it is not lawless.
| Interpretation | Based on | When it applies | Risk example |
|---|---|---|---|
| Classical | Symmetry and counting | Finite, equally likely outcomes | Each face of a fair die |
| Empirical | Observed frequencies | Repeatable process with history | Historical default rate by rating |
| Subjective | Reasoned degree of belief | One off events, no symmetry | Chance of a specific policy decision |
So what unites them? Whichever route produced the number, once it exists it must satisfy the same three rules, which we turn to next. The interpretation tells you how to get a probability; the axioms tell you how it must behave afterwards.
The Fundamental Principles of Probability
Every valid probability, no matter how it was arrived at, obeys three basic principles, usually called the axioms of probability. They are almost obvious, but they are the foundation everything else is derived from, so it pays to state them precisely.
where P(A) is the probability of event A. Axiom 1: every probability lies between 0 and 1 inclusive.
where S is the sample space. Axiom 2: the probability that some outcome in the sample space occurs is 1, because the sample space is exhaustive.
where A ∪ B is the event that A or B occurs. Axiom 3: for events that cannot happen together, the probability of either one is the sum of their probabilities.
From these three, useful results follow immediately. Because an event and its complement together make up the whole sample space and cannot both happen, their probabilities must add to 1. That gives the complement rule, which is often the fastest path to an answer.
where Ac is the complement of A, the event that A does not occur.
When two events can overlap, the third axiom does not apply directly, because adding P(A) and P(B) counts the overlap twice. Subtracting it once corrects this and gives the general addition rule.
where P(A ∩ B) is the joint probability that both A and B occur. When A and B are mutually exclusive this term is 0 and the rule collapses to Axiom 3.
Adding P(A) and P(B) without subtracting the overlap. This only works when the events are mutually exclusive. If a portfolio can breach a limit for two reasons that can both occur, adding the two breach probabilities overstates the true chance of a breach, because the case where both happen is counted twice.
Mutually Exclusive Events
Two events are mutually exclusive, also called disjoint, if they cannot both occur on the same trial. In set language their intersection is empty, so the joint probability is zero: P(A ∩ B) = 0. A single bond, rated once, cannot be both AAA and BBB at the same moment; a coin toss cannot land heads and tails together. When events are mutually exclusive, the general addition rule loses its overlap term and the probability of one or the other is just the sum.
A rating committee will assign one, and only one, internal grade to a new borrower. Based on the analyst’s file, the committee’s probabilities are 0.15 for Grade 1, 0.45 for Grade 2, 0.30 for Grade 3, and 0.10 for Grade 4. What is the probability that the borrower receives Grade 1 or Grade 3?
- Recognise that a borrower gets exactly one grade, so the four grade events are mutually exclusive.
- Because they are mutually exclusive, the probability of Grade 1 or Grade 3 is the plain sum of the two probabilities.
- Add: 0.15 + 0.30 = 0.45.
Answer: 0.45. So what does this mean? Because a borrower cannot hold two grades at once, there is no overlap to remove, and the chance of landing in either of the two grades is simply the combined weight of those grades, 45 percent.
Exhaustive Events
A set of events is exhaustive if together they cover the entire sample space, so at least one of them must occur. The four rating grades in the previous example are exhaustive because every borrower must receive one of them, and their probabilities sum to 0.15 + 0.45 + 0.30 + 0.10 = 1. When a set of events is both mutually exclusive and exhaustive, it is called a partition of the sample space: the events do not overlap and they leave nothing out. Partitions matter because the total probability theorem later in this chapter is built on exactly this structure.
Notice that mutually exclusive and exhaustive are separate properties. Two events can be mutually exclusive without being exhaustive, for example rolling a 1 and rolling a 2 on a die, which do not overlap but also do not cover the outcomes 3 through 6. They can be exhaustive without being mutually exclusive, for example rolling an even number and rolling a number 3 or higher, which together cover every face but overlap on 4 and 6. Only when both hold at once do you have a clean partition.
If a list of events is mutually exclusive and exhaustive, its probabilities must sum to exactly 1. This is a quick sanity check on any problem that hands you a set of scenario probabilities: if they do not add to 1, either a scenario is missing or the scenarios overlap.
Independent Events
Two events are independent if the occurrence of one carries no information about whether the other occurs. Formally, A and B are independent when their joint probability equals the product of their individual probabilities.
where P(A ∩ B) is the probability both events occur. Independence is precisely the condition that makes this product rule valid.
Here is the distinction that the exam loves to test. Mutually exclusive is not the same as independent, and the two are usually incompatible. If two events are mutually exclusive and both have positive probability, then knowing one occurred tells you the other definitely did not, since they cannot happen together. That is the strongest possible dependence, the opposite of independence. So two events with non zero probabilities cannot be both mutually exclusive and independent. Mutually exclusive is about whether events can coexist; independent is about whether they carry information about each other. Different questions, different answers.
A firm runs two trading desks whose overnight systems fail for unrelated technical reasons. The probability that Desk A has an outage on a given night is 0.05, and for Desk B it is 0.08. Treating the two outages as independent, find the probability that both desks are down on the same night, and the probability that at least one desk is down.
- Both down is the intersection. Under independence, multiply: 0.05 × 0.08 = 0.004.
- At least one down is the complement of neither being down. The probability Desk A stays up is 1 − 0.05 = 0.95, and Desk B stays up is 1 − 0.08 = 0.92.
- Neither down, again by independence, is 0.95 × 0.92 = 0.874.
- At least one down is 1 − 0.874 = 0.126.
Answer: both down 0.004, at least one down 0.126. So what does this mean? Simultaneous failure is rare, four tenths of one percent, but the chance that the night is disrupted somewhere is far larger at about 12.6 percent, which is why an operations team plans for single desk incidents far more often than for a joint collapse.
Assuming independence because it makes the arithmetic easy. In markets, defaults and losses tend to cluster in bad times, so events that look independent in calm periods become strongly dependent in a crisis. Multiplying probabilities as if events were independent, when they are not, understates the chance of many things going wrong together. Always ask whether independence is justified before you use the product rule.
Unconditional and Conditional Probability
An unconditional probability, P(A), is the probability of an event with no side information assumed. It is sometimes called a marginal probability. A conditional probability, written P(A | B) and read “the probability of A given B”, is the probability that A occurs once we already know that B has occurred. Conditioning on B means we throw away every part of the sample space where B did not happen and re calculate A’s chance inside the smaller world where B is true.
where P(A ∩ B) is the joint probability of A and B, and P(B) is the probability of the conditioning event. The division re-scales the joint probability so that it is measured relative to the world in which B occurs.
The formula is worth reading slowly, because it is the single most important relationship in the chapter. The conditional probability is the joint probability of both events divided by the probability of the one you are conditioning on. Intuitively, out of all the ways B can happen, you are asking what fraction also involve A. If B is very likely and A almost always accompanies it, P(A | B) is high; if A and B rarely coincide relative to how often B occurs, it is low.
Independence now has a clean second meaning. A and B are independent exactly when P(A | B) = P(A), that is, when conditioning on B does not change the probability of A at all. This is the same idea as the product rule from before: if P(A | B) = P(A), then P(A ∩ B) = P(A | B) P(B) = P(A) P(B). The two definitions of independence are two faces of one fact.
Watch the order of conditioning. P(A | B) and P(B | A) are different quantities and are rarely equal. A classic trap gives you the probability that a warning signal fires given a firm defaults, then asks for the probability the firm defaults given the signal fired. Those are not the same number, and converting between them requires Bayes theorem, covered at the end of this chapter.
Joint Probability and the Multiplication Rule
The joint probability P(A ∩ B) is the chance that both A and B occur on the same trial. Rearranging the conditional probability formula gives the multiplication rule, which is how joint probabilities are actually built in practice.
where either factorisation is valid. The joint probability is a conditional probability multiplied by the probability of the conditioning event.
The rule says you may multiply two probabilities to get a joint probability, but the second factor must be a conditional probability, not an unconditional one, unless the events are independent. Only under independence does P(A | B) reduce to P(A), letting you multiply the two plain probabilities. This is the precise home of the “when can I just multiply” question that trips candidates up.
In a small two obligor book, the probability that Obligor X defaults over one year is 0.10. Defaults here are not independent, because the two firms share a customer. Given that X defaults, the probability that Obligor Y also defaults rises to 0.40. What is the probability that both default within the year?
- Identify what is given: P(X defaults) = 0.10, and P(Y defaults | X defaults) = 0.40.
- Apply the multiplication rule with the conditional factor: P(both) = P(Y | X) × P(X).
- Multiply: 0.40 × 0.10 = 0.04.
Answer: 0.04. So what does this mean? Even though each firm alone is only moderately risky, the shared exposure makes a joint default a real 4 percent event. Had you wrongly assumed independence, you would have multiplied 0.10 by Y’s standalone default probability, which is smaller, and understated the joint risk. The conditional factor is what captures the linkage.
Events A and B have P(A) = 0.5, P(B) = 0.4, and P(A ∩ B) = 0.2. Are A and B independent? Are they mutually exclusive?
Show answer
Check independence by comparing P(A ∩ B) with P(A) times P(B): 0.5 × 0.4 = 0.2, which equals the given joint probability, so A and B are independent. They are not mutually exclusive, because a mutually exclusive pair would need P(A ∩ B) = 0, and here it is 0.2. This is a clean illustration that independent events not only can, but here do, occur together.
Conditional Independence
Two events can be dependent overall yet become independent once you condition on a third event. A and B are conditionally independent given C if, within the world where C is true, knowing A tells you nothing further about B.
where all probabilities are conditioned on event C. Independence holds inside C even if A and B are dependent when C is ignored.
This idea is central to how risk models are built. Two firms’ defaults are dependent when you look across all economic conditions, because both are dragged down together in a recession. But if you fix the state of the economy, that is, condition on C being “recession” or “expansion”, their remaining default risk may be treated as independent, driven only by firm specific factors. The common driver, the economy, is what created the dependence, and conditioning on it removes the shared part. Many credit portfolio models rest on exactly this assumption of conditional independence given a common factor.
Treating conditional independence and ordinary independence as the same. Events that are independent given the economic state are usually not independent when the state is unknown, because the shared state induces correlation. Assuming plain independence in that setting again understates the chance of joint losses.
The Total Probability Theorem
Often you can find the probability of an event inside each of several scenarios but you want its overall, unconditional probability. The total probability theorem does this by taking a probability weighted average of the conditional probabilities across a partition of the sample space, that is, across a set of scenarios that are mutually exclusive and exhaustive.
where S1 through Sn form a partition of the sample space: mutually exclusive and exhaustive scenarios. Each term weights the probability of A within a scenario by how likely that scenario is.
The structure mirrors how risk teams actually think. You rarely know a default rate outright, but you can estimate it separately for an expansion, for normal conditions, and for a recession, and you can attach a probability to each of those states. The total probability theorem stitches those pieces into one number.
An analyst splits the coming year into three mutually exclusive and exhaustive states and estimates a portfolio’s default rate in each.
| Economic state | Probability of state | Default rate in state |
|---|---|---|
| Expansion | 0.30 | 1% |
| Normal | 0.50 | 3% |
| Recession | 0.20 | 9% |
- Confirm the states form a partition: the probabilities 0.30 + 0.50 + 0.20 = 1, and a year falls in exactly one state.
- Weight each state’s default rate by the probability of that state: (0.30 × 0.01) + (0.50 × 0.03) + (0.20 × 0.09).
- Compute each term: 0.003, 0.015, and 0.018.
- Add: 0.003 + 0.015 + 0.018 = 0.036.
Answer: 3.6 percent. So what does this mean? The single unconditional default rate of 3.6 percent already bakes in the small chance of a severe recession. Notice it sits above the normal state rate of 3 percent, pulled up by the heavy 9 percent recession tail even though a recession is only 20 percent likely. Averaging the three rates without weighting would have been wrong; the weights are what make the number honest.
Using Exhibit 2, suppose an analyst had instead just averaged the three default rates as (1% + 3% + 9%) divided by 3. What answer would that give, and why is it wrong?
Show answer
The simple average is (0.01 + 0.03 + 0.09) ÷ 3 = 0.0433, or 4.33 percent. It is wrong because it treats all three states as equally likely, giving the rare recession the same weight as the common normal state. The total probability theorem weights each rate by how probable its state actually is, which lowers the answer to 3.6 percent. Equal weighting only coincides with the correct answer when the states really are equally likely.
Bayes Theorem
The total probability theorem runs in one direction: from causes, the economic states, to an effect, the default rate. Very often we need to run the other way. We observe an effect, some piece of evidence, and want to infer the probability of the cause behind it. That reversal is exactly what Bayes theorem performs. It converts P(evidence | cause) into P(cause | evidence).
where P(B) is the prior probability of the cause before seeing the evidence, P(A | B) is the likelihood of the evidence when the cause is present, P(A) is the total probability of the evidence, and P(B | A) is the posterior, the updated probability of the cause after the evidence.
Read as a process, Bayes theorem takes a prior belief P(B), multiplies it by how strongly the evidence points to the cause, and divides by how common the evidence is overall, to produce a revised posterior belief P(B | A). The denominator P(A) is usually found with the total probability theorem, which is why the two results are taught together: one feeds the other.
A monitoring model raises a red flag on stressed borrowers. Historically, 6 percent of borrowers in a book default within a year. The model flags 80 percent of the borrowers who go on to default, but it also flags 10 percent of the borrowers who do not default. A borrower has just been flagged. What is the probability this borrower actually defaults?
- Write down the pieces. Prior default probability P(D) = 0.06, so P(no default) = 0.94. Likelihood of a flag given default P(F | D) = 0.80. Flag given no default P(F | no D) = 0.10.
- Find the overall probability of a flag using total probability: P(F) = (0.80 × 0.06) + (0.10 × 0.94) = 0.048 + 0.094 = 0.142.
- Apply Bayes theorem: P(D | F) = [ P(F | D) × P(D) ] ÷ P(F) = 0.048 ÷ 0.142.
- Compute: 0.048 ÷ 0.142 = 0.338.
Answer: about 33.8 percent. So what does this mean? A flag raises this borrower’s default probability from the base rate of 6 percent to roughly 34 percent, a large jump, yet the borrower is still more likely than not to survive. The reason is that defaulters are rare, so even a fairly accurate model produces many false alarms among the large healthy population. This is why a single warning signal informs, but does not by itself condemn.
Return to the three states of Exhibit 2 with priors 0.30 expansion, 0.50 normal, and 0.20 recession. Suppose a “weak quarter” for the portfolio occurs with probability 0.10 in an expansion, 0.30 in a normal state, and 0.70 in a recession. A weak quarter has just been observed. What is the probability the economy is in recession?
- Find the overall probability of a weak quarter with total probability: P(W) = (0.10 × 0.30) + (0.30 × 0.50) + (0.70 × 0.20) = 0.03 + 0.15 + 0.14 = 0.32.
- Apply Bayes theorem for the recession state: P(recession | W) = (0.70 × 0.20) ÷ 0.32 = 0.14 ÷ 0.32.
- Compute: 0.14 ÷ 0.32 = 0.4375.
Answer: about 43.8 percent. So what does this mean? Before the data, a recession was only 20 percent likely. A single weak quarter more than doubles that belief to about 44 percent, because weak quarters are far more typical of recessions than of the other states. Bayes theorem is the machinery that turned the prior of 20 percent into the posterior of 44 percent.
An anti fraud system reviews transactions of which 0.5 percent are genuinely fraudulent. The system catches 90 percent of true frauds, and it wrongly alerts on 3 percent of legitimate transactions. A transaction has triggered an alert. How likely is it to be fraud?
- Priors and likelihoods: P(fraud) = 0.005, P(legit) = 0.995, P(alert | fraud) = 0.90, P(alert | legit) = 0.03.
- Total probability of an alert: P(alert) = (0.90 × 0.005) + (0.03 × 0.995) = 0.0045 + 0.02985 = 0.03435.
- Bayes theorem: P(fraud | alert) = 0.0045 ÷ 0.03435.
- Compute: 0.0045 ÷ 0.03435 = 0.131.
Answer: about 13.1 percent. So what does this mean? Despite the system catching nine out of ten frauds, an alert corresponds to actual fraud only about 13 percent of the time. Because genuine fraud is so rare, the small 3 percent false alert rate applied to the huge legitimate population swamps the true positives. This is the base rate fallacy, and it is exactly why alert queues are triaged rather than blocked outright.
A Bayes problem is almost always a signal that you must first compute the denominator with the total probability theorem. Lay the numbers out as prior times likelihood for each branch, sum them to get P(evidence), then divide the relevant branch by that sum. Setting the branches out in a small table prevents the most common slip, which is forgetting the false positive branch entirely.
In Worked Example 5, what is the probability that a borrower who was not flagged still defaults? Interpret the result.
Show answer
The probability of no flag is 1 − 0.142 = 0.858. The joint probability of default and no flag is P(no flag | default) × P(default) = (1 − 0.80) × 0.06 = 0.20 × 0.06 = 0.012. So P(default | no flag) = 0.012 ÷ 0.858 = 0.014, about 1.4 percent. The interpretation is that a clean report cuts the default probability well below the 6 percent base rate, from 6 percent to under 1.5 percent, which is what makes the absence of a flag genuinely reassuring even though it is not a guarantee.
A risk officer says, “Two positions are mutually exclusive, so they must be independent.” Is this correct?
Show answer
No. Mutually exclusive events with positive probabilities are the opposite of independent. If one occurs, the other cannot, so learning about one gives maximum information about the other. Independence would require that knowing one tells you nothing about the other. The officer has merged two distinct ideas: mutual exclusivity is about whether events can coexist, independence is about whether they carry information.
Chapter Summary
- A random experiment has an unpredictable outcome drawn from a known sample space; an event is any subset of that sample space, and a probability is a number from 0 to 1 measuring an event’s likelihood.
- Probabilities can be classical (from symmetry), empirical (from observed frequencies), or subjective (a reasoned degree of belief), but all obey the same axioms.
- The three axioms: probabilities lie in 0 to 1, the sample space has probability 1, and for mutually exclusive events probabilities add. From them follow the complement rule and the general addition rule.
- Mutually exclusive events cannot both occur, so their joint probability is 0. Exhaustive events cover the whole sample space. A set that is both forms a partition.
- Independent events satisfy P(A and B) equal to P(A) times P(B). Mutual exclusivity and independence are different ideas and, for positive probabilities, cannot hold together.
- Conditional probability is P(A given B) equal to P(A and B) divided by P(B). Rearranged, it gives the multiplication rule for joint probabilities.
- Conditional independence means events are independent once a common factor is fixed, even if they are dependent overall; it underlies credit portfolio models.
- The total probability theorem finds an unconditional probability as a probability weighted average across a partition of scenarios.
- Bayes theorem reverses a conditional probability, updating a prior into a posterior after evidence, with the denominator supplied by the total probability theorem.
Frequently Asked Questions
What is the difference between the sample space and an event?
The sample space is the complete set of every possible outcome of a random experiment, so nothing outside it can happen. An event is any subset of that sample space, meaning it is a collection of one or more outcomes that we care about. In short, the sample space is everything that can occur, and an event is the particular something we are measuring the probability of.
What does it mean for two events to be mutually exclusive?
Two events are mutually exclusive when they cannot both occur on the same trial, so their intersection is empty and the probability of both happening together is zero. Because they never overlap, the probability that one or the other occurs is simply the sum of their individual probabilities. A single bond being rated exactly AAA and exactly BBB at the same time is an example, since one rating rules out the other.
Are mutually exclusive events also independent?
No, and they are almost always the opposite. If two events are mutually exclusive and each has a non zero probability, then learning that one occurred tells you the other definitely did not, which is the strongest possible dependence. Independence means one event carries no information about the other, so two events with positive probability cannot be both mutually exclusive and independent.
What is the difference between unconditional, conditional, and joint probability?
An unconditional probability, written P(A), is the chance of an event with no extra information assumed. A conditional probability, written P(A given B), is the chance of A once we already know B has happened, so it re-scales the world to only the cases where B is true. A joint probability, written P(A and B), is the chance that both events happen together on the same trial.
When can I multiply two probabilities together to get the joint probability?
In general the joint probability is P(A and B) equals P(A given B) times P(B), so you multiply a conditional probability by the probability of the conditioning event. You may multiply the two unconditional probabilities directly, P(A) times P(B), only when the events are independent, because independence is exactly the condition that makes P(A given B) equal to P(A). Multiplying raw probabilities without checking independence is one of the most common exam errors.
What is the total probability theorem used for?
The total probability theorem lets you find the overall probability of an event by splitting the world into a set of mutually exclusive and exhaustive scenarios, finding the probability of the event inside each scenario, and taking a probability weighted average across them. It is the natural tool whenever an outcome depends on which state of the world you are in, such as a default rate that differs across economic expansions, normal conditions, and recessions.
What does Bayes theorem actually do?
Bayes theorem reverses the direction of a conditional probability, turning the probability of evidence given a cause into the probability of the cause given the evidence. It combines a prior belief about the cause with how strongly the evidence points to it, and returns an updated, or posterior, belief. This is the formal way to revise a probability after new information arrives, which is why it underlies credit scoring, fraud detection, and model updating in risk.
What is the difference between P(A given B) and P(B given A)?
They condition on different things and are usually not equal. P(A given B) assumes B is known and asks about A, while P(B given A) assumes A is known and asks about B. Confusing the two is called the inversion fallacy, for example reading the probability that a test flags a fraud given the account is fraudulent as if it were the probability the account is fraudulent given a flag. Bayes theorem is precisely the rule that converts one into the other.
