FRM Part 1 · Quantitative Analysis · Chapter QTA 4
Multivariate Random Variables
Learning Objectives
- Explain how a probability matrix can be used to express a probability mass function.
- Calculate the marginal and conditional distributions of a discrete bivariate random variable.
- Explain how the expectation of a function is calculated for a bivariate discrete random variable.
- Define covariance and explain what it measures.
- Explain the relationship between the covariance and correlation of two random variables, and how these are related to the independence of the two variables.
- Explain and illustrate the effects of applying linear transformations on the covariance and correlation between two random variables.
- Calculate the variance of a weighted sum of two random variables.
- Calculate the conditional expectation of a component of a bivariate random variable.
- Describe the features of an independent and identically distributed (iid) sequence of random variables.
- Explain how the iid property is helpful in calculating the mean and variance of a sum of iid random variables.
Risk almost never comes one variable at a time. A portfolio holds many positions, a bank runs many loan books, and the whole point of diversification is that these pieces do not move in lockstep. To model that, a single random variable is not enough; what is needed is a way to describe two or more random variables together, including how they move relative to one another. That is the job of a multivariate random variable.
This chapter works almost entirely with the two-variable, or bivariate, case, because everything important shows up there and the notation stays light. The tools carry over unchanged to three or more variables. Two threads run through the chapter. The first is description: the joint distribution and the marginal and conditional distributions read off it. The second is dependence: covariance and correlation, how they behave under rescaling, and how they drive the variance of a sum, which is the mathematics behind portfolio risk.
Key Takeaways
- A joint distribution, shown as a probability matrix, gives the probability of every combination of the two variables; the entries are non-negative and sum to 1.
- A marginal distribution is recovered by summing the matrix across the other variable; a conditional distribution divides a joint probability by a marginal.
- Covariance measures how two variables move together; correlation rescales it into the range from minus one to plus one, so it is comparable across variables.
- Independence forces zero correlation, but zero correlation does not force independence, because correlation sees only linear relationships.
- The variance of a weighted sum adds the individual variances plus a covariance term, and that covariance term is the source of diversification.
- For an iid sequence, the mean of a sum is n times the mean and the variance is n times the variance, so a sum’s standard deviation grows with the square root of n.
Joint Distributions and the Probability Matrix
A bivariate random variable is a pair of random variables treated together, written (X, Y). Its joint probability mass function gives the probability of each combination of values, the chance that X equals some value and Y equals some value at the same time. Because there are two variables, the natural way to display it is a grid: a probability matrix with the values of one variable down the rows and the other along the columns, and each cell holding the joint probability of that pair.
where p(x, y) is the joint probability of the pair (x, y). Like any PMF, every entry is between 0 and 1, and summing over the whole grid gives exactly 1, since some pair must occur.
One running example carries the chapter. A bank tracks defaults over a quarter in two loan books. Let X be the number of defaults in Book A, which can be 0, 1, or 2, and let Y be the number in Book B, which can be 0 or 1. Exhibit 1 is the probability matrix, with each cell the joint probability of that combination. The margins, the extra row and column, are the marginal distributions built in the next section.
| Y = 0 | Y = 1 | p(x) | |
|---|---|---|---|
| X = 0 | 0.30 | 0.10 | 0.40 |
| X = 1 | 0.20 | 0.15 | 0.35 |
| X = 2 | 0.10 | 0.15 | 0.25 |
| p(y) | 0.60 | 0.40 | 1.00 |
Marginal Distributions
A marginal distribution is the distribution of one variable on its own, ignoring the other. It is recovered from the joint matrix by summing over everything the other variable can be: sum each row to get the marginal of the row variable, and sum each column to get the marginal of the column variable. Those sums are exactly the margins printed in Exhibit 1, which is where the name comes from.
where pX(x) is the marginal PMF of X, formed by summing the joint probabilities across every value of Y. The marginal of Y is found the same way, summing across X.
From the margins, the marginal PMF of Book A defaults is 0.40, 0.35, 0.25 for 0, 1, 2 defaults, and Book B defaults are 0.60, 0.40 for 0, 1. Each is an ordinary univariate PMF: non-negative and summing to 1. The means follow directly, E[X] = 0.85 defaults and E[Y] = 0.40 defaults, and these anchor the covariance work later.
Conditional Distributions
A conditional distribution describes one variable once the other is known. If Book B is observed to have exactly one default, what is the distribution of defaults in Book A? The answer rescales a single column of the matrix so that it sums to 1, by dividing each joint probability by the marginal probability of the conditioning value.
where the numerator is the joint probability and the denominator is the marginal probability of the value being conditioned on. Dividing by that marginal is what makes the conditional probabilities sum back to 1.
Given that Book B has one default (Y = 1), find the conditional distribution of defaults in Book A.
Step 1. Take the Y = 1 column, (0.10, 0.15, 0.15), and its marginal, p(Y = 1) = 0.40.
Step 2. Divide each entry by 0.40.
Answer: the conditional distribution is (0.250, 0.375, 0.375), and it sums to 1. Compare it with the marginal (0.40, 0.35, 0.25): knowing Book B defaulted has shifted weight toward more defaults in Book A. The two books are not independent, which the covariance will confirm.
Expectation of a Function
The expected value of a function of both variables, g(X, Y), is the probability-weighted average of the function across the whole grid. Every cell contributes, weighted by its joint probability, even when the function happens to use only one of the variables. The double sum runs over both dimensions.
where g(x, y) is any function of the two values and p(x, y) is the joint probability of that cell. The sum weights each possible outcome of the function by how likely its cell is.
Using Exhibit 1, find E[XY], the expected value of the product of defaults in the two books. This quantity is the building block of the covariance.
Step 1. The product is 0 in every cell where X = 0 or Y = 0, so only the two cells with X ≥ 1 and Y = 1 contribute.
Answer: E[XY] = 0.45. The double sum has six terms, but four are zero because the product is zero whenever either book has no defaults, leaving only the two cells where both books took losses together.
Covariance: How Two Variables Move Together
Covariance measures whether two variables tend to move in the same direction. It is the expected product of their deviations from their means, and it has a shortcut form that mirrors the variance shortcut: the mean of the product minus the product of the means. A positive covariance means the two rise and fall together; a negative one means one rises as the other falls; zero means no linear tendency either way. Covariance is a generalization of variance, and in fact the covariance of a variable with itself is just its variance.
where μX and μY are the two means. The right-hand shortcut, the mean of the product minus the product of the means, is the quicker route once E[XY] is in hand.
Using E[XY] = 0.45 from Worked Example 2, and E[X] = 0.85, E[Y] = 0.40 from the marginals, find the covariance of defaults in the two books.
Step 1. Apply the shortcut form.
Answer: Cov(X, Y) = 0.11, a positive number, so the two books tend to default together, which matches the conditional distribution from Worked Example 1. The size, 0.11, is hard to read on its own because it carries the units of defaults times defaults; that is what correlation fixes next.
When there are several variables, the covariances are usually collected into a covariance matrix: a square grid with each variable’s variance down the leading diagonal and the covariance between each pair in the off-diagonal cells. The matrix is symmetric, because the covariance of X with Y is the same as the covariance of Y with X. This object is the standard input to portfolio risk: give a model the covariance matrix of a set of assets and the weights, and it can compute the variance of the whole portfolio in one step. For two variables it is just a two-by-two grid, but the same structure scales to hundreds of positions.
Correlation and Independence
Correlation rescales the covariance into a pure number by dividing it by the product of the two standard deviations. The result always lies between minus one and plus one, so it can be compared across any pair of variables, whatever their units. A correlation near plus one is a strong same-direction relationship, near minus one a strong opposite-direction one, and near zero no linear relationship.
where σX and σY are the standard deviations. Dividing by them cancels the units, which is why correlation is called a scale-free measure.
The bounds are worth understanding rather than just memorizing. A correlation of plus one means the two variables sit exactly on an upward-sloping straight line: knowing one pins down the other with no scatter at all. Minus one is the same perfect line sloping downward. Real financial variables almost never reach either extreme; a correlation of 0.6 between two equity indices is already considered high, and most pairs of assets sit well inside the bounds, which is what leaves room for diversification.
The standard deviations of the two books work out to σX = 0.792 and σY = 0.490 (from Var(X) = E[X²] − (E[X])² = 1.35 − 0.7225 = 0.6275, and Var(Y) = 0.40 − 0.16 = 0.24). Find the correlation.
Step 1. Divide the covariance by the product of the standard deviations.
Answer: a correlation of about 0.28, a mild positive relationship. The raw covariance of 0.11 was hard to judge; the correlation of 0.28 says plainly that the two books move together, but only weakly.
Treating zero correlation as proof of independence. Independence does force zero correlation, but the reverse fails: correlation measures only linear association, so two variables can be tightly linked in a curved way and still show zero correlation. The classic case is a standard normal and its own square. One completely determines the other, so they are as dependent as can be, yet their correlation is exactly zero. Independence is the stronger condition, and it requires the joint PMF to equal the product of the marginals in every cell.
Linear Transformations of Covariance and Correlation
Rescaling and shifting the variables, a routine step when changing units or currencies, has clean effects. Adding constants shifts locations and changes neither covariance nor correlation, because both are built from deviations around the mean. Rescaling multiplies the covariance by the product of the scale factors, so covariance is sensitive to scale. Correlation, by contrast, keeps only the signs of the scale factors.
where a and c shift and b and d rescale. The shifts drop out of both. Covariance scales by the product bd; correlation keeps only the signs, so two positive rescalings leave it unchanged and a single negative one flips its sign.
Correlation is immune to any positive rescaling of either variable. Measuring returns in percent or in basis points, or losses in dollars or in millions, leaves the correlation exactly the same. That invariance is precisely why correlation, not covariance, is the number quoted when the strength of a relationship is reported: it means the same thing regardless of the units the desk happens to use.
Exhibit 2 gathers the dependence toolkit in one place, since these four ideas are easy to blur together under exam pressure.
| Measure | Definition | Range | What it tells you |
|---|---|---|---|
| Covariance | E[XY] − E[X]E[Y] | any real | Direction of co-movement; size depends on units |
| Correlation | Cov / (σXσY) | −1 to +1 | Strength and direction; scale-free |
| Independence | p(x,y) = pX(x)pY(y) | — | No relationship of any kind; forces zero correlation |
| Variance of a sum | a²V(X) + b²V(Y) + 2ab·Cov | ≥ 0 | Combined risk, including the co-movement term |
The Variance of a Weighted Sum
Here the covariance earns its keep. The variance of a weighted sum of two variables is not simply the weighted sum of their variances; it carries a third term built from the covariance. That extra term is the entire mathematics of portfolio risk and diversification.
where a and b are the weights. When X and Y are independent, or merely uncorrelated, the covariance is 0 and the variance of the sum collapses to the sum of the scaled variances.
A portfolio puts 60 percent in asset A and 40 percent in asset B. Asset A has a variance of 0.04 (standard deviation 20 percent), asset B a variance of 0.01 (standard deviation 10 percent), and their covariance is 0.006 (a correlation of 0.30). Find the portfolio’s standard deviation.
Step 1. Apply the variance-of-a-sum formula with weights 0.6 and 0.4.
Step 2. Take the square root for the standard deviation.
Answer: a portfolio standard deviation of about 13.7 percent. The weighted average of the two volatilities is 0.6(20%) + 0.4(10%) = 16 percent, yet the portfolio comes in at 13.7 percent. That gap is diversification: because the correlation is below 1, the covariance term is smaller than it would be if the assets moved in lockstep, and the combined risk falls below the average of the parts.
Conditional Expectation
A conditional expectation is the mean of one variable once the other is known. It is an ordinary expected value, but computed with the conditional distribution rather than the marginal, so it answers questions like “what is the expected loss on this book, given the other book has already taken a hit”. In risk, conditioning on a bad outcome elsewhere is exactly how stress scenarios and expected shortfall are framed.
where pX|Y is the conditional PMF from earlier. The formula is the usual weighted average, but the weights are the conditional probabilities, so the answer reflects the world in which Y is already known.
Using the conditional distribution (0.250, 0.375, 0.375) from Worked Example 1, find the expected number of defaults in Book A given that Book B had a default.
Step 1. Weight each value of X by its conditional probability.
Answer: 1.125 expected defaults, up from the unconditional 0.85. Learning that Book B took a default raises the expected damage in Book A by about a third, which is the positive covariance expressed as a forward-looking number. This is the shape of a stress calculation: condition on the bad event, then read the expected loss.
Independent and Identically Distributed Sequences
The cleanest multivariate setting is a sequence of variables that are all cut from the same cloth and carry no information about one another. Such a sequence is called independent and identically distributed, or iid. “Identically distributed” means every variable has the same distribution, and therefore the same mean and variance. “Independent” means no variable tells you anything about any other, so every covariance between distinct members is zero. Treating a run of daily returns as repeated draws from one return distribution is the standard iid assumption.
Both halves of the assumption do work. “Identically distributed” is what lets a single mean and a single variance stand in for the whole sequence; without it, each variable could have its own moments and nothing would simplify. “Independent” is what removes the covariance terms; without it, the movements of one variable would feed into the variance of the sum through the cross terms seen earlier. Together they turn a messy multivariate problem into simple arithmetic in n.
The iid assumption makes sums easy. Because expectation is always additive, the mean of a sum of n iid variables is n times the common mean. Because independence kills every covariance term, the variance of the sum is just n times the common variance, with no cross terms to add.
where μ and σ² are the common mean and variance and n is the number of variables. The variance grows linearly with n, so the standard deviation grows with the square root of n, not with n itself.
A desk models 250 trading days as iid daily returns, each with a mean of 0.04 percent and a standard deviation of 1.2 percent. What are the mean and standard deviation of the annual return, the sum of the 250 daily returns?
Step 1. Mean of the sum: n times the daily mean.
Step 2. Standard deviation of the sum: the square root of n times the daily standard deviation.
Answer: an expected annual return of 10 percent with a standard deviation of about 19 percent. Note that the mean scaled by the full 250 while the risk scaled only by the square root of 250. This square-root-of-time rule is why annual volatility is far less than 250 times daily volatility, and it is one of the most used results in market risk.
Two variables have a correlation of exactly 0. A colleague concludes they must be independent. Give a case where this conclusion is wrong.
Show answer
Take a standard normal variable and its own square. The square is completely determined by the original variable, so the two are as dependent as possible, yet their correlation is exactly 0 because the relationship is symmetric and curved, not linear. Correlation measures only linear association, so zero correlation rules out a straight-line relationship but not dependence in general. Independence would require the joint distribution to factor into the product of the marginals, which fails here.
Two assets each have a return variance of 0.04. If they are combined with equal weights and their covariance is 0.02, what is the variance of the equally weighted portfolio? What if the covariance were 0?
Show answer
With weights of 0.5 each: Var = (0.5)²(0.04) + (0.5)²(0.04) + 2(0.5)(0.5)(0.02) = 0.01 + 0.01 + 0.01 = 0.03. If the covariance were 0, the last term vanishes and the variance would be 0.02. The positive covariance adds 0.01 of extra risk, which is exactly why correlated positions diversify less than uncorrelated ones.
Why is the conditional expectation E[X | Y = y] often more useful in risk management than the plain expectation E[X]?
Show answer
Because risk questions are usually about what happens once something bad has already occurred. The plain expectation averages over all states of the world, calm and stressed alike. The conditional expectation fixes the state, for example a large loss in a related book or a sharp market drop, and gives the expected outcome inside that state. Stress tests and expected shortfall are both conditional expectations: they ask for the expected loss given that a tail event has happened.
Daily returns are iid with a daily standard deviation of 1 percent. Under this assumption, what is the standard deviation over a 100-day horizon, and why is it not 100 percent?
Show answer
The variance of a sum of 100 iid variables is 100 times the daily variance, so the standard deviation is the square root of 100 times the daily standard deviation, which is 10 × 1 percent = 10 percent, not 100 percent. Risk scales with the square root of the horizon because variance, not standard deviation, adds across independent periods. Multiplying the daily standard deviation by the full 100 would double-count by ignoring that independent moves partly offset.
Chapter Summary
- A bivariate distribution is shown as a probability matrix, with non-negative entries that sum to 1; each cell is a joint probability p(x, y).
- A marginal distribution is found by summing the matrix across the other variable; a conditional distribution divides a joint probability by the marginal of the conditioning value.
- The expectation of a function g(X, Y) is the double sum of g(x, y) weighted by the joint probabilities.
- Covariance, E[XY] − E[X]E[Y], measures co-movement; correlation divides it by the two standard deviations to give a scale-free number between minus one and plus one.
- Independence implies zero correlation, but zero correlation does not imply independence, since correlation captures only linear association.
- Adding constants leaves covariance and correlation unchanged; rescaling multiplies covariance by the product of the scale factors, while correlation keeps only their signs.
- Var(aX + bY) = a²Var(X) + b²Var(Y) + 2ab·Cov(X, Y); the covariance term is the mathematics of diversification.
- A conditional expectation is the mean computed with the conditional distribution, and it is how stress scenarios and expected shortfall are framed.
- For an iid sequence, the mean of a sum is nμ and the variance is nσ², so a sum’s standard deviation grows with the square root of n and an average’s shrinks as one over the square root of n.
Frequently Asked Questions
What is the difference between a joint, a marginal, and a conditional distribution?
A joint distribution gives the probability of a specific combination of both variables at once, for example the probability that book A has one default and book B has none. A marginal distribution describes just one variable on its own, ignoring the other, and is found by summing the joint probabilities across everything the other variable can be. A conditional distribution describes one variable once the value of the other is known, and is found by dividing the joint probability by the marginal probability of the value being conditioned on.
What does covariance measure, and why is correlation usually reported instead?
Covariance measures how two variables move together: it is positive when they tend to rise and fall together, negative when one rises as the other falls, and zero when there is no linear relationship. Its drawback is that it depends on the units and scale of the variables, so its size is hard to interpret on its own. Correlation is the covariance divided by the product of the two standard deviations, which strips out the scale and forces the result between minus one and plus one, so it is comparable across any pair of variables.
If two variables are uncorrelated, are they independent?
No. Independence implies zero correlation, but zero correlation does not imply independence. Correlation only detects linear relationships, so two variables can be strongly related in a nonlinear way and still have zero correlation. A standard example is a variable and its own square: they are clearly dependent, since one determines the other, yet their correlation can be zero. Independence is the stronger property, and it requires the joint distribution to equal the product of the marginals everywhere.
How does a linear transformation change covariance and correlation?
Adding constants to the variables shifts their locations but leaves both covariance and correlation unchanged, because both are built from deviations around the mean. Rescaling the variables by constants multiplies the covariance by the product of those scale factors, so covariance is sensitive to scale. Correlation, however, only picks up the signs of the scale factors: rescaling by two positive numbers leaves the correlation identical, while a negative scale factor flips its sign. This is why correlation is called scale-free.
Why is the variance of a sum not just the sum of the variances?
The variance of a sum of two variables is the sum of their variances plus twice their covariance. The covariance term appears because the two variables can move together: when they are positively correlated, their swings reinforce each other and the combined variance is larger than the sum of the parts. Only when the variables are independent, or otherwise uncorrelated, does the covariance term vanish and the variance of the sum reduce to the sum of the variances. This covariance term is the mathematical root of portfolio diversification.
What does it mean for a sequence to be independent and identically distributed?
An independent and identically distributed, or iid, sequence is a set of random variables that all come from the same single distribution, so they share the same mean and variance, and that are mutually independent, so no one of them carries information about any other. It is the standard assumption for a clean sample of data, such as daily returns treated as repeated draws from one return distribution, and it makes the mathematics of sums and averages especially simple.
How do the mean and variance of a sum of iid variables scale with the sample size?
For a sum of n independent and identically distributed variables, the mean of the sum is n times the common mean and the variance of the sum is n times the common variance, because independence makes all the covariance terms zero. The standard deviation of the sum therefore grows with the square root of n, not with n. For the average of the same variables, the mean stays equal to the common mean while the variance shrinks to the common variance divided by n, which is why larger samples give more precise averages.
Loading comments...
Add your Thoughts: