CFA Level 1 · Module 02 Quantitative Methods · Chapter 10
Much of finance is the search for relationships: does a stock move with the market, do sales rise with advertising, does a bond spread widen with leverage? Simple linear regression is the tool that turns such a question into a line, a single equation that summarizes how one variable tends to change when another does. This reading builds that line, tests how well it fits, and puts it to work in the most famous application of all, estimating a stock’s sensitivity to the market.
The reading has a natural order. First, the model and how its line is fitted by least squares. Second, how to judge the line: the share of variation it explains, the typical size of its errors, and the formal test of whether the relationship is real. Third, the assumptions that must hold for the results to be trusted, and the residual checks that reveal when they do not. Finally, prediction, alternative functional forms, and the capital asset pricing model as a regression.
The calculations are light but must be exact, and the vocabulary is dense. The exam rewards candidates who can estimate a slope and intercept, interpret R-squared correctly, spot a violated assumption, and read the coefficients of a market-model regression as an alpha and a beta.
Regression describes how a dependent variable, the one you want to explain, responds on average to an independent variable, the one you use to explain it. The dependent variable is often written Y and the independent variable X. In finance X might be the market’s return and Y a stock’s return, or X a company’s leverage and Y its credit spread. Regression does not prove that X causes Y; it measures how they move together and packages that co-movement into a line that can be interpreted and used for prediction.
Calling it simple linear regression signals two things. Simple means there is one independent variable, as opposed to multiple regression with several. Linear means the relationship is modeled as a straight line, so a one-unit change in X is associated with the same change in Y wherever you are on the line. These are strong simplifications, but they make the method transparent and are often a reasonable first description of a relationship, which is why regression is the starting point for so much of quantitative finance.
It is hard to overstate how widely this one tool is used. Estimating a stock’s beta, testing whether a factor explains returns, relating a bond’s spread to its rating, forecasting sales from an economic indicator, all are simple regressions at heart, and multiple regression, met at higher levels, is the same idea with more independent variables. Learning to fit, read, and criticize a single-variable regression is therefore the foundation for a large share of applied quantitative finance, which is why the curriculum spends a full reading on it despite the light arithmetic.
The model states that each observed Y equals a straight-line function of X plus an error. The line has an intercept and a slope, and the error captures everything about Y that X does not explain, the countless other influences bundled into a single random term.
where Yi is the dependent variable, Xi is the independent variable, b0 is the intercept, b1 is the slope, and εi is the error term for observation i, the vertical distance from the point to the true line.
The intercept and slope are the population parameters the regression seeks to estimate. Once estimated from data, they give a fitted line that predicts Y for any X. The error term is central rather than incidental: the whole theory of regression is really a set of assumptions about how these errors behave, and the credibility of the results stands or falls on those assumptions, examined later in the reading.
The error term deserves respect because it is where honesty about the model lives. No straight line will pass through every point, and the errors admit that the world is noisier than the model. They stand in for every influence on Y that X leaves out, from other economic forces to pure randomness. The assumptions examined later are entirely statements about these errors, and when the errors misbehave, when they trend, fan out, or cluster, the tidy results of the regression can no longer be trusted, however good the line looks.
Given a scatter of points, infinitely many lines could be drawn through them. The least squares criterion picks the one that makes the errors as small as possible in a specific sense: it minimizes the sum of the squared vertical distances from the points to the line. Squaring the distances means positive and negative misses do not cancel and large misses are penalized more, and it yields a unique best line with clean formulas for its slope and intercept.
where b1 is the estimated slope, b0 is the estimated intercept, and X̄ and Ȳ are the sample means. The slope is the covariance of X and Y divided by the variance of X; the intercept forces the line through the point of means.
Setup. Four paired observations of an independent variable X and a dependent variable Y are (2, 5), (4, 9), (6, 11), and (8, 15). The means are X̄ = 5 and Ȳ = 10. Estimate the regression line.
Answer: the fitted line is Ŷ = 2 + 1.6X. For each one-unit rise in X, Y is estimated to rise by 1.6 units, and when X is zero the line predicts a Y of 2. Those two numbers, slope and intercept, are the entire output of the fit.
The slope is the covariance of X and Y divided by the variance of X, which ties this reading straight back to the statistics chapter. Covariance sets the direction and raw strength of the relationship, and dividing by the variance of X rescales it into the change in Y per unit of X. Seeing the slope as a rescaled covariance makes its formula something to understand rather than memorize.
It is worth noticing what the least squares line always does: it passes through the point of the two means, and its residuals sum to zero. Those are not coincidences but consequences of the criterion. Because the line is anchored at the average of X and the average of Y, and because it balances the positive and negative misses, the fitted line is in a precise sense the balance point of the data. This is also why adding a wildly atypical observation can swing the line noticeably, since least squares, by squaring the misses, pays outsized attention to points far from the rest.
The slope is the heart of the regression. It is the estimated change in the dependent variable for a one-unit change in the independent variable, and its sign tells the direction of the relationship. A slope of 1.6 says Y rises 1.6 units for every one-unit rise in X; a negative slope would say they move oppositely. The slope is where the economic meaning lives, which is why interpreting it correctly, in the units of the actual variables, matters as much as computing it.
The intercept is the predicted value of Y when X is zero. Sometimes this is meaningful, and sometimes X can never realistically be zero, in which case the intercept is just the mathematical anchor that positions the line rather than a quantity to interpret on its own. In the market model examined later, the intercept has a famous interpretation as alpha, the return not explained by the market, so whether the intercept matters depends entirely on the application.
A fitted line is only useful if it actually explains the data, so regression decomposes the variation in Y into a part the line explains and a part it does not. The total sum of squares measures the total variation of Y around its mean. It splits into the regression sum of squares, the variation the line explains, and the sum of squared errors, the variation left unexplained in the residuals.
where SST is the total sum of squares, SSR is the regression (explained) sum of squares, SSE is the sum of squared errors (unexplained), and R2 is the coefficient of determination, the fraction of the total variation in Y that the regression explains.
R-squared ranges from zero to one and is read as a percentage of variation explained. An R-squared of 0.90 means the line accounts for 90% of the variation in Y, leaving 10% to everything else. Higher is better, but a high R-squared does not prove the model is correct or that X causes Y; it only says the line fits the observed points closely. In simple regression, R-squared also equals the square of the correlation between X and Y, tying this measure back to the statistics reading.
Setup. For the fitted line Ŷ = 2 + 1.6X on the four points above, the total sum of squares is 52 and the sum of squared errors is 0.8. Find R-squared.
Answer: an R-squared of about 0.985. Nearly all the variation in Y is captured by X, and only 1.5% is left in the residuals. Real financial regressions rarely fit this well; a market-model R-squared for a single stock is often well below one half.
A high R-squared is not a certificate of correctness. It says the line fits the observed points closely, nothing more. A model can fit beautifully and still be misspecified, driven by a lurking third variable, or useless for prediction outside the sample. Treat R-squared as one piece of evidence about fit, never as proof that the model is right or that X drives Y.
Remember that in simple regression R-squared equals the square of the correlation between X and Y. A correlation of 0.7 implies an R-squared of 0.49, so about half the variation is explained. Questions sometimes give the correlation and ask for R-squared, or the reverse, and the square-and-square-root relationship is the whole trick.
R-squared reports the proportion of variation explained, but not the typical size of the errors in the units of Y. The standard error of the estimate fills that gap. It is essentially the standard deviation of the residuals, a measure of how far, on average, the actual points fall from the fitted line. A smaller standard error of the estimate means the points hug the line tightly and predictions are more precise.
where se is the standard error of the estimate, SSE is the sum of squared errors, and n is the number of observations. The divisor is n minus 2 because two parameters, the slope and the intercept, were estimated from the data.
The two goodness measures answer different questions and belong together. R-squared says what fraction of the variation the model explains, a unit-free proportion. The standard error of the estimate says how big the leftover errors are, in the units of Y. A model can have a respectable R-squared yet still make predictions with an uncomfortably wide error, so a careful analyst reports both rather than leaning on either alone.
The analysis of variance, or ANOVA, table organizes the sums of squares into a formal test of whether the regression as a whole is significant. It lists the explained and unexplained variation, their degrees of freedom, and the mean squares obtained by dividing each sum of squares by its degrees of freedom. The ratio of the two mean squares is an F-statistic, which tests the null hypothesis that the slope is zero, that is, that X explains nothing.
| Source | Sum of squares | Degrees of freedom | Mean square |
|---|---|---|---|
| Regression (explained) | SSR | 1 | SSR ÷ 1 |
| Error (unexplained) | SSE | n − 2 | SSE ÷ (n − 2) |
| Total | SST | n − 1 | – |
A large F-statistic, with a correspondingly small p-value, means the explained variation is large relative to the unexplained, so the null of no relationship is rejected and the regression is deemed significant. In simple regression the F-test and a t-test of the slope give the same conclusion, because there is only one independent variable, but the ANOVA framework becomes essential in multiple regression, which is why it is introduced here.
The F-test has a clean intuition. Its numerator is the variation the line explains, per degree of freedom, and its denominator is the variation it leaves unexplained, per degree of freedom. When the line explains a lot relative to what it misses, the ratio is large and the relationship is judged real; when the line explains little more than noise, the ratio is near one and the null of no relationship survives. In simple regression this F-test and the t-test that asks whether the single slope differs from zero always agree, because with one independent variable there is only one relationship to test.
Least squares gives a line for any data, but the line’s statistical properties, the tests and intervals, depend on four assumptions about the errors. Linearity: the true relationship is a straight line. Independence: the errors are uncorrelated with each other. Homoskedasticity: the errors have constant variance across all levels of X. Normality: the errors are normally distributed. When these hold, the least squares estimates are reliable and the usual tests are valid.
The way to check the assumptions is to examine the residuals, the differences between the actual and fitted values. A plot of residuals against X should look like a formless cloud with no pattern. A curve in the residuals signals that the true relationship is not linear. A fan shape, with the spread widening as X grows, signals heteroskedasticity, non-constant error variance. A drift or cycle signals that the errors are not independent. Reading residual plots is how an analyst catches a broken assumption that the coefficients alone would never reveal.
Trusting the regression output without checking the residuals. A high R-squared can coexist with a badly curved or fanning residual pattern, meaning the linear model is wrong even though it appears to fit. The coefficients and R-squared do not announce a violated assumption; only a look at the residuals does.
A fitted line predicts the dependent variable for any chosen value of the independent variable, simply by plugging X into the equation. But a point prediction is incomplete, because the true Y will scatter around the line. A prediction interval puts a range around the point forecast, built from the standard error of the estimate and the chosen confidence level, expressing how much the actual outcome might differ from the prediction.
Setup. Using the fitted line Ŷ = 2 + 1.6X, predict Y when X equals 10, and describe how a prediction interval would frame that estimate.
Answer: a point prediction of 18, best reported with a prediction interval around it. Note also that predicting at X equals 10 extends beyond the observed range of 2 to 8; such extrapolation is risky, because the linear relationship may not hold outside the data used to fit it.
Beware extrapolation. A regression is only evidence about the range of X it was fitted on. Predicting far outside that range assumes the straight-line relationship continues where you have no data, which is exactly where it is most likely to break. If a question predicts at an X well beyond the sample, the intended point is often that the prediction is unreliable.
A prediction interval is also not the same width everywhere. It is narrowest near the average of X, where the data are densest and the line is best pinned down, and it widens as the prediction moves toward the edges of the observed range and beyond. This is the formal reason extrapolation is doubly dangerous: not only might the straight-line relationship fail outside the data, but even if it holds, the uncertainty around the prediction grows the further you travel from the center of the sample. A point forecast that ignores this widening understates how little is really known far from the data.
Not every relationship is a straight line in the raw variables, but many become straight after a transformation, and regression can still be used. Taking the natural logarithm of one or both variables produces different functional forms. A log-linear form, with Y logged, models a constant percentage change in Y for a unit change in X, useful for growth. A linear-log form, with X logged, models a constant change in Y for a percentage change in X. A log-log form, with both logged, gives a slope that is an elasticity, the percentage change in Y for a percentage change in X.
The value of these forms is that they let a straight-line method describe curved relationships, provided the curvature matches the transformation. Choosing the right form is guided by theory and by the residual plots: if a linear fit leaves a curved residual pattern, a logarithmic form often straightens it. The mechanics of the regression are unchanged; only the variables entering it are transformed, and the interpretation of the slope shifts accordingly, from a level change to a percentage change or an elasticity.
| Form | Variables | Slope interpretation |
|---|---|---|
| Linear | Y on X | Unit change in Y per unit change in X |
| Log-linear | log Y on X | Percentage change in Y per unit change in X |
| Linear-log | Y on log X | Unit change in Y per percentage change in X |
| Log-log | log Y on log X | Elasticity: percentage change in Y per percentage change in X |
The reading’s headline application is the market model, a regression of a stock’s returns on the market’s returns, which estimates the stock’s beta. Beta is the slope: it measures how much the stock’s return moves for a one-unit move in the market’s return, and it is the standard measure of a stock’s systematic risk. A beta of one moves with the market, above one is more volatile than the market, and below one is less. The intercept of this regression is the stock’s alpha, the average return not explained by market movements.
where E(Ri) is the expected return on the asset, Rf is the risk-free rate, βi is the asset’s beta, and E(Rm) is the expected market return, so that E(Rm) minus Rf is the market risk premium.
Beta measures only systematic risk, the part of a stock’s movement tied to the market. The regression’s residuals, the scatter around the line, are the stock’s specific risk, which diversification can remove. This is why the market rewards beta but not the residual variation: beta captures the risk you cannot diversify away, while the leftover is risk you should not be paid for holding.
Reading a market-model alpha as skill without checking its reliability. A positive intercept looks like return earned beyond the market, but a single regression’s alpha carries an estimation error, and a small or noisy alpha may be indistinguishable from zero. Judge alpha against its standard error before calling it evidence of a manager beating the market.
Setup. A regression of a stock’s returns on the market’s returns gives a beta of 1.6. The risk-free rate is 3% and the expected market return is 10%. Find the stock’s expected return under the capital asset pricing model.
Answer: an expected return of 14.2%. The beta of 1.6, estimated as a regression slope, says the stock is 60% more sensitive to the market than average, so it earns a proportionally larger risk premium. This is the bridge from a statistical regression to the pricing of risk that the asset-pricing readings develop.
Estimating beta in practice involves choices that affect the answer. The analyst must pick a return frequency, often monthly, a length of history, often several years, and a market index to stand in for the market. Different choices produce different betas, so a published beta is an estimate with a method behind it, not a fixed truth about the stock. Betas are also often adjusted toward one, on the empirical observation that extreme betas tend to drift toward the market average over time. For the exam the essential point is that beta is a regression slope, estimated with error and sensitive to how the regression was set up.
A regression gives a slope numerator (the sum of the products of the deviations) of 45 and a denominator (the sum of squared X deviations) of 30. What is the estimated slope?
b1 = 45 ÷ 30 = 1.5. For each one-unit rise in X, Y is estimated to rise by 1.5 units.
A regression has a total sum of squares of 200 and a sum of squared errors of 50. What is R-squared, and how is it read?
SSR = 200 − 50 = 150, so R2 = 150 ÷ 200 = 0.75. The regression explains 75% of the variation in the dependent variable, leaving 25% unexplained.
A residual plot shows the spread of residuals widening steadily as X increases. Which assumption is violated?
Homoskedasticity, the assumption of constant error variance. A fan shape indicates heteroskedasticity, where the errors grow larger for larger values of X, which undermines the usual standard errors and tests.
A stock has a beta of 0.8. The risk-free rate is 4% and the expected market return is 9%. What is its expected return under the CAPM?
4% + 0.8 × (9% − 4%) = 4% + 0.8 × 5% = 4% + 4% = 8%. With a beta below one, the stock earns less than the market’s return, reflecting its lower systematic risk.
Pulled together, the reading gives a disciplined way to move from a scatter of points to a defensible statement about a relationship. Fit the line, judge how much it explains and how large its errors are, test whether the relationship is real, check the assumptions in the residuals, and only then predict, mindful of the interval and of the edges of the data. The market model is the showcase because it turns this general machinery into the single most-used number in equity analysis, a stock’s beta, but the same steps apply to any two variables an analyst suspects are linked. Regression rewards the careful and punishes the credulous, and knowing its limits is as important as knowing its formulas.
No. Regression measures how X and Y move together, not why. A strong relationship can arise from causation, reverse causation, or a third variable driving both. Establishing causation requires argument and evidence beyond the fit itself.
The sum of the squared vertical distances from the data points to the line, the squared residuals. Squaring stops positive and negative errors from cancelling and penalizes large misses more, producing a single best-fitting line with clean formulas for its slope and intercept.
As the fraction of the variation in the dependent variable that the regression explains, from zero to one. An R-squared of 0.6 means 60% of the variation is explained and 40% is not. It measures fit, not correctness or causation, and a high value does not validate the model.
R-squared is a unit-free proportion of variation explained; the standard error of the estimate is the typical size of the residuals in the units of Y. One says how much is explained, the other how big the leftover errors are, and both are needed to judge a model.
By plotting the residuals. A patternless cloud supports the assumptions; a curve suggests non-linearity, a fan shape suggests non-constant variance, and a drift or cycle suggests the errors are not independent. The residuals reveal problems the coefficients hide.
Because the regression is only evidence about the range of X it was fitted on. Predicting far outside that range assumes the straight-line relationship continues where there is no data to support it, which is precisely where it is most likely to fail.
The slope is beta, the stock’s sensitivity to the market and its measure of systematic risk. The intercept is alpha, the average return not explained by the market. Together they summarize how a stock behaves relative to the market in a single regression.
Through estimating a slope and intercept, computing and interpreting R-squared, reading an ANOVA table, identifying a violated assumption from residuals, and interpreting beta and alpha from a market-model regression. The arithmetic is light; precise interpretation earns the marks.
Loading comments...
Add your Thoughts: