FRM Part 1 · Quantitative Analysis · Chapter QTA 10
Stationary Time Series
Learning Objectives
- Describe the requirements for a series to be covariance stationary.
- Define the autocovariance function and the autocorrelation function.
- Define white noise, and describe independent white noise and normal (Gaussian) white noise.
- Define and describe the properties of autoregressive (AR) processes.
- Define and describe the properties of moving average (MA) processes.
- Explain how a lag operator works.
- Explain mean reversion and calculate a mean-reverting level.
- Define and describe the properties of autoregressive moving average (ARMA) processes.
- Describe the application of AR, MA, and ARMA processes.
- Describe sample autocorrelation and partial autocorrelation.
- Describe the Box-Pierce Q statistic and the Ljung-Box Q statistic.
- Explain how forecasts are generated from ARMA models.
- Describe the role of mean reversion in long-horizon forecasts.
- Explain how seasonality is modeled in a covariance-stationary ARMA.
A time series is data indexed by time: a sequence of interest rates, spreads, returns, or growth rates, one observation after another. What makes it different from the cross-sections of earlier chapters is memory. Today’s value is usually linked to yesterday’s, and that link is precisely what makes forecasting possible. Time-series analysis is the study of that memory, how to describe it, model it, and project it forward.
The foundation is stationarity, the condition that a series behaves consistently enough over time for its past to inform its future. On that foundation sit three model families, autoregressive (AR), moving average (MA), and their combination (ARMA), which between them capture most of the memory in economic and financial data. This chapter builds each one, shows how to read a series’ fingerprints, the autocorrelation and partial autocorrelation functions, to pick the right family, how to check the fit, and how to turn a fitted model into a forecast that gracefully reverts to the long-run mean.
Key Takeaways
- A series is covariance stationary when its mean and variance are constant and its autocovariance depends only on the lag, not on time.
- The ACF measures correlation at each lag; the PACF measures it after removing the intervening lags. Their shapes identify the model.
- White noise, mean zero with no autocorrelation, is the building block; AR, MA, and ARMA models are built from it.
- An AR(1) is stationary when its coefficient is below 1 in absolute value; it mean-reverts to δ / (1 − φ), more slowly the closer φ is to 1.
- AR models have a decaying ACF and a PACF that cuts off; MA models are the mirror image; ARMA models have both decaying.
- Forecasts are rolled forward with future shocks set to zero, and they revert to the unconditional mean as the horizon grows.
Covariance Stationarity
Covariance stationarity is the condition that lets historical data speak to the future. A series is covariance stationary when its statistical character does not drift over time, made precise by three requirements: a constant mean, a constant and finite variance, and an autocovariance that depends only on the distance between two observations, not on when they occur. Exhibit 1 states them.
| Requirement | Condition | Meaning |
|---|---|---|
| Constant mean | E[Yt] = μ for all t | No trend; the level does not drift |
| Constant finite variance | Var(Yt) = γ0 < ∞ | The spread is stable and bounded |
| Time-invariant autocovariance | Cov(Yt, Yt−h) = γh | Dependence depends only on the lag h, not on t |
Why insist on it? Because only a stationary series has relationships stable enough to estimate from the past and apply to the future. A non-stationary series, one that trends or whose volatility grows, produces estimates that never settle down and can generate spurious relationships, unrelated series that appear strongly linked simply because both wander over time. The next chapter deals with making non-stationary series stationary; this one assumes stationarity throughout.
Autocovariance, ACF, and PACF
The memory of a stationary series is summarized by how correlated it is with its own past. The autocovariance at lag h is the covariance between an observation and one h periods earlier; it is the time-series analog of ordinary covariance, applied to a series with itself. Dividing it by the variance gives the autocorrelation, which lies between minus one and plus one and is easier to interpret. Collected across all lags, the autocorrelations form the autocorrelation function, or ACF.
where γh is the autocovariance at lag h and γ0 the variance. The autocorrelation at lag 0 is always 1, and the function is symmetric, so the lag h and lag minus h values are equal.
The ACF has a companion, the partial autocorrelation function, or PACF. The autocorrelation at lag h blends the direct link between the two observations with the indirect links that run through the periods in between. The partial autocorrelation strips out those intermediate links and reports the direct correlation only. The distinction is what makes the two functions such a powerful diagnostic: as later sections show, AR and MA models leave opposite fingerprints on the ACF and the PACF, and reading those fingerprints is how a model is chosen.
White Noise
The simplest time series, and the atom from which the others are built, is white noise: a sequence with mean zero, constant finite variance, and zero autocorrelation at every lag. No value carries any information about any other, so white noise is pure, unforecastable randomness. It is automatically covariance stationary. Its importance is structural: every model in this chapter is a rule for turning a stream of white-noise shocks into a series with memory, and a fitted model is judged good when its leftover residuals look like white noise.
where the three conditions are mean zero, constant variance, and no autocorrelation. Nothing is assumed about the shape of the distribution, only about its first two moments and its lack of memory.
Three variants matter, and Exhibit 2 separates them. Independent white noise strengthens “uncorrelated” to fully independent. Gaussian, or normal, white noise adds that each value is normally distributed, a convenient but strong extra assumption often refuted by real returns. Dependent white noise is the subtle and important one: it is uncorrelated but not independent, which allows the level to be unpredictable while the volatility stays persistent. This is exactly the behavior of financial returns, and it is captured by ARCH-type models where quiet and turbulent periods cluster.
| Type | Property | Example use |
|---|---|---|
| White noise | Mean 0, constant variance, no autocorrelation | The generic building block |
| Independent white noise | Also fully independent across time | Clean simulation input |
| Gaussian white noise | Independent and normally distributed | Textbook models and tests |
| Dependent white noise | Uncorrelated but not independent | Volatility clustering (ARCH) |
Autoregressive (AR) Models
The most widely used time-series model is the autoregressive one, which regresses a series on its own past. A first-order autoregression, AR(1), links each value to the one immediately before it, plus a shock.
where δ (delta) is a constant, φ (phi) is the autoregressive coefficient measuring persistence, and εt is white noise. The AR(1) is covariance stationary only when the coefficient satisfies |φ| < 1; a coefficient of 1 or more produces a non-stationary, explosive series.
The coefficient governs everything. Its size sets the persistence: a value near 1 means shocks fade slowly and the series moves in long, smooth swings, while a value near 0 means the series snaps back almost at once. The mean, variance, and autocorrelations all follow from it. The mean is the constant divided by one minus the coefficient, and the ACF decays geometrically as the coefficient raised to the lag.
where the ACF, φ raised to the lag, decays smoothly toward zero when |φ| < 1. The PACF of an AR(1), by contrast, is non-zero only at the first lag and exactly zero thereafter, which is the signature used to identify it.
A monthly credit spread follows Yt = 0.5 + 0.8 Yt−1 + εt. Find the first two autocorrelations.
Step 1. The ACF of an AR(1) is the coefficient raised to the lag, with φ = 0.8.
Answer: the autocorrelations are 0.80 at one lag and 0.64 at two, decaying geometrically. This slow decay is the mark of a persistent series: a spread that is high this month is very likely still high next month, and only gradually forgets where it has been.
The order generalizes. An AR(p) regresses the series on its last p values, and its properties broadly mirror the AR(1): the ACF decays (possibly with oscillations) while the PACF cuts off after lag p. Stationarity of an AR(p) requires the sum of the coefficients to be below 1, a necessary condition that generalizes the |φ| < 1 rule.
Mean Reversion and the Long-Run Level
A stationary AR process is mean-reverting: whenever it wanders away from its long-run average, it is pulled back. That long-run average, the mean-reverting level, is the value the series gravitates to over time, and it is exactly the unconditional mean derived above, the constant divided by one minus the coefficient. The coefficient sets the speed of reversion: closer to 1 means a weaker pull and slower return; closer to 0 means a strong pull and a quick snap back.
where the level is the point of attraction for the series. A shock pushes the series away, and each subsequent period the series closes a fraction (1 − φ) of the remaining gap back to this level.
For the credit spread Yt = 0.5 + 0.8 Yt−1 + εt, find the mean-reverting level, and say what a coefficient of 0.8 implies about the speed of reversion.
Step 1. Apply the mean-reverting level formula, with δ = 0.5 and φ = 0.8.
Answer: the spread reverts to a long-run level of 2.5. With a coefficient of 0.8, each month the series closes only 20% of its gap to 2.5, so reversion is slow and a shock lingers for many months. Had the coefficient been 0.2, the series would close 80% of the gap each month and snap back almost immediately.
The Lag Operator
Writing higher-order models cleanly calls for a piece of notation, the lag operator, written L, which shifts a series one step back in time. Applying it once turns the current value into the previous one; applying it p times reaches back p periods; applied to a constant it leaves the constant unchanged. Strings of lag operators combine into lag polynomials that express an entire model compactly.
where L shifts the time index back by one, its power p shifts back by p, and it has no effect on a constant. An AR(1), for instance, becomes (1 − φL)Yt = δ + εt, with the whole autoregressive structure folded into the polynomial 1 − φL.
The lag operator is more than shorthand. Whether a lag polynomial can be inverted determines whether an AR process is stationary and whether an MA process is invertible, the two technical conditions that make a model well behaved and its parameters uniquely identified. For the exam, the essential point is mechanical fluency: recognize that L shifts time back, that powers stack, and that a model can be read off its lag polynomial.
Moving Average (MA) Models
Where an autoregression builds memory from past values, a moving average model builds it from past shocks. A first-order moving average, MA(1), makes the current value depend on the current shock and the one immediately before it.
where μ is the mean, θ (theta) weights the previous shock, and the ε terms are white noise. Because the mean of every shock is zero, the mean of the process is simply μ.
Two properties make the MA distinctive. First, a moving average is always covariance stationary, whatever its coefficients, because it is just a finite weighted sum of well-behaved white-noise terms; there is no stationarity condition to check. Second, it has a strictly limited memory: an MA(1) is correlated with its own past only one period back, because only the shared shock links two observations. Its ACF is non-zero at lag 1 and exactly zero at every lag beyond, the mirror image of the AR’s slowly decaying ACF.
where the single non-zero autocorrelation sits at lag 1 and everything beyond is zero. An MA(q) generalizes this: its ACF cuts off after lag q, while its PACF decays gradually, exactly reversing the AR pattern.
A series follows Yt = μ + εt + 0.6 εt−1. Find the autocorrelations at lags 1 and 2.
Step 1. The lag-1 autocorrelation is θ over 1 plus θ squared, with θ = 0.6.
Step 2. Beyond lag 1, the MA(1) ACF is zero.
Answer: an autocorrelation of about 0.44 at lag 1 and zero at lag 2 and beyond. The sharp cutoff after one lag is the fingerprint of an MA(1): the series remembers exactly one period of shock and nothing more, unlike the AR’s gradually fading memory.
ARMA Models
Combining the two families gives the autoregressive moving average model, ARMA, which lets a series depend on both its own past values and past shocks. A simple ARMA(1,1) adds one autoregressive term and one moving-average term.
where φ is the autoregressive coefficient and θ the moving-average coefficient. The mean is still δ / (1 − φ), and covariance stationarity depends only on the AR part, requiring |φ| < 1; the MA part never threatens stationarity.
The value of ARMA is parsimony: a mixed model can often match a series’ memory with fewer parameters than a pure AR or MA would need, which reduces the estimation error that comes with fitting many coefficients. The cost is that its fingerprint is less clean. An ARMA has both an ACF and a PACF that decay gradually, with neither cutting off sharply, so it is recognized precisely by the absence of a clean cutoff in either function.
Identifying a Model: ACF and PACF Signatures
Here the ACF and PACF earn their keep. Each model family leaves a distinctive pair of fingerprints, and matching the sample ACF and PACF of a series to these signatures is the standard first step in choosing a model. The rule is short enough to memorize and is a reliable exam question. Exhibit 3 states it, and Figure 2 draws it.
| Model | ACF | PACF |
|---|---|---|
| AR(p) | Decays gradually (may oscillate) | Cuts off after lag p |
| MA(q) | Cuts off after lag q | Decays gradually (may oscillate) |
| ARMA(p, q) | Decays gradually | Decays gradually |
Read the cutoff. A clean cutoff in the PACF after lag p, with a decaying ACF, says AR(p). A clean cutoff in the ACF after lag q, with a decaying PACF, says MA(q). No clean cutoff in either, both just decaying, says ARMA. This single reading rule turns a pair of plots into a shortlist of candidate models, which are then estimated and compared. In practice the choice is refined with information criteria such as the AIC and BIC, which penalize extra parameters to balance fit against the bias-variance tradeoff.
Checking Residuals: the Ljung-Box Test
After a model is fitted, its residuals should be white noise: if any autocorrelation remains, the model has left forecastable structure on the table and is not yet adequate. This is checked with a joint test of whether a whole group of residual autocorrelations are zero together. The Box-Pierce statistic sums the squared sample autocorrelations, scaled by the sample size, and follows a chi-squared distribution. The Ljung-Box statistic is a refinement that reweights the terms to behave better in smaller samples, and it is the version used in practice.
where T is the sample size and ρ̂h the sample autocorrelation at lag h, summed over the lags being tested. Both are chi-squared; a large value rejects the null that all the tested autocorrelations are zero.
A fitted model’s Ljung-Box statistic over the first 10 residual lags is 25, and the chi-squared critical value at 5% for the relevant degrees of freedom is about 18. Is the model adequate?
Step 1. Compare the statistic with the critical value.
Answer: the statistic exceeds the critical value, so the null that the residual autocorrelations are all zero is rejected. The residuals are not white noise, meaning the model has left predictable structure behind, and it should be revised, typically by adding an AR or MA term, until the residuals pass the test.
Forecasting and Long-Horizon Mean Reversion
The payoff of a fitted model is a forecast. Forecasts are built recursively, one step at a time, using three simple rules: any value already observed is used as it is; any future shock is replaced by its expected value of zero; and a multi-step forecast substitutes the earlier forecasts wherever it needs values not yet observed. Rolling these rules forward produces the forecast at any horizon.
where YT is the last observed value and h the forecast horizon. As h grows, the term φh shrinks to zero (since |φ| < 1), so the influence of the last value fades and the forecast converges to δ / (1 − φ), the mean-reverting level.
The spread Yt = 0.5 + 0.8 Yt−1 + εt is currently at YT = 3.0. Forecast the next two periods, and state the long-horizon forecast.
Step 1. One-step: set the future shock to zero and plug in the current value.
Step 2. Two-step: reuse the one-step forecast in place of the unobserved value.
Answer: forecasts of 2.9 and 2.82, edging down from 3.0 toward the long-run level of 2.5, which is the forecast at a distant horizon. The series was above its mean, so every forecast steps back toward 2.5, more slowly because the coefficient is high. This is mean reversion seen through the lens of forecasting: the further ahead you look, the more the forecast is just the unconditional mean.
Modeling Seasonality
Many series carry a calendar rhythm: retail sales peak in December, electricity demand swings with the seasons, some rates move on a quarterly cycle. This seasonality is captured by placing AR or MA terms not at the adjacent lag but at the seasonal lag, the number of periods in one full cycle. For quarterly data that lag is four, so a seasonal autoregressive term links each quarter to the same quarter a year before; for monthly data it is twelve.
where the L4 term captures the year-over-year seasonal link and the L term captures ordinary short-run persistence. Multiplying out shows this is a higher-order ARMA whose ACF spikes at the seasonal lags, four, eight, twelve, and so on.
The important points are that seasonality is modeled inside the same ARMA framework, not bolted on separately, and that it leaves a recognizable trace: an ACF with pronounced spikes at multiples of the seasonal period. A seasonal model stays covariance stationary as long as its coefficients satisfy the usual stationarity conditions, so all the machinery of this chapter, identification, estimation, residual checking, and forecasting, carries over unchanged to seasonal data.
A series has a constant mean and a constant variance, but its correlation between consecutive observations was 0.3 in the 1990s and 0.7 in the 2010s. Is it covariance stationary?
Show answer
No. Covariance stationarity requires the autocovariance to depend only on the lag between observations, not on when they occur. Here the lag-1 dependence changed over time, from 0.3 to 0.7, so the third requirement fails even though the mean and variance are constant. All three conditions must hold; a stable mean and variance are not enough on their own.
A series’ sample ACF decays slowly toward zero, while its sample PACF is large at lag 1 and near zero for every lag after. Which model does this point to?
Show answer
An AR(1). The signature of an autoregressive model is a gradually decaying ACF paired with a PACF that cuts off, and a cutoff after lag 1 fixes the order at one. Had the ACF cut off after lag 1 and the PACF decayed instead, the answer would be an MA(1); had both decayed, an ARMA. The location of the sharp cutoff, and which function shows it, identifies the family and the order.
An interest rate follows Yt = 1.0 + 0.75 Yt−1 + εt. What is its long-run level, and is the process stationary?
Show answer
The process is stationary because the coefficient 0.75 is below 1 in absolute value. Its mean-reverting level is the constant divided by one minus the coefficient: 1.0 / (1 − 0.75) = 1.0 / 0.25 = 4.0. So the rate gravitates to 4.0 over time, closing 25% of any gap each period, which is a moderate speed of reversion.
Using that same model with the rate currently at 6.0, what is the one-step-ahead forecast, and what happens to the forecast as the horizon grows very large?
Show answer
The one-step forecast sets the future shock to zero: 1.0 + 0.75 × 6.0 = 5.5, a step down from 6.0 toward the mean. As the horizon grows, the coefficient raised to the horizon shrinks to zero, so the forecast converges to the long-run level of 4.0. Far enough ahead, the best forecast is simply the mean-reverting level, no matter where the series starts, which is mean reversion expressed in forecasts.
Chapter Summary
- A series is covariance stationary when its mean is constant, its variance is constant and finite, and its autocovariance depends only on the lag, not on time.
- The ACF gives the correlation at each lag; the PACF gives it after removing the intervening lags. Their shapes identify the model.
- White noise (mean 0, constant variance, no autocorrelation) is the building block; independent, Gaussian, and dependent variants differ in their extra structure.
- An AR(1), Yt = δ + φYt−1 + εt, is stationary when |φ| < 1, has mean δ/(1 − φ), variance σ²/(1 − φ²), and ACF φh.
- A stationary AR mean-reverts to δ/(1 − φ); a coefficient closer to 1 means slower reversion.
- The lag operator L shifts time back one step and lets models be written as lag polynomials.
- An MA(q) is always stationary; its ACF cuts off after lag q while its PACF decays, the mirror image of the AR.
- An ARMA has both an ACF and a PACF that decay gradually, and it can match a series with fewer parameters.
- Identify a model from the cutoff: PACF cutoff means AR, ACF cutoff means MA, neither means ARMA; the Box-Pierce and Ljung-Box tests check that residuals are white noise.
- Forecasts are rolled forward with future shocks set to zero, and they revert to the unconditional mean as the horizon grows; seasonality is modeled with AR or MA terms at the seasonal lag.
Frequently Asked Questions
What makes a time series covariance stationary?
A time series is covariance stationary when three conditions hold: its mean is constant over time, its variance is constant and finite, and its autocovariance between two observations depends only on the gap between them, not on where in time they sit. In plain terms, the statistical behavior of the series does not drift as time passes. This matters because only a stationary series has stable relationships that can be estimated from historical data and used to forecast the future; a non-stationary series can produce spurious relationships and estimates that do not settle down.
What is the difference between the ACF and the PACF?
The autocorrelation function, or ACF, gives the correlation between an observation and one h periods earlier, for each lag h. The partial autocorrelation function, or PACF, gives the correlation between the same two observations after removing the influence of all the observations in between. The ACF captures both the direct link and the indirect links that pass through intermediate periods, while the PACF isolates the direct link only. Their contrasting shapes across lags are the main tool for identifying which time-series model fits a series.
What is white noise, and what are its variants?
White noise is the fundamental building block of time-series models: a process with mean zero, constant finite variance, and zero autocorrelation at every lag, so no value carries information about any other. Independent white noise strengthens this to full independence, not just zero correlation. Gaussian, or normal, white noise adds that each value is normally distributed. Dependent white noise is uncorrelated but not independent, which allows persistent volatility even though the level is unpredictable, the pattern seen in the ARCH models used for financial returns.
What is an autoregressive model, and what is mean reversion?
An autoregressive model relates the current value of a series to its own past values plus a shock; a first-order model, AR(1), uses just the previous value. Its coefficient measures persistence, and the process is stationary only when that coefficient is below one in absolute value. Mean reversion is the tendency of such a stationary process to be pulled back toward its long-run average whenever it strays. The mean-reverting level is that long-run average, equal to the intercept divided by one minus the autoregressive coefficient, and a coefficient closer to one means slower, more gradual reversion.
How do you tell AR, MA, and ARMA models apart?
The autocorrelation and partial autocorrelation functions have distinctive shapes for each model. An autoregressive model of order p has an ACF that decays gradually and a PACF that cuts off sharply after lag p. A moving average model of order q is the mirror image: its ACF cuts off after lag q and its PACF decays gradually. An ARMA model, which combines the two, has both an ACF and a PACF that decay gradually without any sharp cutoff. Matching the sample ACF and PACF shapes to these signatures is how a candidate model is identified.
What are the Box-Pierce and Ljung-Box tests?
Both are joint tests of whether a group of autocorrelations are all zero, used to check whether a fitted model’s residuals behave like white noise. The Box-Pierce statistic sums the squared sample autocorrelations scaled by the sample size, and follows a chi-squared distribution. The Ljung-Box statistic is a refinement that weights the autocorrelations to perform better in smaller samples, and is the preferred version in practice. A large statistic rejects the null of no autocorrelation, signaling that the model has left predictable structure in the residuals and is not yet adequate.
How are forecasts generated from an ARMA model, and why do they revert to the mean?
Forecasts are built recursively using three rules: any already-observed value is used as is, any future shock is set to its expected value of zero, and multi-step forecasts substitute the earlier forecasts for the values not yet observed. Applying these repeatedly rolls the model forward one step at a time. Because a stationary process has a bounded memory, the influence of the last observed value fades as the horizon lengthens, so the forecasts drift toward the unconditional mean. At a long enough horizon, the best forecast is simply the long-run mean of the series, which is the mean-reversion property expressed in forecasting terms.
How is seasonality modeled in a time series?
Seasonality, a repeating pattern tied to the time of year, is modeled by adding autoregressive or moving average terms at the seasonal lag rather than only at adjacent lags. For quarterly data the seasonal lag is four, so a seasonal autoregressive term links each quarter to the same quarter a year earlier. Combined with the usual short-lag terms, this produces a model whose autocorrelation function shows spikes at the seasonal lags, and it keeps the series covariance stationary while capturing the recurring calendar pattern.
Loading comments...
Add your Thoughts: