Paper Trail #17: Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation (Andrews, 1991) — the paper that gave us Newey-West-standard-error automatic bandwidths, and quietly recommended a different kernel altogether
Donald W. K. Andrews (1991) “Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation” Econometrica 59(3):817-858 (May 1991), DOI 10.2307/2938229. The paper that took Newey and West’s 1987 HAC estimator, proved it was asymptotically inefficient, offered a better kernel (Quadratic Spectral), and — almost as an aside — gave the world the automatic bandwidth formula every stats package now ships as the default for autocorrelation-corrected standard errors.
Primary source: 85-page Cowles Foundation Discussion Paper No. 877 (July 1988), the working-paper precursor to the Econometrica publication, verified today via pymupdf extraction (125,991 characters, text-native). Every quantitative claim below traces to a numbered equation in that PDF.
The kernel family (Section 2, eq 2.7)
Andrews collects the HAC kernel estimator into a single family indexed by a bandwidth parameter M and a kernel function k(x):
| Kernel | Definition for |x| ≤ 1 (else 0) | q | Attributed to |
|---|---|---|---|
| Truncated | k(x) = 1 | 1 | White (1984) |
| Bartlett | k(x) = 1 − |x| | 1 | Newey and West (1987) |
| Parzen | 1 - 6x² + 6|x|³ for |x|≤½; 2(1-|x|)³ for ½<|x|≤1 | 2 | Gallant (1987) |
| Tukey-Hanning | (1 + cos(πx)) / 2 | 2 | — (not psd in general!) |
| Quadratic Spectral | 25/(12π²x²) · [sin(6πx/5)/(6πx/5) − cos(6πx/5)] | 2 | Andrews’s preferred choice |
The q column is Andrews’s kernel characteristic index — for the Bartlett kernel, the taper k(x) = 1 − |x| has a first-order kink at zero, so q=1. For Parzen, Tukey-Hanning, and QS the kernel is smooth at zero, so q=2. That q index controls whether the optimal bandwidth grows as T^(1/3) (Bartlett) or T^(1/5) (Parzen/TH/QS), and it determines the asymptotic MSE convergence rate — the whole efficiency-ordering flows from this single index.
The automatic bandwidths (Section 5, eq 5.9)
Parzen kernel: M* = 2.6614 · (a · T)1/5
Tukey-Hanning kernel: M* = 1.7462 · (a · T)1/5
Quadratic Spectral kernel: S* = 1.3221 · (a · T)1/5
The four constants are the ones every practitioner cites. For the Bartlett/Newey-West estimator — the industry default — 1.1447 and the T^(1/3) growth rate. For QS, the same a parameter but growth rate T^(1/5) and constant 1.3221. The equations are the same shape; only the growth rate differs by kernel index q.
The AR(1) plug-in for “a” (Section 5, eqs 5.3-5.4)
The parameter a is a spectral-density-at-zero curvature measure. If the series were literally an AR(1) with parameter ρ, Andrews computes it in closed form:
q = 2 (Parzen, TH, QS): a = 4·ρ² / (1 − ρ)⁴
This is the “plug-in” Andrews described in Section 7: use the sample lag-1 autocorrelation ρ̂(1) as your ρ, evaluate the appropriate formula for the kernel you’re using, get a, and read M*off eq 5.9 above. The two AR(1) formulae are NOT interchangeable — using the q=2 formula with the Bartlett kernel is an off-by-a-factor error that today’s Stats #17 walks through: the correct Bartlett value on our 196 NFP × USDJPY sample is a = 0.311, giving M* = 4.51. Using the q=2 formula would give a = 0.903, giving M* = 6.4 — an over-blocked estimator that would look plausible but is provably wrong for the kernel being used.
Why Andrews prefers the QS kernel over Bartlett/Newey-West
Andrews’s Theorem 1 establishes the asymptotic MSE convergence rate for each kernel:
| Kernel | q | MSE rate | Note |
|---|---|---|---|
| Bartlett (NW/BT) | 1 | O(T-2/3) | slower — asymptotically inefficient |
| Parzen (GAL/PR) | 2 | O(T-4/5) | faster; 0.9% less efficient than QS |
| Tukey-Hanning (TH) | 2 | O(T-4/5) | 8.6% less efficient than QS; NOT always psd |
| Quadratic Spectral (QS) | 2 | O(T-4/5) | Andrews’s optimal minimax choice |
Reading Section 9’s Monte Carlo Table 2 (T=128, six AR(1) processes with parameters v ∈ {0.3, 0.5, 0.7, 0.99, -0.3, -0.7}): QS, Gallant/Parzen, and Tukey-Hanning “consistently exhibit a distinct, but not huge, advantage over the NW/BT estimator” (Andrews’s wording). The advantage widens at T=256 as the asymptotic rate O(T^(-4/5)) overtakes O(T^(-2/3)) more decisively. At T=64 the advantage compresses but doesn’t vanish. And in Andrews’s Section 10 summary, his preferred estimator is the QS with automatic bandwidth: “If computational time is a binding constraint, the Parzen kernel can be used in place of the QS kernel.” Bartlett/NW isn’t on the recommended list at all.
Why the industry uses Bartlett/Newey-West anyway
Path dependence and a psd guarantee. Bartlett was published first (Newey-West 1987, four years before this paper), so it had users and citations by the time Andrews wrote. The Bartlett kernel produces positive-semi-definite covariance matrix estimates by construction from a simple triangular taper — which is a hard requirement for statistical inference (a non-PSD covariance matrix gives you nonsensical standard errors and negative confidence-interval widths). QS and Parzen are also PSD but they’re implemented by more complicated kernel functions. Tukey-Hanning is a cautionary tale — Andrews’s Comment near Theorem 2 explicitly notes it “does not necessarily generate psd estimates.”
The result: every stats package’s “autocorrelation- corrected standard errors” default is Bartlett/Newey-West, with the Andrews bandwidth (or the Newey-West 1994 simpler rule) picking M. Andrews would tell you to check the QS option in your package (it’s usually there — Stata calls it kernel(qs); Python’s statsmodels calls it kernel=“qs”) if you care about the last 1-3% of efficiency. Most practitioners don’t, and get on with Newey-West Bartlett.
The Newey-West 1994 follow-up
Newey and West themselves published a follow-up in 1994 (Review of Economic Studies 61(4):631-653) that offers a simpler automatic bandwidth for the Bartlett kernel:
No AR(1) plug-in, no sample-dependent aparameter — just the sample size. For today’s T=196: M = 4 · 1.96^(2/9) = 4.63, floor to M = 4. Andrews’s data-dependent rule gave M* = 4.51 on the same sample — near-identical answers, arrived at by different reasoning. The two rules typically differ by 1-2 lags in practice; use whichever your stats package defaults to.
Cross-links
Today’s Stats #17 uses this paper’s eq (7.2) Bartlett formula directly to get M* = 4.51 on the 196 NFP × USDJPY 15m |move_pips| series that has been the running example across Stats #10 (AR(1) diagnosis), Stats #15 (stationary bootstrap), and Stats #16 (Politis-White automatic block length). Paper Trail #14 (Politis-Romano 1994 stationary bootstrap) is the block-resampling alternative to Andrews’s HAC approach; both are attacking the same problem (standard errors for autocorrelated series) from different mathematical angles. Modern practice mixes them: Bartlett/Newey-West for regression-model standard errors, block bootstrap for confidence intervals on nonlinear statistics.
What this doesn’t say
The paper doesn’t solve the “how many lags should I use” problem in one shot.It offers a principled AR(1)-plug-in automatic bandwidth that’s much better than picking M by eye, and it derives the asymptotic rates showing when each rule is optimal. But the plug-in is AR(1)-parametric; if your series has strong non-AR(1) higher-lag structure (like the +0.220 lag-11 echo on the NFP × USDJPY series today), the Andrews rule under-covers it. A more data-adaptive alternative is the Politis-White (2004) flat-top window with a data-driven cutoff — the topic of Stats #16 yesterday.
Andrews’s asymptotic-efficiency ordering is finite-sample-approximate, not exact. Table 2 at T=128 shows the QS/Parzen/TH advantage over Bartlett is real but modest — MSE ratios in the 0.8-0.95 range for most model configurations. For a modest sample size (T=196 today), the gain from switching to QS would be on the order of a few percent in MSE, translating to a few percent in standard-error estimates. Worth taking if free; not worth engineering a whole migration to.
The paper doesn’t address bootstrap or resampling methods. Andrews works entirely in the analytic HAC-kernel framework. The block bootstrap (Paper Trail #14 Politis-Romano 1994) and the stationary bootstrap use entirely different machinery — resampling contiguous blocks of observations rather than smoothing the sample spectral density. The two approaches solve overlapping but distinct problems: HAC for standard errors of the sample mean of an autocorrelated series, block bootstrap for confidence intervals on any statistic of an autocorrelated series.
Read the paper → (Cowles Foundation Discussion Paper 877, the working-paper version of the 1991 Econometrica publication)