Statistics for Traders #10: autocorrelation and why 196 NFP releases behave like 115 independent samples once you look at the volatility clustering
The 196-release NFP history on USDJPY shows near-zero autocorrelation in signed move direction (as efficient-markets weak-form predicts) but a +0.26 lag-1 autocorrelation in absolute move size — well above the ±0.14 IID confidence band for n=196. Volatility clusters. Treat that 196-sample history as if it were ~115 independent observations for confidence-interval purposes — otherwise your CIs will be systematically too narrow by about 30 percent.
What autocorrelation is, in one paragraph
Autocorrelation is the correlation of a time series with its own lagged self. Lag-1 autocorrelation is the correlation between x_t and x_(t-1), computed across every pair of consecutive observations. Formally: ρ_k = Σ (x_t − x̄)(x_(t−k) − x̄) / Σ (x_t − x̄)². A positive autocorrelation means “big values tend to follow big values, small tend to follow small”. Zero means “the last value tells you nothing about the next” — which is what the IID (independent and identically distributed) assumption formally requires. Every confidence-interval and standard-error formula you’ve seen ultimately relies on IID.
Three series, one release history, three different answers
Take the 196 non-contaminated NFP releases (2010-01-08 through 2026-08-07) on USDJPY at the 15-minute window. Every release has three natural numeric summaries: the surprise z-score (how many standard deviations from consensus), the signed 15-minute move_pips (which way USDJPY went, in pips), and the absolute |move_pips| (how much USDJPY moved, ignoring direction). Compute the sample autocorrelation of each:
| Lag | surprise_z | move_pips (signed) | |move_pips| |
|---|---|---|---|
| 1 | +0.081 | −0.090 | +0.260 |
| 2 | −0.289 | −0.053 | +0.244 |
| 3 | +0.034 | +0.031 | +0.045 |
| 4 | −0.007 | +0.070 | +0.212 |
| 5 | −0.010 | −0.003 | +0.066 |
| 6 | +0.054 | +0.116 | +0.133 |
| 12 | −0.014 | +0.036 | +0.109 |
Under the null hypothesis of IID data, sample autocorrelations at any single lag are approximately normal with mean 0 and standard error 1/√n. For n=196 that’s a 95% band of ±2/√196 = ±0.143. Coral-highlighted values in the table exceed that band.
Reading the table
Signed move_pips is essentially white noise — nothing exceeds ±0.14 at any lag. This is the efficient-markets weak-form story (Paper Trail #4, Fama 1970): the direction of the next NFP move on USDJPY is unforecastable from the direction of past moves. If it weren’t, there’d be a straightforward arbitrage in buying or selling USDJPY 15 minutes before every NFP based on last month’s reaction. The data says: don’t bother.
|move_pips| is strongly autocorrelated at lags 1, 2, and 4 (+0.26, +0.24, +0.21). Volatility clusters — big-move NFP prints tend to be followed by more big-move NFP prints. This is exactly the phenomenon ARCH (Paper Trail #5, Engle 1982) and GARCH (today’s Paper Trail #10, Bollerslev 1986) were invented to model. On NFP the mechanism is macro-regime clustering: months where NFP is uncertain (early cycle turn, recession-recovery boundary, COVID reopening) cluster together in time, and USDJPY’s reaction magnitude tracks that uncertainty.
surprise_z lag-2 = −0.289is a side finding worth flagging but not over-reading. Consecutive-2-month NFP surprises tend to be OPPOSITE-signed. The most likely mechanism is a BLS revision artifact — when last month’s print gets revised, the current-month consensus anchors to the revised prior, and if revisions overshoot in one direction the current-month surprise tends to overshoot in the other. This isn’t a tradeable signal (the surprise is a difference from consensus, not a price move), and re-computing on a different 196-release window could give a materially different number. Note it, don’t bet on it.
Effective sample size: 196 → 115
For a first-order autoregressive process — the simplest model of a series that depends on its own last value — the effective sample size for estimating the mean is:
n_eff = n × (1 − ρ) / (1 + ρ)
Derivations in Wilks (2011) “Statistical Methods in the Atmospheric Sciences” §5.2.4 and Zwiers & von Storch (1995). Plug in the |move_pips| lag-1 autocorrelation:
n_eff = 196 × (1 − 0.26) / (1 + 0.26) = 196 × 0.587 = 115.1
Confidence intervals scale like 1/√n, so a properly-sized CI is wider than the naive-IID version by √(196 / 115) = 1.30— a 30 percent penalty. The AR(1) approximation UNDER-counts the penalty when autocorrelation persists at higher lags, and here we see +0.24 at lag-2 and +0.21 at lag-4 too — so 30% is a lower bound. A more honest formula (Bartlett’s window over the significant lags) would push the penalty to roughly 40%.
Practical consequence for the tool’s bucket stats
Signed medians and per-trade Sharpe ratios: no change needed. The signed move_pips series shows near-zero autocorrelation, so the standard confidence-interval formulas walked through in Stats #3 (distribution-free CI for the median) and Stats #7 (t-based CI for the mean) apply directly. Direction-of-move CIs are IID-clean on monthly-frequency events.
Volatility summaries (p25/p75 width, |move| median, IQR, Sharpe/Sortino ratios): widen your CIs by ~30%.The clustering is in the magnitude, so any statistic that summarizes typical range needs the autocorrelation penalty. The bucket table’s p25/p75 columns look precise (two decimal places of a specific pip value), but if you build a confidence interval for those numbers assuming IID you’ll systematically under-cover.
Sample-size opacity fading (n<30 shown faint) is conservative, not too aggressive.The tool draws bucket lines faint when a bucket has fewer than 30 observations, signalling low statistical confidence. Under IID that’s a reasonable threshold. Under clustering, effective sample size is smaller than nominal, so the visual signal is if anything too optimistic on the un-faded (n≥30) buckets. The n≥30 rule translates to roughly n_eff ≥ 20 under moderate clustering — still enough for basic descriptive summaries, but tail statistics on those buckets deserve extra skepticism.
What this doesn’t say
The AR(1) effective-sample-size formula is a first-order approximation.When autocorrelation persists at lags 2, 4, and beyond (as it does here), the true effective sample size is smaller than what the AR(1) formula gives. Bartlett’s window over the significant lags is the standard next step, and it typically pushes the penalty higher. Treat 30% as a lower bound and 40% as a working estimate for the NFP × USDJPY series specifically.
Autocorrelation of |move_pips| is not the same as autocorrelation of a trading strategy’s P&L. A strategy that trades a signed move (long or short based on surprise bucket) has P&L autocorrelation equal to the signed-move autocorrelation (near zero here), not the absolute- move autocorrelation. The volatility clustering matters for SIZING the strategy (position size given expected volatility regime) and for computing per-trade Sharpe/Sortino confidence intervals — not for the strategy’s directional edge or its per-trade mean.
Bootstrap resampling doesn’t automatically fix this.The simple bootstrap (draw with replacement from the sample) implicitly assumes IID. For an autocorrelated series you need the BLOCK bootstrap, which draws contiguous blocks rather than single observations. That’s the follow-up topic Stats #7 flagged as “the next tool for fat-tailed small-n data” — worth a full installment when the queue reaches it, because getting the block length wrong is the main practitioner mistake.