What's the one-line summary of RDA?

Regularized Discriminant Analysis (Friedman 1989) is a two-parameter (λ, γ) shrinkage family that bridges LDA and QDA by writing Σ_k(λ, γ) = (1-γ)[(1-λ)Σ_k + λΣ_pool] + (γ/p) tr[(1-λ)Σ_k + λΣ_pool] · I. (λ, γ) ∈ [0,1]² and the four corners are QDA (0,0), LDA (1,0), nearest-means with per-class-scaled identity (0,1), and weighted-nearest-means (1,1). Model-selection per Section 4 picks (λ*, γ*) by minimizing LOO cross-validated misclassification risk over a grid.

What does RDA give on the UK Core CPI × GBPCHF panel?

Grid search over 21×21 = 441 points at step 0.05: BEST RDA LOO 89/196 = 45.41% at (λ*=0.90, γ*=0.05). Baseline anchors (2026-09-05 LDA / 2026-09-07 QDA): LDA (λ=1, γ=0) 84/196 = 42.86%; QDA (λ=0, γ=0) 79/196 = 40.31%; regularized QDA at reg_param=0.20 (Stats #38's best regularization) 83/196 = 42.35%; nearest-means (0,1) 73/196 = 37.24%; weighted-NM (1,1) 69/196 = 35.20%; majority baseline 62/196 = 31.63%. RDA beats every corner and every sibling technique in Stats #34-#38 — closes the multivariate-classifier arc.

Where does the optimum sit and what does that mean?

(λ*=0.90, γ*=0.05) — mostly LDA territory (λ=1 is the LDA corner) with a small additional eigenvalue-shrinkage γ=0.05. This says: on this panel the pooled Σ carries essentially all the discriminant signal (as Stats #38 already suggested via regularized QDA → LDA parity), and a small additional ridge on the smallest eigenvalues of the pooled Σ helps tighten in_line and big_beat class decisions by +6 combined LOO predictions vs LDA. It's a 'regularized LDA' more than a QDA-flavored solution. If the true class covariances DID differ substantially (they don't here) the optimum would sit further left (smaller λ, closer to QDA).

Per-class LOO recall — does RDA fix LDA's zero big_miss?

No. RDA (λ*=0.90, γ*=0.05) big_miss recall stays 0/13 = 0% — the log-prior handicap on n=13 tail buckets vs the n=62 in_line dominates, and small (λ, γ) tweaks can't overcome a ~1.4 nat prior penalty. Where RDA HELPS vs LDA: in_line 34/62 vs LDA's 30/62 (+4 predictions) and big_beat 4/15 vs LDA's 2/15 (+2). Where it LOSES slightly: small_beat 25/53 vs LDA's 26/53 (-1). Net: +5 = 89 vs 84. Fixing the zero-recall big_miss would require a different mechanism — stripping empirical priors to uniform, or a class-balanced sampling approach.

Is the 89 vs 84 advantage statistically meaningful?

Small. n=196, Δ=5 predictions. Binomial noise scale is sqrt(196 × 0.5 × 0.5) ≈ 7. So a 5-prediction gap between two classifiers on the same LOO cross-validation is one binomial-standard-deviation apart, not two — you couldn't confidently pick RDA over LDA from this dataset alone. What Friedman 1989's simulation studies (Section 6, Tables 1-6) show is that RDA's advantage grows when class covariances differ substantially and sample sizes are small — the UK Core CPI × GBPCHF panel doesn't hit either condition strongly, so the observed edge is modest. The METHOD generalizes to datasets where those conditions bind.

How does this close the multivariate-classifier arc?

Stats #34 (MANOVA — is there mean structure?) → #35 (1D canonical projection, 36.73%) → #36 (LDA full posterior, 42.86%) → #37 (Mahalanobis centroid distance) → #38 (QDA per-class Σ_k, 40.31% — overfits) → #39 (multinomial logit, 44.39%). Stats #49 RDA closes the arc with the shrinkage bridge that (a) covers all of LDA/QDA/NM as special cases, (b) picks the mixing parameters (λ, γ) via joint LOO grid search, and (c) beats all six prior methods on this panel. Queue rotates to: suppressor-variable partial-vs-semi-partial (Stats #25); higher-order VAR-b prewhitening (Stats #20); Newcombe's paired-differences companion (Stats #46).

← All insights

Statistics18 September 2026· 7 min read

Statistics for Traders #49: Regularized Discriminant Analysis (Friedman 1989 RDA) — the two-parameter (λ, γ) shrinkage bridge between LDA (Stats #36) and QDA (Stats #38) on the same UK Core CPI × GBPCHF 6-window panel. Grid search on 441 (λ, γ) points at step 0.05 finds LOO-CV optimum 89/196 = 45.41% at (λ=0.90, γ=0.05) — BEATS LDA's 84/196 = 42.86% by +1.02pp and QDA's 79/196 = 40.31% by +5.10pp. Optimum sits near the LDA corner with a small additional ridge — Friedman's shrinkage regularizes LDA's pooled covariance eigenvalues rather than shifting weight toward QDA's per-class covariances. Closes the six-post multivariate-classifier arc Stats #34-#39 with the queued follow-up promised in Stats #38 (2026-09-07).

Regularized Discriminant Analysis (Friedman 1989 RDA) is the two-parameter (λ, γ) shrinkage bridge between LDA (Stats #36) and QDA (Stats #38). Applied to the same UK Core CPI × GBPCHF 6-window panel (n=196, K=5, p=6), a 441-point grid search over (λ, γ) ∈ [0,1]² at step 0.05 finds LOO-CV optimum 89/196 = 45.41% at (λ*=0.90, γ*=0.05)— BEATS LDA’s 84/196 = 42.86% by +1.02pp and QDA’s 79/196 = 40.31% by +5.10pp. Closes the six-post multivariate-classifier arc Stats #34-#39 with the queued follow-up promised in Stats #38.

The RDA covariance-mixing formula (Friedman 1989 Eqs 16 & 18)

Σ_k(λ) = (1 − λ) · Σ_k + λ · Σ_pool (Eq 16b)

Σ_k(λ, γ) = (1 − γ) · Σ_k(λ) + (γ/p) · tr[Σ_k(λ)] · I (Eq 18)

where Σ_k is the per-class sample covariance, Σ_pool the pooled covariance, p the feature dimension, and (λ, γ) ∈ [0,1]² the two shrinkage parameters.

Four corners of the (λ, γ) plane recover well-known classifiers: (0, 0) is QDA (per-class Σ_k), (1, 0) is LDA (pooled Σ), (0, 1) is nearest-means with per-class-scaled identity, and (1, 1)is a weighted nearest-means classifier. Friedman’s Section 4 model-selection procedure picks (λ*, γ*) by minimizing LOO cross-validated misclassification risk over a grid: “the size of the optimization grid N_p is typically 25 to 50 points.” We use a 21×21 = 441-point grid at step 0.05.

Heatmap of LOO-CV correct-prediction count for RDA on the UK Core CPI × GBPCHF panel (n=196), across a 21×21 grid of (λ, γ) values at step 0.05. λ (horizontal axis) mixes per-class covariance (λ=0) with pooled covariance (λ=1). γ (vertical axis) shrinks toward a scalar multiple of identity (γ=1 = full shrinkage). Color scale runs from cool-dim (69 correct) to coral-bright (89 correct). Four corners labeled: QDA at (0,0) with 79, LDA at (1,0) with 84, nearest-means/scale at (0,1) with 73, weighted-NM at (1,1) with 69. The best point is marked with a ring at (λ*=0.90, γ*=0.05) — 89 correct — situated near the LDA corner with a small vertical shrinkage. The LOO landscape has multiple near-optimal regions along the top-right edge (λ ≈ 0.85-1.0, γ ≈ 0-0.15). — LOO-CV heatmap over 441 (λ, γ) points. Corners: QDA (0,0)=79, LDA (1,0)=84, NM/scale (0,1)=73, weighted-NM (1,1)=69. Best RDA (ring): 89/196 at (λ*=0.90, γ*=0.05).

Grid search — key rows and columns

Method (λ, γ)	LOO correct	LOO %	Prior post
Majority-class baseline	62/196	31.63%	(in_line, K=5)
Weighted NM (1.00, 1.00)	69/196	35.20%	RDA corner
Nearest-means (0.00, 1.00)	73/196	37.24%	RDA corner
QDA (0.00, 0.00)	79/196	40.31%	Stats #38
QDA reg=0.20 (γ-analog)	83/196	42.35%	Stats #38
LDA (1.00, 0.00) ★	84/196	42.86%	Stats #36
Multinomial logit (out-of-family)	87/196	44.39%	Stats #39
RDA (λ=0.90, γ=0.05) ★★	89/196	45.41%	Stats #49 today

RDA at (λ*=0.90, γ*=0.05)beats LDA by +5 predictions, beats QDA by +10, and beats multinomial logit (Stats #39) by +2. Δ=5 vs LDA is inside binomial noise scale √(n · 0.5²) ≈ 7 — RDA’s advantage on THIS panel is modest, but the method generalizes to panels where class covariances differ substantially (Friedman 1989 Section 6 simulation studies, Tables 1-6).

Per-class LOO recall — where the +5 gain comes from

Bucket (n)	LDA recall	RDA recall	Δ (RDA − LDA)
big_miss (13)	0/13 (0.00%)	0/13 (0.00%)	0
small_miss (53)	26/53 (49.06%)	26/53 (49.06%)	0
in_line (62) ★	30/62 (48.39%)	34/62 (54.84%)	+4
small_beat (53)	26/53 (49.06%)	25/53 (47.17%)	-1
big_beat (15) ★	2/15 (13.33%)	4/15 (26.67%)	+2
TOTAL	84/196	89/196	+5

The +5 gain is concentrated in in_line (+4) and big_beat (+2) with a small loss on small_beat (-1). big_miss stays at 0/13 recall — small (λ, γ) tweaks cannot overcome the ~1.4 nat log- prior handicap on n=13 tail buckets versus the n=62 in_line bucket. Stripping empirical priors to uniform would be the alternative fix (see Stats #38 discussion).

Closing the multivariate-classifier arc

Stats #34 (MANOVA — mean structure?) → #35 (1D canonical projection, 36.73%) → #36 (LDA full posterior, 42.86%) → #37 (Mahalanobis centroid distance) → #38 (QDA per-class Σ_k, 40.31% — overfits) → #39 (multinomial logit, 44.39%). Stats #49 RDA closes the arc with the shrinkage bridge that (a) covers all of LDA/QDA/NM as (λ, γ) corners, (b) picks the mixing parameters via joint LOO grid search, and (c) at 89/196 = 45.41% BEATS every prior method on this panel. Queue rotates to: suppressor-variable partial-vs-semi-partial (queued since Stats #25); higher-order VAR-b prewhitening (queued since Stats #20); Newcombe’s paired-differences companion (Statistics in Medicine 17:2635-2650, queued 2026-09-15).

Verification note

Panel verified 2026-09-18 against live /api/v1/news-impact/releases across all 6 windows on FF:GBP_CORE_CPI_YOY × GBPCHF (n=198 per window, n=196 after joining on release_ts across all six windows — matches Stats #36-#38 exactly). RDA implemented from Friedman 1989 Eqs (16b) + (18) in a one-off Python script (numpy 2.4.6 + scipy 1.17.1); LDA/QDA cross-checked against sklearn 1.9.1 LinearDiscriminantAnalysis and QuadraticDiscriminantAnalysis. LOO-CV grid: 441 points on (λ, γ) ∈ [0,1]² at step 0.05.

Keep reading