Core Path · L6.2.10

Backtesting ES and whole distributions

14 minExpected shortfallAcerbi-SzekelyPITBerkowitz testElicitabilityPLA test

By the end you will be able to

  1. 1Compute Acerbi and Székely's Z1 and Z2 for an ES forecast and judge them against critical values simulated from the model itself.
  2. 2Test a whole forecast distribution with the PIT histogram and Berkowitz's likelihood ratio, and read what their shapes say.
  3. 3Explain why ES alone is not elicitable, and rank models with the tick and FZ0 scores and a Diebold-Mariano test.
  4. 4Run the P&L attribution test's Spearman and KS statistics (M-D05: Rank correlation and distribution tests) on the sleeve's risk models and place each in its zone.

Why it matters

Historical simulation's 99% VaR on the sleeve passes the conditional coverage test of L6.2.09: Backtesting VaR: coverage and independence at 5% (p 0.079). Its 97.5% expected shortfall, the number FRTB capital is built on, fails Acerbi and Székely's test with p = 0.002. The exceptions are too many and too big: on average 5.5% beyond the ES the model forecast for them. A VaR test asks whether the line was crossed. An ES test asks how far.