Core Path · L3.7.14

Model risk: an introduction

12 minModel riskModel inventoryValidationSS1/23Extrapolation

By the end you will be able to

  1. 1Say what counts as a model, using the PRA's definition, and why model errors are operational losses.
  2. 2Describe a model inventory with risk tiers, and what independent validation checks.
  3. 3Show how a model that fits its data well fails when it is used outside that data.
  4. 4Outline the PRA's five principles for model risk management in SS1/23.

Why it matters

In January 2020 RiskAtoms Bank changed the spreadsheet that works out early-repayment charges on mortgages. A date in the formula was wrong. For 3.2 years, 14,000 customers who repaid early were overcharged, by £557 on average. Refunds with interest cost £8.8m. The spreadsheet was a model, but nobody had called it one, so nobody had checked it.

Core explanation

Running model risk in a bank

Illustrative dataRiskAtoms Bank plc's operational risk book

The same £67bn bank as Modules 3.5 and 3.6, at 31 December 2025: 1,107 records of losses of £10,000 or more and near misses over 2016–2025 (£240.5m of net operational losses), an RCSA of 17 key risks and 29 controls, ten key risk indicators, seven workshop scenarios and five important business services. Every event, person, supplier and figure is made up for teaching; none is any bank's data. The same book runs through every lesson in Module 3.7, so the figures agree.

Model is one of the bank's risk categories: 34 losses over ten years, £10.68m net. Model error in pricing or customer charges scores 9 inherent and 9 residual: it has not come down at all, because both its controls are only partly effective in operation. The KRI for material models overdue for validation has been amber since July 2025 and read 7.7% in December 2025 (amber 5%, red 10%).

L2.2.07: Strategic, reputational and model risk defined model risk and followed a model through its life. This lesson is about running it across a whole bank: finding every model, ranking them, checking them, and knowing when one is being used where it should not be. In this module model risk is an operational risk: a wrong model that overcharges customers or misprices a hedge is a failed process.

What counts as a model

The PRA's supervisory statement SS1/23 defines a model as “a quantitative method, system, or approach that applies statistical, economic, financial, or mathematical theories, techniques, and assumptions to process input data into output”. That reaches far beyond capital models: fraud scores, deposit behaviour, provisions, complaint forecasts and, as the bank learnt, a charges spreadsheet. Deterministic calculations that fall outside the definition still need sound, documented controls.

The early-repayment charge error

The inventory and its tiers

Each model gets a risk tier from two scores of 1 to 3: materiality and complexity. A product of 6 or more is tier 1, 3 or 4 is tier 2, and the rest tier 3. Tiers 1 and 2 are material and are validated every 2 years; tier 3 every 3.

Extra for this lessonThe bank's model inventory at 31 December 2025 (illustrative)

142 models: 14 in tier 1, 25 in tier 2 and 103 in tier 3. Three of the 39 material models are overdue, which is the book's KRI: 3 / 39 = 7.7%. A sample:

ModelM × CTierValidatedDueStatus

MD-001 Retail mortgage PD (IRB)

Capital and pricing

3 × 3Tier 12025-03-312027-03-31In date

MD-004 IFRS 9 expected credit loss, mortgages

Provisions

3 × 3Tier 12024-11-302026-11-30In date

MD-012 Mortgage prepayment (CPR)

ALM hedging and IRRBB

3 × 2Tier 12023-06-302025-06-30Overdue

MD-015 Non-maturity deposit behaviour

IRRBB and the structural hedge

3 × 2Tier 12024-03-312026-03-31In date

MD-021 Trading desk VaR

Market risk limits

2 × 3Tier 12025-06-302027-06-30In date

MD-033 Liquidity stress outflows

Survival horizon, ILAAP

3 × 2Tier 12025-01-312027-01-31In date

MD-040 Payment fraud score

Real-time payment holds

2 × 3Tier 12024-08-312026-08-31In date

MD-041 AML transaction monitoring scenarios

Alerts and suspicious activity reports

2 × 2Tier 22025-05-312027-05-31In date

MD-052 Operational risk loss distribution (LDA)

Pillar 2 and appetite

2 × 3Tier 12023-07-312025-07-31Overdue

MD-067 Early-repayment charge calculator (spreadsheet)

Customer charges

2 × 2Tier 22023-07-312025-07-31Overdue

MD-078 Complaints volume forecast (spreadsheet)

Redress provisions

2 × 1Tier 32025-02-282028-02-28In date

MD-090 Branch cash forecast

Cash logistics

1 × 1Tier 32024-04-302027-04-30In date

What validation checks

Independent validation asks four questions. Is the theory sound for this use? Are the data right and representative? Does the code do what the documents say (often by rebuilding the model independently)? And does the model perform: back-testing against outcomes, and testing on data it has not seen? Between validations, the model owner monitors outputs against outcomes each month (control C27). M6.8: Model risk, validation and machine learning covers validation techniques in depth.

Sort itWhat kind of model failure is this?

Sort each finding by where the model went wrong.

  • The charges spreadsheet counts the charge period from the wrong date

  • A straight line is used for prepayments, which have a floor and a ceiling

  • The fraud score was trained before scammers moved to impersonating delivery firms

  • A deposit model built for the structural hedge is used to set liquidity stress outflows

The quiet failure: extrapolation

The most common way a sound model fails is extrapolation. The bank's mortgage prepayment model (MD-012) predicts the prepayment rate from customers' incentive to refinance. It was fitted on 2016 to 2019, when the incentive ran between −0.2 and 1.4 percentage points. When rates rose in 2022 the incentive went to −2.5pp: far outside anything the model had seen.

The formal version

The line y^=a+b x\hat y = a + b\,x is fitted by ordinary least squares on n=48n = 48 months:

b=SxySxx,a=yˉ−b xˉ⇒y^=3.017+5.757 xR2=1−∑(yi−y^i)2∑(yi−yˉ)2=0.848,se=∑(yi−y^i)2n−2=1.154\begin{aligned} b &= \frac{S_{xy}}{S_{xx}}, \quad a = \bar y - b\,\bar x \qquad\Rightarrow\qquad \hat y = 3.017 + 5.757\,x \\ R^2 &= 1 - \frac{\sum (y_i - \hat y_i)^2}{\sum (y_i - \bar y)^2} = 0.848, \qquad s_e = \sqrt{\frac{\sum (y_i - \hat y_i)^2}{n - 2}} = 1.154 \end{aligned}

The standard error of a single new prediction at x0x_0 grows with its distance from the centre of the data, xˉ=0.602\bar x = 0.602:

se(x0)=se1+1n+(x0−xˉ)2Sxx,Sxx=10.29se(xˉ)=1.166,se(−2)=1.496y^(−2)±t0.975, 46 se(−2)=−8.50±2.013×1.496=[−11.5, −5.5]\begin{aligned} \text{se}(x_0) &= s_e \sqrt{1 + \frac{1}{n} + \frac{(x_0 - \bar x)^2}{S_{xx}}}, \qquad S_{xx} = 10.29 \\ \text{se}(\bar x) &= 1.166, \qquad \text{se}(-2) = 1.496 \\ \hat y(-2) \pm t_{0.975,\,46}\,\text{se}(-2) &= -8.50 \pm 2.013 \times 1.496 = [-11.5,\ -5.5] \end{aligned}

Customers actually prepay at about 3.0% at that incentive, far outside the interval. The interval assumes the line is the right model; it says nothing about whether it is.

Try it

Move the incentive away from the fitted data and watch the prediction and its interval. Then reveal what customers did in 2022 and 2023.

Try itUse the prepayment model outside its data

The 2022–2023 months

−15%−10%−5%0%5%10%15%−2.5−2.0−1.5−1.0−0.50.00.51.01.5Refinancing incentive, percentage pointsPrepayment rate (CPR)0.60pp
2016–2019 (fitted)Fitted line
Line predicts6.5%
95% prediction interval4.1% to 8.8%
Inside the fitted data?Yes
Fit in sample (R²)0.85

A prepayment rate cannot be negative, yet the line predicts one for any incentive below −0.52pp. The interval widens only a little away from the data: it measures noise around the line, not whether a line was the right model. Illustrative, seeded data.

Worked example

Validating the prepayment model, late

Extra for this lessonMD-012, mortgage prepayment (illustrative)

Tier 1, last validated 2023-06-30, due 2025-06-30: one of the three overdue models behind the KRI. It sets the expected life of fixed-rate mortgages for the interest rate hedge (L3.6.08: Optionality in loans). Seeded, illustrative data.

What the validator finds

Step 1 of 5
  1. 1

    The fit looks good

    On the 48 months it was fitted on, the line has R² of 0.85 and an average error of 0.91 percentage points of CPR. In 2023 the model owner's monthly checks still reported it as working.

What the model risk committee notes

The model did not break; the world moved out of its range. Validation has to test the conditions a model will be used in, not only those it was built on, and the inventory has to know when those change.

Regulatory lens

  • PRA SS1/23. Published in May 2023 and in force from 17 May 2024 for banks with permission to use internal models for capital (IRB, IMA or IMM). Five principles: model identification and classification; governance; model development, implementation and use; independent model validation; and model risk mitigants, such as post-model adjustments.
  • Senior responsibility. SS1/23 expects the board to set a model risk appetite and to appoint an accountable senior manager to implement the model risk management framework.
  • Operational losses and customers. The ERC refunds were an operational loss (execution, delivery and process management) and a Consumer Duty issue: a model that charges customers wrongly causes foreseeable harm (L3.7.12: Conduct risk and customer outcomes). The bank's model error scenario puts such an event at once in 15 years, typically £6m: £0.61m a year expected.

Spark Atoms

Spark AtomHalf the risk, on paper
L2.2.07: Strategic, reputational and model risk told the London Whale story. The numbers: JPMorgan's new VaR model, approved on 28 January 2012, put the average daily VaR of the Chief Investment Office at $67m in the first quarter of 2012. Run on the old model, it was $129m. Errors in the new model had muted volatility by a factor of two. Losses later passed $6bn.
Spark AtomTwenty validations a year
With 39 material models on a two-year cycle, the bank's validation team must complete about 20 full validations a year, plus the 103 tier 3 models every three years. Fall a few behind and the KRI moves: each overdue material model is 2.6%.

Key takeaways

  • A model is any quantitative method that turns inputs into outputs using theory and assumptions; spreadsheets count.
  • The inventory comes first: an unlisted model is never validated (the ERC error ran 3.2 years and cost £8.8m).
  • Tier by materiality and complexity; validate material models before use and on a cycle, and monitor outputs against outcomes in between.
  • Good in-sample fit is not validity: the prepayment line has R² 0.85 but errs about 9.5 times more out of sample.
  • SS1/23 sets five principles: identification and classification, governance, development and use, independent validation, and risk mitigants.

Reaction

Check what stuck

Question 1 · objective 2

The bank scores a new model 3 for materiality and 2 for complexity. Tiers are set by the product: 6 or more is tier 1, 3 or 4 is tier 2. Which tier is it, and how often is it validated?

Question 2 · objective 2

Three of the bank's 39 material models are overdue for validation. If one more falls overdue, what does the KRI read, in %?

%

Question 3 · objective 1

Why did the early-repayment charge error go unnoticed for over three years?

Question 4 · objective 3

The fitted line is CPR = 3.0 + 5.8 × incentive. What does it predict when the incentive is −1 percentage point, in %?

%

Question 5 · objective 3

The prepayment line has an R² of 0.85. What does that tell the validator?

Glossary

Model inventory
Model inventory: the firm's list of every model, with its owner, use, risk tier, validation dates and known limitations.
Risk tier
Risk tier: a model's place in the inventory by materiality (how much depends on it) and complexity (how much can go wrong); it sets how deep and how often validation is.
Independent validation
Independent validation: a review, by people not involved in building or using the model, of its concepts, data, implementation and performance.
End-user computing
End-user computing (EUC): spreadsheets and other tools built by business users outside the firm's controlled systems.
Extrapolation
Extrapolation: using a model for inputs outside the range of the data it was built on.
Prediction interval
Prediction interval: the range in which a single new observation should fall with a stated probability, if the model is right.
Post-model adjustment
Post-model adjustment: an override or overlay applied to a model's output to correct a known weakness until the model is fixed.
Conditional prepayment rate
Conditional prepayment rate (CPR): the share of a mortgage book repaid early in a year, as an annual percentage.

Deep Dives & Origins

Deep Dives

Origins

  • Prudential Regulation Authority, SS1/23 Model risk management principles for banks, 17 May 2023 (updated April 2026).
  • Forvis Mazars, Model risk management series on SS1/23 principles one and two; Deloitte, Countdown to compliance with SS1/23 Model Risk Management: Implications for Board members, 14 September 2023.
  • IFRE, Derivatives: JPM report unveils VaR errors, 2013.

Watch this lesson

Video version

Prefer to watch? The same lesson as a 12-minute video will sit here, made from this page's script.

Back to top ↑