Technology and change risk · RiskAtoms Academy

Know these first

L2.2.05: Operational risk
L3.5.09: Intraday liquidity

By the end you will be able to

1Describe the sources of technology risk (system failures, change, legacy technology, capacity) and why change is the riskiest moment in a system's life.
2Explain the controls on change (testing, approval, staged release, back-out plans, monitoring) and use KRIs and a change portfolio to spot trouble early.
3Compute the availability of a chain of systems, the hours down a year, and how often one outage outlasts an impact tolerance.
4Trace the book's October 2025 payments outage from its warning signs to its cost.

Why it matters

On 14 October 2025 RiskAtoms Bank upgraded the database under its payments platform. The upgrade had been put off twice and was approved without a tested way to undo it. It failed. Payments were down for 31 hours, and the bank booked £18.4m of compensation, scheme charges and recovery costs: 63% of all its operational losses that year. The warnings had been flashing since May.

Core explanation

Systems fail most when someone changes them

Illustrative dataRiskAtoms Bank plc's operational risk book

The same £67bn bank as Modules 3.5 and 3.6, reporting at 31 December 2025: ten years of internal loss events above £10,000 (2016 to 2025), an RCSA, ten key risk indicators, seven workshop scenarios, five important business services and five suppliers. Every event, person, supplier and figure is made up for teaching and seeded so it never changes; none is any bank's data. Every lesson in Module 3.7 uses this book, so the figures agree.

The book's technology and change category holds 33 losses, a net £21.5m over 2016–2025, and the October outage is 86% of it. It also holds 24 technology near misses, such as failed changes backed out before customers noticed.

Technology risk is losses from systems that stop working or work wrongly. Its outages land in business disruption and system failures; its processing errors in execution, delivery and process management (L2.2.05: Operational risk). It has four main sources:

Change. A change to a live system is its riskiest moment. In a 2021 review of 23 firms, the FCA counted over a million production changes in 2019; 1.6% caused an incident, rising to 3.8% for major changes, and changes were the root cause of 24% of high-severity incidents.
Legacy technology. Over 90% of the firms in that review still relied on it in some form, and those with more legacy technology had higher change failure rates.
Capacity and ageing. Systems that slow down or fall over as volumes grow, or as parts age out of support.
Dependencies. A service needs a chain of systems, many run by suppliers (L3.7.10: Third-party and outsourcing risk). It is down when any link is.

The October 2025 outage, from first signal to cost

TimelineRiskAtoms Bank's payments platform, 2025

1 / 7

May

Patches go amber

Critical security patches more than 14 days overdue reach 5, the amber threshold. The patch backlog is a sign that the platform team has more work than people.

KRI	Mar	Apr	May	Jun	Jul	Aug	Sep	Oct	Nov	Dec
Critical patches overdue	2	3	5	7	9	11	12	8	4	3
Payments availability, %	99.97	99.96	99.96	99.94	99.82	99.41	99.32	95.60	99.71	99.94
Failed payments per 10,000	9.8	9.0	6.7	11.5	13.2	16.8	18.9	41.6	14.1	6.9
Complaints per 1,000 accounts	1.49	1.53	1.41	1.42	1.47	1.51	1.61	2.64	1.94	1.45

Green, amber and red against each KRI's thresholds, 2025 (L3.7.05: Key risk indicators in practice). October's availability of 95.6% is the outage itself.

The RCSA rates the risk (R03, “Failed IT change disrupts payments”) at 16 inherent and 12 residual: still High. Only the monitoring control (C07) works in both design and operation, and monitoring tells you a platform is down; it does not stop it going down (L3.7.04: Controls: design, testing and assurance).

Controlling change

ExploreA change, from idea to live

Click each stage to see the control and how it fails.

Assess

Classify the change: what it touches, how big, how new. A change to a system under an important business service is high-risk by default. Deferring a change does not make it smaller; it usually makes it bigger.

Sort itWhich control would have caught it?

Each failure got past the others. Which control is the one that would have stopped it or limited it?

A new interest calculation has a rounding error that only shows on accounts with more than one product.
A configuration update crashes every server it reaches, and it is sent to all of them at once.
A database upgrade fails halfway; the old version cannot be restored quickly and the platform is down for a day.
A memory leak slowly fills a server over a week until payments start to time out.
A migration moves five million customers to a new platform in one weekend; problems appear on Monday.

Illustrative dataLesson extraA change portfolio

Six changes the bank has in flight, scored by a teaching rule: important business services touched × (size + novelty, each 1 to 3) × 2 if there is no rehearsed back-out plan × (1 + 0.5 for each deferral). High at 24 or more, Medium at 8 or more. Real firms use richer rules; the point is to rank changes so the riskiest get the most scrutiny.

Change	Services	Size + novelty	Back-out rehearsed	Deferrals	Score	Rating
Payments database upgrade, as approved in October 2025	IBS1, IBS4	2 + 2	No	2	32	High
Core banking version upgrade (Quillmoor)	IBS1, IBS2, IBS3, IBS5	3 + 1	Yes	1	24	High
Second-region failover for payments (A05)	IBS1, IBS4	3 + 3	Yes	0	12	Medium
Mortgage servicing moves to a new platform	IBS5	3 + 2	No	0	10	Medium
Payments database upgrade, re-planned under A04	IBS1, IBS4	2 + 2	Yes	0	8	Medium
Monthly mobile app release	IBS2	1 + 1	Yes	0	2	Low

The October upgrade as approved scores 32, the highest in the portfolio. Re-planned with a rehearsed back-out and no more deferrals, the same upgrade scores 8. The core banking upgrade scores high for a different reason: it touches four of the five important business services.

The formal version

Availability. A part with $f$ outages a year, each lasting on average $r$ hours (its mean time to restore), is down about $f r$ hours of the $8{,}760$ in a year:

A = 1 - \frac{f\,r}{8\,760}, \qquad \text{hours down a year} = (1 - A) \times 8\,760

A chain. A service that needs every part is up only when all of them are. If failures are independent, availabilities multiply:

A_{\text{service}} = A_1 \times A_2 \times \cdots \times A_n \;<\; \min_i A_i

How long one outage lasts. An impact tolerance limits the length of a single disruption, not the total. If outage lengths are exponential with mean $r$ , the chance that one runs past $T$ hours, and the expected number that do in a year, are

P(D > T) = e^{-T/r}, \qquad \text{breaches a year} = \sum_i f_i\, e^{-T/r_i}

Availability depends on $f r$ only; breaches depend on $r$ through $e^{-T/r}$ , which is tiny when $r$ is much shorter than $T$ and grows fast as $r$ approaches it. A part that rarely fails but takes long to restore can dominate the breaches while barely denting availability. Real outage lengths have heavier tails than the exponential, so this understates long outages.

Try it

The parts behind Faster Payments, with made-up rates. Look at the bars: the cloud region is a small part of the hours down but almost all of the tolerance breaches. Switch on failover. Then switch to CHAPS, with its 4-hour tolerance, and try to get breaches below one in ten years.

Try itAvailability is not the same as resilience

Service

Cloud region fails

Payments platform (RiskAtoms Bank)

Outages a year3.0

Mean hours to restore2.0 h

Hours down a year6.0

Outages over 24 h a year0.0000

Core banking (Quillmoor Systems)

Outages a year1.0

Mean hours to restore4.0 h

Hours down a year4.0

Outages over 24 h a year0.0025

Cloud region (Cirrovane Cloud)

Outages a year0.5

Mean hours to restore8.0 h

Hours down a year4.0

Outages over 24 h a year0.0249

Scheme gateway (Velmarra Payments)

Outages a year1.0

Mean hours to restore1.0 h

Hours down a year1.0

Outages over 24 h a year0.0000

Availability of the service99.83%

Outages longer than 24 h a year0.027

Hours down a year15.0

One tolerance breach every37 years

Outages a year5.5

October 2025's one outage31 h

The service is up only when every part is; failures are taken as independent. Each outage's length is drawn from an exponential distribution with the part's mean, so long outages are possible but rare; real outages have heavier tails. Illustrative rates, not the bank's.

Worked example

Faster Payments: available, but resilient?

The four parts behind Faster Payments (IBS1), with the lesson's rates; impact tolerance 24 hours. Payments cannot yet fail over to the second cloud region.

From parts to availability to breaches

Step 1 of 5

Hours down a year, part by part

Part	Outages a year	Mean hours	Hours down	Availability
Payments platform (RiskAtoms Bank)	3.0	2	6	99.932%
Core banking (Quillmoor Systems)	1.0	4	4	99.954%
Cloud region (Cirrovane Cloud)	0.5	8	4	99.954%
Scheme gateway (Velmarra Payments)	1.0	1	1	99.989%

CHAPS is harder

CHAPS (IBS4) needs fewer parts but has a 4-hour tolerance. With the same rates it is 99.87% available and expects 0.73 breaches a year, about one every 1.4 years; failover only brings that to 0.49. A short tolerance needs fast restores, not just rare failures.

Regulatory lens

The Basel Committee's 2021 Principles for the Sound Management of Operational Risk give change management a principle of its own (Principle 7): a comprehensive, properly resourced process for assessing the risks of new products, activities, processes and systems, with monitoring after they go live. Principle 10 asks for a robust programme for information and communication technology risk. In the UK, outages at banks are watched closely: the Treasury Committee found at least 158 IT failures at nine big banks between January 2023 and February 2025, together lasting over 800 hours. The operational resilience rules ask whether each important business service stays within its impact tolerance in severe but plausible scenarios (L3.7.15: Operational resilience), and recovery planning is in L3.7.16: Business continuity and crisis management. The October outage enters capital through the loss component (L3.7.07: Measuring frequency and severity).

Spark Atoms

Spark AtomTSB, April 2018

TSB moved about five million customers from Lloyds Banking Group's systems to a new platform built by Sabis, a company owned by its parent, Sabadell, in one weekend. A significant proportion of its 5.2 million customers were affected by the disruption that followed. An independent review by Slaughter and May found the platform was not ready and testing was insufficient. In December 2022 the FCA and PRA fined TSB £48.65m in total, including for failing to manage the risks of its IT outsourcing; it had already paid £32.7m in redress.

Spark AtomCrowdStrike, July 2024

On 19 July 2024 a faulty content update to CrowdStrike's security software crashed about 8.5 million Windows devices, under 1% of the total, and disrupted banks and airlines. CrowdStrike's root cause analysis found a mismatch between 21 input fields the update supplied and 20 the sensor expected, and introduced staged deployment rings and more testing. One unstaged change, everywhere at once.

Key takeaways

Change is the riskiest moment in a system's life: the FCA found it behind a quarter of high-severity incidents.
The book's October 2025 outage (31 hours, £18.4m) followed five months of amber and red KRIs and an upgrade approved without a tested back-out plan.
Test, approve, stage, be able to back out, monitor: each catches a different failure, and approval alone can become a formality.
A chain of systems is less available than any of its parts: multiply the availabilities.
Availability counts hours; impact tolerances limit the length of one outage. Restore time, not failure rate, decides whether a service breaches.

Reaction

Check what stuck

Question 1 · objective 3

A payment needs two systems in a chain: one available 99.9% of the time, the other 99.5%. Failures are independent. What is the availability of the payment service, in %?

Question 2 · objective 3

A system is available 99.9% of the time. How many hours a year is it down?

hours

Question 3 · objective 3

A platform has 3 outages a year, each lasting on average 2 hours (exponentially distributed). How many outages a year are expected to last longer than 4 hours?

Question 4 · objective 3

Platforms A and B are both available 99.9% of the time. A has one outage of about 8.8 hours a year; B has 35 outages of 15 minutes. The service has a 4-hour impact tolerance. Which breaches it?

AA onlyBB onlyCBothDNeither: they have the same availability

Question 5 · objective 2

A software vendor pushes an update to every customer's machines at once, and it crashes them. Which control would most have limited how many machines it reached?

AA staged rollout: a small group first, then wider once it is seen to workBA change advisory board meetingCA higher availability targetDMore detailed post-incident review

Glossary

Technology risk: The risk of loss from failures of systems and infrastructure, and from changes to them: outages, failed upgrades, ageing technology, capacity running out.
Change: Any alteration to a live system: a software release, a configuration change, an upgrade, a migration. Most outages start with one.
Change advisory board: Change advisory board: the group that reviews and approves higher-risk changes before they go live.
Back-out plan: The tested steps to undo a change and return a system to its last working state if the change goes wrong.
Staged release: Releasing a change to a small share of systems or users first, watching, then widening: a bad change hits few before it is stopped.
Emergency change: A change made outside the normal process to fix an urgent problem, with less testing and review.
Legacy technology: Old systems that are hard to change, test or support, often because few people still understand them.
Availability: The share of the time a service is working: 99.9% means about 8.8 hours down a year.
Mean time to restore: Mean time to restore: the average length of an outage, from failure to service back.
Impact tolerance: The longest disruption to an important business service the firm can tolerate before customers or markets suffer intolerable harm (L3.7.15).

Deep Dives & Origins

Deep Dives

Financial Conduct Authority, Implementing technology change, multi-firm review, February 2021. Introductory.
Slaughter and May, independent review of TSB's 2018 migration to a new IT platform, November 2019. Intermediate.

Origins

Financial Conduct Authority, Implementing technology change, 5 February 2021.
Basel Committee on Banking Supervision, Revisions to the Principles for the Sound Management of Operational Risk, BIS, March 2021, Principles 7 and 10.
Bank of England, “TSB fined for operational resilience failings”, 20 December 2022.
Computer Weekly, “TSB programme pulled apart in report on IT meltdown”, November 2019.
CrowdStrike, Channel File 291 incident: root cause analysis, executive summary, 6 August 2024.
TechCrunch, “Microsoft says 8.5M Windows devices were affected by CrowdStrike outage”, 20 July 2024.
Computer Weekly, “Big bank systems down for over 800 hours in last two years due to IT outages”, March 2025.

Watch this lesson

Video version

Prefer to watch? The same lesson as a 12-minute video will sit here, made from this page's script.

Previous · L3.7.08

Cyber risk

Next · L3.7.10

Third-party and outsourcing risk