Controls: design, testing and assurance · RiskAtoms Academy

By the end you will be able to

1Classify a control as preventive or detective, manual or automated, and say what each is good for.
2Plan a control test for design and for operation, and judge how much a sample can prove.
3Turn a failed test into a finding and an action, and track it to closure.
4Move a risk from inherent to residual on RiskAtoms Bank's RCSA, and read what the result depends on.

Why it matters

RiskAtoms Bank had a control for exactly the failure that cost it £18.4m in October 2025: “change advisory board approves high-risk changes with a tested back-out plan”. The October database upgrade was approved anyway, after two deferrals, without a tested back-out plan. On paper the risk was controlled. A control is only worth what a test of it shows.

Core explanation

From a control on paper to a control that works

L2.4.05: Risk and control self-assessment introduced the RCSA: rate a risk before controls, assess the controls, rate it after. This lesson is about the middle step, where most of the work is: what the key controls are, how the bank knows they work, and what happens when they do not.

Four kinds of control

A preventive control stops the event: a hard trading limit, a second approval. A detective control finds it fast enough to limit the damage: a reconciliation, a monitoring alert, independent price verification (L3.2.18: Valuation control and the daily risk report). Each can be automated or manual. The bank's 29 key controls:

RiskAtoms Bank's key controls by type and nature: number, and number effective in both design and operation
	Automated	Manual	Total
Preventive	6 (5 working)	10 (5 working)	16
Detective	6 (5 working)	7 (4 working)	13
All			29 (19 working)

Automated controls run every time, but only as well as they were built and only while nobody changes them; that is what IT general controls over change and access protect. Manual controls can judge what a system cannot, but people skip them when busy. Good design mixes both: an automated check that stops most errors and a manual review for what it misses. Sort six of the bank's controls:

Sort itPreventive or detective?

Each is one of RiskAtoms Bank's key controls.

Automated duplicate and limit checks on payment files
Daily automated reconciliations with ageing of breaks
Segregation of duties and four-eyes approval on dormant-account changes
Real-time monitoring of payment platform health with alerts
Call-back to a known number before changing payee details
Monthly independent review of suspense and dormant-account movements

Design and operation, and how each is tested

Every key control gets two verdicts. Design effectiveness: run as intended by someone competent, would it meet its objective? Operating effectiveness: did it actually run that way over the period? Testers use four procedures, from weakest evidence to strongest: inquiry (asking), observation (watching), inspection (examining records) and re-performance (running it again). Inquiry alone is never enough. A walkthrough of one transaction usually settles the design; operation needs a sample.

Design × operating results of the bank's 29 key controls (rows: design, columns: operating)
Design	Effective	Partially effective	Ineffective
Effective	19	5	1
Partially effective	0	3	1
Ineffective	0	0	0

19 of 29 pass both. 5 are well designed but only partly followed, 1 well designed but not followed at all (critical patching), and 4 have a design gap. Only controls that pass both tests reduce the residual rating.

How big a sample? A test samples runs and counts exceptions. If the control fails in a share $p$ of runs, a random sample of $n$ shows none with probability $(1-p)^n$ . Banks set sample sizes by how often a control runs; here is an illustrative standard, with the chance each sample misses a 10% failure rate:

RiskAtoms Bank's testing standard (illustrative extra) and what it can miss
Control runs	Runs a year	Sample	Chance of a clean sample if 10% fail
Many times a day	about 250 days × many	25	7.2%
Daily	about 250 a year	20	12.2%
Weekly	52 a year	5	59.0%
Monthly	12 a year	2	81.0%
Quarterly	4 a year	2	81.0%
Yearly	1 a year	1	90.0%

ExploreWill the test find the failure?

Share of runs where the control fails10%

Sample size25

Chance the sample shows no failure7.2%

Sample needed for 95% confidence29

Assumes the sample is drawn at random and each run fails independently. A clean sample is evidence, not proof: at a 10% failure rate a sample of 25 still comes back clean about one time in fourteen.

The call-back control failed its May 2025 test: it was skipped in 4 of 40 sampled payee changes (10%), one of them in the April email-compromise fraud. Rated partially effective, it got no credit.

Findings, actions and assurance

A failed test becomes a finding with a severity, and each finding gets an action with an owner and a due date. The second line tracks them and reports the overdue ones; an overdue action on a High finding is a risk in itself.

Findings and actions at 31 December 2025
Finding	Control	Severity	Issue	Action status
F01	C01	Medium	Quarterly restore test of the payments database took 30 hours, against a 12-hour target	A01 open (due 2026-06-30)
F02	C03	High	11 critical patches more than 14 days overdue, three on payments servers	A02 overdue (due 2025-11-30)
F03	C05	Medium	Call-back skipped in 4 of 40 sampled payee changes, including one in the April fraud	A03 closed (due 2025-09-30)
F04	C06	High	October database upgrade approved without a tested back-out plan after two deferrals	A04 open (due 2026-01-31); A12 open (due 2026-04-30)
F05	C08	Medium	Failover test covered mobile banking only; payments not yet switchable to the second region	A05 open (due 2026-03-31)
F06	C18	Low	Sales sampling for the wealth service fell below the planned 2% for two quarters	A06 closed (due 2025-06-30)
F07	C19	Medium	Forbearance outcomes not monitored by customer vulnerability	A07 closed (due 2025-12-31)
F08	C22	Medium	Data loss prevention rules not applied to the new file-sharing tool	A08 overdue (due 2025-10-31)
F09	C26	Medium	Three material models past their two-year validation date	A09 open (due 2026-02-28)
F10	C27	Low	Monitoring thresholds not set for two of the pricing models	A10 open (due 2026-01-31)
F11	C14	Low	Review evidence not retained for one month	A11 closed (due 2024-06-30)

8 actions are open and 2 overdue, which turns the overdue-actions KRI red (L3.7.05: Key risk indicators in practice). Assurance comes in layers: the first line tests its own key controls, the second line reviews those tests and runs its own themed reviews, and internal audit tests the whole system for the board (L3.7.02: The operational risk framework).

Illustrative dataRiskAtoms Bank plc's operational risk book

The same £67bn bank as Modules 3.5 and 3.6, at 31 December 2025, in pounds: ten years of internal loss events of £10,000 or more (2016–2025), an RCSA of 17 key risks and 29 controls, ten key risk indicators over 24 months, seven workshop scenarios and five important business services. Every event, person, supplier and figure is made up for teaching; none is any bank's data. The same book runs through every lesson in Module 3.7, so the figures agree.

The RCSA ratings, control results, findings and actions are the book's. The testing standard is an illustrative extra. The residual rule is the book's stated teaching rule, not an industry standard.

The formal version

The book rates likelihood $L$ and impact $I$ from 1 to 5 (L2.4.03: Assessing risks: likelihood and impact). With $P_w$ preventive and $D_w$ detective controls that are effective in both design and operation, its teaching rule is:

\begin{aligned} L_{\text{res}} &= \max\!\left(1,\; L_{\text{inh}} - P_w\right) \\ I_{\text{res}} &= \max\!\left(1,\; I_{\text{inh}} - D_w\right) \\ \text{score}_{\text{res}} &= L_{\text{res}} \times I_{\text{res}} \end{aligned}

Scores of 1 to 4 are Low, 5 to 9 Medium, 10 to 16 High and 20 to 25 Very high. The test sample follows from independent runs, each failing with probability $p$ :

\begin{aligned} P(\text{no exception in } n) &= (1-p)^n \\ P(\text{at least one}) = 1 - (1-p)^n &\ge c \\ (1-p)^n &\le 1 - c \\ n\,\ln(1-p) &\le \ln(1-c) \\ n &\ge \frac{\ln(1-c)}{\ln(1-p)} \end{aligned}

Derived on a few lines; the sign flips in the last step because $\ln(1-p) < 0$ . For $p = 10\%$ and $c = 95\%$ : $n \ge \ln 0.05 / \ln 0.9 = 28.43$ , so 29 runs; a sample of 25 leaves $7.2\%$ chance of missing it.

Try it

Pick a risk, change its controls' test results and watch the residual rating. Start with failed IT change (R03): what happens when the change control passes a retest? Then try mis-selling (R09): which control keeps it High?

Try itMove a risk from inherent to residual

Key risk

C06 Change advisory board approves high-risk changes with a tested back-out plan
Preventive · Manual · tested 2025-11-14
DesignOperating
C07 Real-time monitoring of payment platform health with alerts
Detective · Automated · tested 2025-11-14
DesignOperating

5

4RI

3

2

1

12345

Up: likelihood · across: impact · I inherent · R residual

Working preventive controls0

Working detective controls1

Inherent4 × 4 High (16)

Residual4 × 3 High (12)

Bank's key risks High after controls4 of 17

The book's teaching rule, not a standard: each preventive control effective in both design and operation takes 1 off likelihood; each such detective control takes 1 off impact; neither goes below 1. A partially effective control gets no credit. Changes you make here apply to every risk, so the count of High risks moves too.

Worked example

Failed IT change, before and after the outage

In November 2025, after the outage, the payments RCSA is rerun for failed it change disrupts payments (R03). Work each step before you open it.

R03 from inherent to residual

Step 1 of 5

1
Inherent risk
Without controls, a failed change disrupting payments is likely (once to four times a year, 4) and major (£10m to £50m, 4): 4 × 4 = High (16).

Why the rule is only a teaching rule

Real RCSAs rate residual risk by judgement, informed by the test results. A simple rule makes the link visible, but it treats every working control as worth exactly one point, ignores how much each control covers, and gives nothing for a partly working control.

Regulatory lens

The 2021 Basel principles ask for a control monitoring and assurance framework that tests key controls for design and operation across prevention, detection and response (paragraph 35(d)), and for a strong control environment (Principle 9): approval authorities, limit monitoring, safeguards over assets, training, reconciliations, segregation of duties and mandatory absence (paragraphs 47 to 51). For controls over financial reporting, auditors of US-listed companies follow the PCAOB's AS 2201, whose language of design and operating effectiveness and its four test procedures this lesson borrows. Control failures that cause losses feed the loss data and, through it, capital (M4.4: Counterparty, operational and large exposures).

Spark Atoms

Spark AtomTwo weeks away from the desk

The Basel principles list a mandatory minimum absence period among the traditional controls. A trader hiding positions must keep tending them; force two weeks away and someone else sees the book. RiskAtoms Bank pairs it with daily independent P&L reconciliation on the Markets desk (C15).

Spark AtomTesting a control once

Under AS 2201 an auditor may rely on an entirely automated control without retesting it every year, if tests show the IT general controls over program changes, access and operations work and the program has not changed since it was last tested. A manual control has no such shortcut: people do not stay the same from one year to the next.

Key takeaways

Controls are preventive or detective, automated or manual. RiskAtoms Bank has 16 preventive and 13 detective key controls; 12 are automated.
A control needs two verdicts, design and operation; only 19 of the bank's 29 pass both, and only those reduce residual risk.
Test evidence ranges from inquiry to re-performance; a random sample of n runs misses a failure rate p with probability (1 − p)^n.
Failed tests become findings with severities, and actions with owners and dates; overdue actions are themselves reported as a risk.
Failed IT change stays High (16 inherent, 12 residual) because its change control failed; a passed retest would make it 9, Medium.

Reaction

Check what stuck

Question 1 · objective 2

A control fails in 10% of its runs. A tester picks 25 runs at random. What is the chance that none of them shows a failure, in %?

%

Question 2 · objective 1

'Daily automated reconciliations with ageing of breaks' is which kind of control?

ADetective and automatedBPreventive and automatedCDetective and manualDPreventive and manual

Question 3 · objective 4

Internal fraud by operations staff (R07) is rated likelihood 3, impact 3. Its two controls, one preventive and one detective, are both effective in design and operation. Under the book's rule, what is the residual score?

Question 4 · objective 2

The critical-patching control (C03) is rated effective in design but ineffective in operation. What does that mean for the RCSA?

AThe control would work if it ran, but it did not run reliably, so it gets no credit until a retest passesBIt gets half credit because the design is soundCThe design must be rewritten before anything elseDThe rating can stay as it is because the policy exists

Question 5 · objective 2

How many randomly chosen runs must a tester sample to be at least 95% sure of seeing one or more failures, if 5% of runs fail?

Glossary

Control: Control: a step built into a process to stop a risk happening or to catch it quickly and limit the damage.
Key control: Key control: a control the bank relies on for a key risk, so it is documented, owned and tested.
Preventive control: Preventive control: stops the event happening, such as a hard limit or a second approval.
Detective control: Detective control: finds the event after it happens, quickly enough to limit the loss, such as a reconciliation or monitoring alert.
Automated control: Automated control: performed by a system every time, such as a payment file check.
Manual control: Manual control: performed by a person, such as a call-back or a review; it can be skipped or done badly.
Design effectiveness: Design effectiveness: whether the control, run as intended by someone competent, would meet its objective.
Operating effectiveness: Operating effectiveness: whether the control actually ran as designed, every time, over the period tested.
Walkthrough: Walkthrough: following one transaction through the process and its controls from start to end, to check the design.
Re-performance: Re-performance: the tester runs the control again independently and compares the result.
Exception: Exception: a sampled run in which the control did not operate as designed.
Finding: Finding: a documented control weakness with a severity rating, raised by a tester, the second line or internal audit.
Action: Action: the fix agreed for a finding, with an owner and a due date, tracked until it is closed.
IT general controls: IT general controls: controls over system changes, access and operations that keep automated controls working as built.
Assurance: Assurance: an independent opinion that controls are designed and working, for someone who relies on them.

Deep Dives & Origins

Deep Dives

Public Company Accounting Oversight Board, AS 2201: An Audit of Internal Control Over Financial Reporting That Is Integrated with An Audit of Financial Statements, paragraphs 42 to 50: how design and operating effectiveness are tested. Intermediate.
Basel Committee on Banking Supervision, Revisions to the Principles for the Sound Management of Operational Risk, BIS, March 2021, Principle 9. Introductory.

Origins

Basel Committee on Banking Supervision, Revisions to the Principles for the Sound Management of Operational Risk, Bank for International Settlements, 31 March 2021, paragraphs 35(d) and 47 to 51.
Public Company Accounting Oversight Board, AS 2201 (adopted in 2007 as Auditing Standard No. 5), paragraphs .42 to .50, .A3, .A7, .A11 and .B28 to .B33.

Watch this lesson

Video version

Prefer to watch? The same lesson as a 12-minute video will sit here, made from this page's script.