A twelve-month PD answers whether a borrower defaults within a year. Lifetime provisioning, prepayment and deposit models ask when: how many defaults in month 7, how many by year 5? And the data are always cut short: loans repay early, and recent ones have only a few months of history. Survival analysis answers the question of when, using every loan for as long as it was seen.
Ten loans, followed for up to twelve months. Four defaulted. Three were still performing at month 12. The other three repaid early. Those last three are censoredCensoring: when the event time is only known to be later than the last time a subject was observed, for example a loan that repaid early or was still performing when the data were cut.: we know they had not defaulted by the time they left, and nothing after.
Months since origination across. A cross marks a default; an open circle marks the last month a loan was seen performing (censored).
What is the twelve-month PD? Counting the censored loans as survivors gives 4 in 10, 40%: too low, because they might still have defaulted. Dropping them gives 4 in 7, 57.1%: too high, because loans that leave early are mostly ones that had not defaulted. The fix is to use each loan in every month it was at riskAt risk: still observed and without the event at the start of a period, so able to have it then., estimate a default rate month by month, and chain the months together. That gives 46%, as worked example 1 shows.
The rules
Three ways to describe the same time. The time to eventTime to event: the time from a starting point, such as origination, to an event, such as default. Written T. T has a survival functionSurvival function S(t): the probability that the event has not happened by time t, P(T > t)., a hazardHazard: the rate of events at time t among those still at risk at t. In monthly data, the probability of the event in month t given none before it. and a cumulative hazardCumulative hazard H(t): the hazard added up from 0 to t. Survival is S(t) = e^−H(t).. Each one determines the others, as M-C14: Survival and marginal default probabilities showed for a constant hazard.
Estimating survival with censoring. The Kaplan-Meier estimatorKaplan-Meier estimator: Ŝ(t) = Π (1 − d/n) over the event times up to t, where d is the number of events and n the number at risk. It uses censored subjects for as long as they were observed. and Greenwood's formulaGreenwood's formula: the usual variance estimate for the Kaplan-Meier curve, Ŝ(t)² Σ d/(n(n − d))..
It is unbiased only if censoring is unrelated to the risk of default: a loan that leaves must be no more and no less likely to have defaulted than those that stay. Loans sold because they looked weak break that.
Adding characteristics. A discrete-time hazard modelDiscrete-time hazard model: a logistic regression on one row per subject per period, with the period's outcome (event or not) as the 0/1 response. is a logistic regression on loan-months.
T is the event time with density f; tj are the times at which events occur, dj the events and nj the subjects at risk just before tj; ti is the last period subject i was observed and δi is 1 if it ended in the event; xi are its characteristics. Censoring is assumed independent of the event time given x. Sources: Kaplan and Meier (1958); Greenwood (1926) for the variance; Kalbfleisch and Prentice (2002), chapters 1 and 2; Allison (1982) for discrete-time hazard models fitted as logistic regressions.
Try it
120 illustrative loans followed monthly for up to 24 months: 23 defaulted, 65 were censored early and 32 were still performing at month 24. The cohort is synthetic, drawn with seed 1323: each month a performing loan defaulted with probability 0.9% (2% for the 30% with a high loan-to-value ratio), otherwise repaid early with probability 1%, and the data stop after 6 to 24 months depending on when it was originated. Switch between the three treatments of censored loans, and move the month to see the table behind the curve.
Try itCensored loans: keep, drop or count as survivors
Loans censored before month 24 are
Kaplan-Meier (ticks: months with censoring) curve the data were drawn fromMonths since origination across, share of loans not yet defaulted up.
Loans at risk at the start of month 1274
Defaults in the month2
Censored in the month4
Hazard d/n2.7%
Kaplan-Meier Ŝ ± Greenwood SE0.834 ± 0.036
Naive survivalpick a naive method
Survival the data were drawn from0.863
Loans used120 of 120
Survival to
Kaplan-Meier (SE)
Censored dropped
Censored as survivors
Drawn from
month 12
0.834 (0.036)
0.673
0.850
0.863
month 24
0.757 (0.047)
0.582
0.808
0.748
Dropping censored loans keeps only 55 of 120 and puts the 24-month PD at 41.8%; counting them as survivors gives 19.2%. Kaplan-Meier gives 24.3%, with a standard error of 4.7 points, against 25.2% in the model the data came from. The early repayments are treated as censoring, so this is the default curve a loan would follow if it could not repay early.
Worked examples
1. Kaplan-Meier for ten loans, by hand
Step 1 of 3
1
The table
Month
At risk n
Defaults d
Censored
Hazard d/n
Ŝ(t)
3
10
1
0
0.100
0.900
4
9
0
1
0.000
0.900
6
8
2
0
0.250
0.675
8
6
0
1
0.000
0.675
9
5
1
0
0.200
0.540
10
4
0
1
0.000
0.540
12
3
0
3
0.000
0.540
2. From a monthly hazard to a lifetime PD
Step 1 of 4
1
Constant monthly hazard
A segment of personal loans has a monthly default hazard of 1.2% for as long as a loan performs. Survival to month t is (1 − 0.012)t.
3. A discrete-time hazard model on the Try it cohort
Step 1 of 3
1
Loan-months
Each loan becomes one row per month observed. Three loans:
Loan
Months observed
Ended in
High loan-to-value
Rows: outcome y by month
A
3
default
yes
0, 0, 1
B
4
censoring
no
0, 0, 0, 0
C
2
censoring
yes
0, 0
9 rows in all; only the month of a default has y = 1. The whole cohort makes 1,720 loan-months.
Common mistakes
Each line below is a tempting slip. Spot what is wrong, then check.
Reaction
Check what stuck
Question 1 · objective 1
A loan's monthly default hazards are 2%, 3% and 1% in its first three months. What is the probability it survives all three? Give four decimals.
Question 2 · objective 3
At the start of a month, 80 loans are at risk and 4 of them default during it. What is the estimated hazard for that month? Give a decimal.
Question 3 · objective 3
The Kaplan-Meier estimate was 0.9 before a month in which 2 of the 50 loans at risk default. What is it after that month? Give three decimals.
Question 4 · objective 1
The cumulative hazard to five years is 0.2. What is the five-year survival probability? Give four decimals.
Question 5 · objective 2
An analyst estimates a two-year PD from a book where many loans were originated recently, using only loans that either defaulted or were observed for the full two years. What goes wrong?
Glossary
Time to event
Time to event: the time from a starting point, such as origination, to an event, such as default. Written T.
Survival function
Survival function S(t): the probability that the event has not happened by time t, P(T > t).
Hazard
Hazard: the rate of events at time t among those still at risk at t. In monthly data, the probability of the event in month t given none before it.
Cumulative hazard
Cumulative hazard H(t): the hazard added up from 0 to t. Survival is S(t) = e^−H(t).
Censoring
Censoring: when the event time is only known to be later than the last time a subject was observed, for example a loan that repaid early or was still performing when the data were cut.
At risk
At risk: still observed and without the event at the start of a period, so able to have it then.
Kaplan-Meier estimator
Kaplan-Meier estimator: Ŝ(t) = Π (1 − d/n) over the event times up to t, where d is the number of events and n the number at risk. It uses censored subjects for as long as they were observed.
Greenwood's formula
Greenwood's formula: the usual variance estimate for the Kaplan-Meier curve, Ŝ(t)² Σ d/(n(n − d)).
Discrete-time hazard model
Discrete-time hazard model: a logistic regression on one row per subject per period, with the period's outcome (event or not) as the 0/1 response.
Singer and Willett, Applied Longitudinal Data Analysis: a patient, applied treatment of discrete-time hazard models, from person-period data to interpretation. Intermediate.
Kalbfleisch and Prentice, The Statistical Analysis of Failure Time Data: continuous-time models, the Cox proportional hazards model and competing risks. Advanced.
Origins
E. L. Kaplan and Paul Meier, “Nonparametric estimation from incomplete observations”, Journal of the American Statistical Association 53(282), 457–481, 1958.
Major Greenwood, The Natural Duration of Cancer, Reports on Public Health and Medical Subjects 33, HMSO, 1926.
John D. Kalbfleisch and Ross L. Prentice, The Statistical Analysis of Failure Time Data, 2nd edition, Wiley, 2002 (chapter 1, introduction, and chapter 2, failure time models).
Paul D. Allison, “Discrete-time methods for the analysis of event histories”, in Samuel Leinhardt (editor), Sociological Methodology 1982, Jossey-Bass, 61–98, 1982.
Judith D. Singer and John B. Willett, Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence, Oxford University Press, 2003.