Default is a yes-or-no event, but a lender needs a probability for each borrower. A straight line through 0s and 1s soon predicts a default probability below zero or above one. Logistic regression bends the line into an S that stays between the two, and it is the model behind most credit scorecards and many PD models.
A binary outcomeBinary outcome: a variable that is either 1 (the event happened, such as a default) or 0 (it did not). has a probability p, and p cannot leave the range 0 to 1. Turn it into oddsOdds: p/(1 − p), the probability of the event divided by the probability it does not happen. A 20% chance is odds of 0.25, one to four., p/(1 − p), and the range becomes 0 to infinity. Take the log, and the log-oddsLog-odds (logit): ln(p/(1 − p)). It runs over the whole number line, with 0 at p = 50%, negative below and positive above. cover the whole number line. A straight line in the characteristics can be fitted there with no risk of an impossible answer. Coming back, the logistic functionLogistic function: σ(z) = 1/(1 + e^−z), the S-shaped curve that turns log-odds z into a probability between 0 and 1. turns any log-odds into a probability.
Log-odds on the horizontal axis, probability on the vertical. The dots are the rows of the table below.
Probability p
Odds p/(1 − p)
Log-odds ln(odds)
1%
0.010
−4.595
5%
0.053
−2.944
20%
0.250
−1.386
50%
1.000
0.000
80%
4.000
1.386
The scale is symmetric: 20% and 80% sit at log-odds of −1.386 and 1.386. In the tails it stretches: going from 5% to 1% takes 1.65 units of log-odds, more than from 20% to 50%. That is why one coefficient can describe a whole loan book, from prime borrowers at a fraction of a per cent to the riskiest tail.
The rules
The model.Logistic regressionLogistic regression: a model in which the log-odds of a 0/1 outcome are a linear function of characteristics: ln(p/(1 − p)) = β₀ + β₁x₁ + … + βₖxₖ. puts a linear function of the characteristics on the log-odds scale.
Fitting it. Maximise the Bernoulli log-likelihood, as in M-D08: Maximum likelihood estimation. The score equations have no closed form, so Newton-RaphsonNewton-Raphson: an iterative method that steps by the slope divided by the curvature. For logistic regression it usually converges in a handful of steps. solves them.
Newton-Raphson starts from fixed values, usually all coefficients at zero, which give every borrower a 50% probability. Each step fits a weighted least squares regression whose weights p(1 − p) come from the current probabilities, which is why it is also called iteratively reweighted least squares. Because the logistic log-likelihood is concave, there is a single peak and the steps reach it in a handful of iterations, unless a characteristic gives separationSeparation: a characteristic that splits defaulters from non-defaulters perfectly. The likelihood then keeps rising as the coefficient grows, and no finite maximum exists., in which case there is no peak at all.
Reading it. A coefficient is a change in log-odds; e to its power is an odds ratioOdds ratio: the factor by which the odds change when a characteristic rises by one unit, e^β. Above 1 raises the odds, below 1 lowers them.. The Wald testWald test: z = β̂/SE(β̂), compared with the standard normal. |z| above 1.96 rejects a zero coefficient at the 5% level. asks whether it is distinguishable from zero.
The formal version
Yi∈{0,1},pi=P(Yi=1∣xi)=1+e−xi⊤β1ln1−pipi=xi⊤β=β0+β1xi1+⋯+βkxikℓ(β)=i=1∑n[yilnpi+(1−yi)ln(1−pi)]∇ℓ(β)=X⊤(y−p),−∇2ℓ(β)=X⊤WX,W=diag(pi(1−pi))β(m+1)=β(m)+(X⊤W(m)X)−1X⊤(y−p(m)),Cov(β^)=(X⊤W^X)−1zj=SE(β^j)β^j≈N(0,1) under βj=0,ORj=eβj,95% CI: eβ^j±1.96SE(β^j)
yi is the outcome of borrower i and xi their characteristics, with a leading 1 for the intercept; X stacks the xi as rows; β holds the k + 1 coefficients; pi is the fitted probability; m counts Newton-Raphson iterations; OR is the odds ratio for a one-unit rise in xj with the others held fixed. The normal approximation for z holds in large samples. Sources: Hosmer, Lemeshow and Sturdivant (2013), chapters 1 to 3; McCullagh and Nelder (1989), the chapter on binary data; Agresti (2013), chapters 4 and 5.
Try it
50 illustrative borrowers, each with a debt-to-income ratio between 12% and 60%, of whom 15 defaulted within a year. The data are synthetic: the outcomes were drawn with a fixed seed from a logistic model with β₀ = −5 and β₁ = 0.1. Move the intercept and the slope until the curve follows the band default rates, and watch ℓ climb. Then jump to the maximum, or click anywhere on the map.
Try itFit a logistic curve by eye
defaulted (top) did not (bottom) default rate in each 10-point bandDebt-to-income (%) across. Solid: your curve. Dashed purple: the maximum likelihood fit.Slope β₁ across, intercept β₀ up. The stronger the shade, the closer ℓ is to its maximum; the strongest region is within 1.92 of it. The open circle is your choice.
Log-likelihood ℓ at your values−24.192
Maximum ℓ̂−21.533
LR = 2(ℓ̂ − ℓ)5.32
Your pointoutside
Predicted PD at 20% debt-to-income11.9%
Predicted PD at 50% debt-to-income37.8%
Odds ratio for +10 points, e^(10β₁)1.65
Sum of predicted PDs (defaults: 15 of 50)12.36
The high ground on the map is a long diagonal ridge: a steeper slope with a lower intercept fits almost as well. The two estimates are strongly linked, with a correlation of −0.96, because every debt-to-income ratio is far from zero. At the maximum the predicted PDs add up exactly to the number of defaults, as the score equation for the intercept requires.
Worked examples
1. Reading a fitted model by hand
Step 1 of 3
1
The model
A lender's model gives log-odds of default = −4.2 + 0.08 × debt-to-income (in %). Take borrowers at 30%, 40%, 50%.
2. Fitting the Try it data by Newton-Raphson
Step 1 of 4
1
The iterations
Step
β₀
β₁
ℓ(β)
0
0.0000
0.00000
−34.6574
1
−3.3141
0.07147
−22.6521
2
−4.6788
0.10049
−21.5984
3
−5.1241
0.10986
−21.5335
4
−5.1629
0.11067
−21.5331
5
−5.1632
0.11068
−21.5331
3. Two characteristics
Step 1 of 3
1
The data
400 illustrative borrowers, generated with seed 2026: debt-to-income as before, and a flag set to 1 for the 81 who missed a payment in the last year. 71 defaulted. The same Newton-Raphson converges in 7 steps.
Common mistakes
Each line below is a tempting slip. Spot what is wrong, then check.
Reaction
Check what stuck
Question 1 · objective 1
A borrower has a 20% probability of default. What are the log-odds? Give three decimals.
Question 2 · objective 1
A fitted model gives log-odds of default = −2 + 0.05 × (debt-to-income in %). What is the predicted probability for a borrower at 30%? Give a decimal to three places.
Question 3 · objective 2
A characteristic has a coefficient of 0.4. By what factor do the odds of default change when it rises by one unit? Give three decimals.
Question 4 · objective 3
A coefficient is estimated at 0.12 with a standard error of 0.04. What is its Wald z statistic?
Question 5 · objective 2
A model has a coefficient of 0.08 on debt-to-income, in percentage points. Which statement about a borrower whose ratio rises by one point is right?
Glossary
Binary outcome
Binary outcome: a variable that is either 1 (the event happened, such as a default) or 0 (it did not).
Odds
Odds: p/(1 − p), the probability of the event divided by the probability it does not happen. A 20% chance is odds of 0.25, one to four.
Log-odds (logit)
Log-odds (logit): ln(p/(1 − p)). It runs over the whole number line, with 0 at p = 50%, negative below and positive above.
Logistic function
Logistic function: σ(z) = 1/(1 + e^−z), the S-shaped curve that turns log-odds z into a probability between 0 and 1.
Logistic regression
Logistic regression: a model in which the log-odds of a 0/1 outcome are a linear function of characteristics: ln(p/(1 − p)) = β₀ + β₁x₁ + … + βₖxₖ.
Odds ratio
Odds ratio: the factor by which the odds change when a characteristic rises by one unit, e^β. Above 1 raises the odds, below 1 lowers them.
Newton-Raphson
Newton-Raphson: an iterative method that steps by the slope divided by the curvature. For logistic regression it usually converges in a handful of steps.
Wald test
Wald test: z = β̂/SE(β̂), compared with the standard normal. |z| above 1.96 rejects a zero coefficient at the 5% level.
Separation
Separation: a characteristic that splits defaulters from non-defaulters perfectly. The likelihood then keeps rising as the coefficient grows, and no finite maximum exists.
McCullagh and Nelder, Generalized Linear Models: logistic regression as one member of a family that also holds linear, Poisson and gamma regression. Advanced.
Origins
David W. Hosmer, Stanley Lemeshow and Rodney X. Sturdivant, Applied Logistic Regression, 3rd edition, Wiley, 2013 (chapters 1 and 2, the model, its fitting and tests; chapter 3, interpreting the fitted model).
Peter McCullagh and John A. Nelder, Generalized Linear Models, 2nd edition, Chapman & Hall, 1989 (the chapter on binary data).
Alan Agresti, Categorical Data Analysis, 3rd edition, Wiley, 2013 (chapter 4, generalized linear models; chapter 5, logistic regression).