PD back-testing for credit decisions
Enter each grade's PD, loans and observed bads to test calibration by grade and across the portfolio. See the results in a minute.
Grades with low PD
4 of 8 grades have more bads than their PD allows at 95% confidence. The portfolio shows 543 bads against 455 expected, with a Jeffreys p-value of < 0.0001.
Predicted PD and observed bad rate
The bar is the 95% Jeffreys interval for the observed rate. A dashed line below the bar means the PD is too low for that grade.
Grade test results
| Grade | Status | Predicted PD | Observed rate | Jeffreys p-value | Binomial p-value | Loans | Observed bads | Expected bads |
|---|---|---|---|---|---|---|---|---|
| A | Within range | 0.30% | 0.25% | 0.6541 | 0.7245 | 2,400 | 6 | 7.2 |
| B | Within range | 0.60% | 0.55% | 0.6325 | 0.6768 | 3,100 | 17 | 18.6 |
| C | Within range | 1.0% | 1.1% | 0.2980 | 0.3273 | 3,500 | 38 | 35.0 |
| D | Within range | 1.8% | 2.0% | 0.1676 | 0.1847 | 3,000 | 61 | 54.0 |
| E | Too low at 95% | 3.0% | 3.7% | 0.0308 | 0.0349 | 2,400 | 88 | 72.0 |
| F | Too low at 95% | 5.0% | 6.1% | 0.0196 | 0.0222 | 1,700 | 104 | 85.0 |
| G | Too low at 99% | 8.5% | 11.0% | 0.0021 | 0.0024 | 1,100 | 121 | 93.5 |
| H | Too low at 95% | 15.0% | 18.0% | 0.0218 | 0.0247 | 600 | 108 | 90.0 |
| Portfolio total | 2.6% | 3.1% | < 0.0001 | 17,800 | 543 | 455.3 |
A small p-value means the grade had more bads than its PD makes likely. A grade within range is not proven right, and a small grade has little power to show anything.
Get the results by email
Put the model in the credit decision and keep every decision and outcome, so you can run this check on your own grades each quarter. Start free
View formulas
Expected bads = Loans × Predicted PD Observed rate = Observed bads ÷ Loans Jeffreys p-value = Cumulative Beta distribution (Predicted PD; Observed bads + ½, Loans − Observed bads + ½) Binomial p-value = Chance of this many bads or more if the PD were right Hosmer–Lemeshow = Σ (Observed bads − Expected bads)² ÷ (Expected bads × (1 − Predicted PD)), χ², Degrees of freedom Too low at 99%: Jeffreys p-value < 0.01 Too low at 95%: < 0.05 Above observed at 95%: Chance of this many bads or fewer if the PD were right < 0.05
The ECB's instructions for reporting internal model validation results use the Jeffreys test for PD back-testing. The tests assume loans default independently. In a downturn, defaults move together and the tests reject more often than their level. The status levels follow this page's convention.
PD back-testing by rating grade
PD back-testing compares each grade's predicted PD with the bads observed in its loans. Enter the PD, loans and observed bads for each grade to see expected bads, the observed rate and the test results.
A grade with more bads than its PD makes likely can affect the price, provision and capital. Review the results for each grade and for the portfolio as part of your model calibration checks.
PD calibration questions
How do I back-test a PD model?
Enter each grade's predicted PD, loans and observed bads. The tool compares observed bads with expected bads using the Jeffreys and one-sided binomial tests.
What is the Jeffreys test for PD?
The Jeffreys test evaluates the Beta distribution for observed bads and loans at the predicted PD. A small p-value says the PD is too low for the bads observed.
How does the binomial test check a rating grade?
The one-sided binomial test calculates the chance of seeing that many bads or more if the PD were right. The tool reports a p-value for each grade.
What does the Hosmer-Lemeshow test show across grades?
The Hosmer-Lemeshow test compares expected and observed bads across grades. The tool reports its p-value for the grades together.
What should I do when a grade has more defaults than its PD?
Compare the grade's observed bad rate with its predicted PD and review its Jeffreys and binomial p-values. The tool marks grades whose PD is too low under its status convention.