Size a champion challenger test for a credit strategy
Enter the current bad rate, the difference worth catching, the challenger's share and the monthly volume. See the loans and months the test needs, then check a finished test.
Months until readable result
The test fills in 4.1 months. The outcome window adds 6, so the result can be read after 10.1 months.
Result time by challenger share
Check a finished test
| Champion | Challenger | |
|---|---|---|
| Loans | ||
| Bad loans | ||
| Rate | – | – |
- Difference
- +0.90 pp
- 95% interval
- +0.05 pp … +1.75 pp
- P-value
- 0.0276
The difference of +0.90 pp is unlikely to be chance at 95% confidence.
Get the result by email
The next step is running the challenger beside the champion on the same applications, with each one's decision recorded. Start free
See the method
Champion loans per challenger k = (1 − Challenger traffic share) ÷ Challenger traffic share Pooled rate = (k × p1 + p2) ÷ (k + 1), p1 = Champion bad rate, p2 = p1 + Difference worth catching Challenger loans = (zα √(Pooled rate × (1 − Pooled rate) × (1 + 1/k)) + zβ √(p1 (1 − p1) ÷ k + p2 (1 − p2)))² ÷ Difference worth catching² Champion loans = k × Challenger loans Test fill time = Challenger loans ÷ (Challenger traffic share × Loans each month) Months until readable result = Test fill time + Outcome window in months zα: 90% 1.645 · 95% 1.960 · 99% 2.576 zβ: 80% 0.842 · 90% 1.282 P-value = 2 × (1 − Φ(|Difference| ÷ √(Pooled rate × (1 − Pooled rate) × (1/n1 + 1/n2))))
The arithmetic uses the normal approximation and assumes loans are assigned at random with independent outcomes. Very small arms or rates near zero need an exact test.
Plan a champion challenger test
Champion challenger testing compares the bad rate of a current strategy with a new strategy on a share of loans. Enter the current rate, the difference worth catching, the challenger's share, the monthly volume and the outcome window.
A smaller challenger share takes longer to produce a readable result because fewer loans enter that arm. Use the result to see the loans required, the time to fill the test and the time until outcomes are known.
Common questions
What is champion challenger testing in lending?
It compares a current champion strategy with a challenger strategy on a share of loans. The comparison uses the bad rate or another rate, the difference worth catching and the loans assigned to each arm.
How many loans does a champion challenger test need?
The required loans depend on the current rate, the difference worth catching, the challenger's share, confidence and power. This tool works out the loans needed for the challenger and the champion.
How long should a credit strategy test run?
The test needs time to fill with enough loans, then the outcome window must pass before the result can be read. This tool combines those periods using the monthly loan volume and the challenger's share.
What does statistical significance mean for a bad rate difference?
It describes whether the observed difference is unlikely to be chance at the selected confidence level. The finished-test check uses both arms' loans and bads to calculate the difference, its interval and the p-value.
Why does a small challenger share make a test slow?
A small share sends fewer loans to the challenger, so it takes longer to collect the loans needed for the comparison. The outcome window still comes after the test fills.