Run champion and challenger tests for credit decisions
A test can mean two logic versions, traffic-splitting code and a spreadsheet. Decisimo runs the test with your risk team.
What a champion and challenger test takes today
Two strategy versions
Developers rebuild the table, scorecard or rule set
Traffic-splitting code
A project routes each loan to one arm
After-test spreadsheet
The analyst compares each arm by hand
Test size unknown
Loan volume and outcome timing are unclear
Rare strategy tests
Few changes reach a live test
Judgement decides
The risk team relies on judgement to release changes
One champion and challenger test, from sizing to the new champion
Size the test
Edit the challenger
Replay both revisions
Review the changes
Route live loans
Compare the outcomes
Deploy the challenger
Replay the challenger before a customer sees it
- Challenger copyThe risk team edits it, and each save creates a revision.
- Two revisionsIt replays baseline and comparison revisions of the decision flow
- Past decisionsUse the decision archive, an uploaded file or the lender's storage.
See which decisions change and how those loans did
- Change countThe result counts changes, such as 30 of 1,001 decisions
- Outcome pairsEach baseline and comparison outcome cell shows its decision count.
- Realized bad rateJoined outcomes add a rate per cell. Mine selected rows for rules.
Split live applications on a value the lender sends
- Lender's splitThe lender sends the value, and no random split is built in
- Bounded challenger rangeThe champion takes the rest, including loans with no split value.
- Decision archiveIt keeps each split value with the decision for comparison in lender data.
Size the test before it starts
- Test inputsChampion bad rate, difference worth catching and challenger share.
- Volume and windowAdd loans per month and the outcome window.
- Required sampleThe tool returns loans per arm and months until results can be read.
The challenger becomes the champion by a release
- Deploy the releaseThe risk team sends it to the endpoint, replacing the live release.
- Compare releasesThe risk team can compare two releases before it deploys either.
- Approval sequenceThe endpoint can require signatures before the risk team deploys.
Questions we get asked
What is champion challenger testing?
Champion challenger testing compares a current lending strategy with a changed challenger on a share of loans. In Decisimo, the risk team replays both revisions, routes live loans using the lender's split value, then deploys the challenger release.
How does champion challenger testing split loans?
The lender's system sends a split value. A fork routes a bounded range to the challenger, and loans with no split value stay with the champion.
How long should a champion challenger test run?
Enter the champion's bad rate, the difference worth catching, the challenger's share, monthly loans and outcome window to get the loans each arm needs and the months to a result. Size the test
Can I use champion challenger testing for collections?
The same method applies to collections treatments: replay the changed strategy, split live accounts using your system's value, then compare outcomes in your data. Champion challenger in collections