Run champion challenger tests for credit decisions

A champion challenger test can mean two versions in code, then traffic splitting and spreadsheet work, so changes go live on judgement. Decisimo sizes the test and runs the challenger through a decision flow before release.

Start free Size the test

Size the test before live applications

The free tool takes the champion's bad rate, the difference worth catching, the challenger's share, monthly loan volume and the outcome window. It gives the loans each arm needs and the months until the result can be read. Size your test.

Replay the challenger on past decisions

The challenger is a changed copy of the champion's table, scorecard or rule set. The impact analyzer runs the baseline and comparison revisions over past decisions or an uploaded file. It counts the decisions that would change. With outcomes joined, each group shows its realized bad rate.

Impact analyzer with an uploaded dataset, baseline Rev 2, comparison Rev 3, and Cut-off impact selected
The impact analyzer shows a baseline revision and a comparison revision run on uploaded data. The risk team can review the proposed change before it reaches a customer.
Impact analyzer shows 30 of 1,001 decisions changing from approve to decline, with a 0.00% realized rate
The results show 30 of 1,001 decisions would change from approve to decline, with the changed group at a realized rate of 0.00%. This lets the risk team inspect the changed decisions alongside what happened to those loans.

Split live applications on a lender-sent value

A fork routes each application by the split value the lender’s system sends. A bounded range with a lower and upper end takes the challenger branch; the champion takes everything else, including an application with no split value. The assistant can add the fork from a chat, with each proposed change approved before it is made. The decision archive keeps the split value with each decision, so the lender compares both arms using its own outcomes.

A split-value fork sends A-1042 (0.07) to the challenger; A-1041, A-1043, A-1044 and A-1045 stay champion
The illustration shows split values routing accounts to the challenger branch or the champion treatment. The challenger takes accounts with a split value from 0 up to 0.1; the rest, including A-1045, which arrived with no split value, stay on the table in use today.

Make the challenger the champion

The challenger becomes the champion when its release is deployed to the endpoint. Two releases can be compared first, and an endpoint approval sequence must be signed before deployment when one is configured. See the credit decision as a whole.

Questions we get asked

What is champion challenger testing?

Champion challenger testing is a way to compare the strategy in use with a changed version on a share of applications. You review what would change, then deploy the challenger release when you decide to make it the champion.

How is traffic split between champion and challenger?

The lender’s system sends a split value with each application. The challenger takes a bounded range with a lower and upper end; the champion takes everything else, so an application with no split value stays with the champion.

Can a challenger be tested before going live?

Yes. The impact analyzer runs the baseline and challenger revisions over past decisions or an uploaded file. It counts changed decisions and shows the realized bad rate when outcomes are joined.

How long should a champion challenger test run?

The required time depends on the champion's bad rate, the difference worth catching, the challenger's share, loan volume and the outcome window. Size the test to see the loans and months required.

Does champion challenger testing work for collections?

Yes. Champion challenger testing can compare collections treatments for accounts. See collections decisioning.

Test your next credit strategy

Start by sizing the test, then replay the challenger on past decisions before routing live applications.

Start free