Swap set analysis
Swap set analysis compares a new credit scorecard with the current one by which applicants each would approve. It separates applicants approved only by the new scorecard, called swap-ins, from those approved only by the current one, called swap-outs, and records applicants both scorecards approve or both decline.
Example
For illustration, suppose each scorecard approves 500 of 1,000 applicants. If 100 are swap-ins and 100 are swap-outs, then 400 are approved by both; if estimated bad rates are 5% for swap-ins and 10% for swap-outs, the new scorecard has the lower bad rate among the applicants whose decisions change at the same approval rate.
Reading the decision and score band changes
At a chosen cut-off, place the current scorecard’s approve or decline decision on one axis and the new scorecard’s decision on the other. The four cells show applicants approved by both, declined by both, approved only by the new scorecard, or approved only by the current one.
A migration matrix adds score bands to the comparison. Each cell counts applications that move from a band under the current scorecard to a band under the new one. For illustration, a cell might show 40 applications moving from the current scorecard’s middle band to the new scorecard’s higher band.
Comparing bad rates and estimating unknown outcomes
At the same approval rate, compare the bad rate for swap-ins with the bad rate for swap-outs. A lower bad rate among swap-ins is evidence in favour of the new scorecard at that approval rate.
The past outcome for swap-ins is unknown because the current scorecard declined them. Their bad rate therefore needs an estimate, using reject inference or a test on a small share of applicants. The comparison depends on that estimate, so distinguish it from outcomes that are already known.
Questions we get asked
Why compare scorecards at the same approval rate?
The comparison asks whether the new scorecard selects a better-performing group at the same approval volume. At that rate, a lower bad rate for swap-ins than for swap-outs favours the new scorecard.
Are applicants approved by both scorecards part of the swap?
No. They fall in the shared-approval cell of the two-by-two comparison, rather than among applicants whose approval decision changes.
Can the past outcome of a swap-in be known?
No, the current scorecard declined that applicant, so the past outcome is unknown. Reject inference or a test on a small share can be used to estimate it.
Decisimo’s swap set calculator shows application movement between score bands as a share of the portfolio, plus swap-ins and swap-outs at the cut-off.
