Cure rates fell in bucket 2. Which release booked those loans?

A collections manager brings you a number. Cure rates in the second delinquency bucket have dropped over the last two months. The book grew in that period, so somebody in the room says you have been approving worse business.

Maybe. There are at least eight reasons that number moves, and lending is the discipline of checking all of them rather than the one that fits your prior:

• the decision logic changed, so these loans were approved on different criteria
• collections is short-staffed, or lost tenured agents
• the call script changed, so the same borrowers get handled differently
• the dialler changed, and right-party contact rate fell with it
• a new marketing channel is bringing a different borrower
• pricing moved, changing who accepts an offer
• the macro picture moved under everyone
• the definition changed, and “cure” means something new in this month’s report

Most of those need a study before they need an answer. Contact rates need a controlled comparison. Agent tenure effects need a cohort study. Macro is a debate.

One of them does not. Whether the decision logic changed is a question with a yes or a no, and the record that settles it is already sitting in your own log. That is why it goes first: not because policy is the likeliest cause, but because it is the only item you can take off the table without designing anything.

This article is about how to eliminate it properly, which means answering a more precise question than “did we change anything”. It means asking: for the loans now sitting in bucket 2, what decision logic actually approved them, and does it differ from the logic running today?

Why the portfolio view hides the answer

The Basel guidance on expected credit losses is blunt about the trap. Exposures “must not be grouped in such a way that an increase in the credit risk of some individual exposures could be obscured by changes in the credit risk of the portfolio as a whole” (BCBS d350, A33). A portfolio-level cure rate is exactly that kind of grouping. It moves for many reasons at once, and a cohort booked under one policy is averaged together with a cohort booked under another.

The same document lists vintage among the characteristics you group by, alongside due status, loan-to-value, product type and geography, and requires the grouping methodology to be documented and approved. Vintage is the standard answer because origination date is a proxy for the policy in force at the time.

It is only a proxy. Two applications approved on the same morning can be decided by different logic if a release went out between them, and a vintage bucket spanning a month will contain several. The precise version of the question is not “when was this booked” but “what approved this”.

What the regulator already asks you to keep

The EBA guidelines on loan origination and monitoring are specific about the record:

The data infrastructure should be detailed and sufficiently granular to capture specific loan-by-loan information, in particular actual credit-granting criteria applied at the point of origination.

EBA/GL/2020/06, paragraph 61

Read the phrase carefully: actual criteria applied at the point of origination. Not the current policy document. Not a description of what the policy was meant to be. The criteria that ran.

Most lenders keep a version history of the policy document and a change log of model deployments, then reconstruct by hand which combination was live on a given date. That reconstruction is where the confidence goes.

What a release is

In Decisimo, publishing a decision flow writes a release: a single compiled snapshot of everything the flow needs to reach a decision. It is not a pointer to the current rule sets. The build walks the flow and inlines the full definition of every artifact it touches:

• rule sets, with their rules, conditions, actions, mappings and aggregates
• decision tables, cell by cell, including operators and result vectors
• scorecards, with predictors, weights, and the bins that segment each predictor
• decision trees, node by node
• functions, models, and external data sources including method, headers and OAuth configuration
• subflows, recursively

Evaluation order is normalised into the snapshot as well. Fork branches and scorecard bins are both sorted at compile time, because first-match-wins ordering is decision logic in its own right: the same rules in a different order are different rules.

At runtime the engine executes that stored content and nothing else. It does not read the design tables to make a decision. This is the property that makes the history worth keeping: editing a scorecard today cannot change what a decision did last March, because last March’s decision ran against a snapshot that still exists.

The chain from a decision to its logic

Every execution writes a row to your own log database, and that row carries the identifier of the deployment that produced it. The column is engine_request_id, and it holds the wf_release_deployment_id. Alongside it sit the full request and response as input_json and output_json.

From there the path is a join:

log row.engine_request_id
   → wf_release_deployment.wf_release_deployment_id
      → .content              the exact logic that executed
      → .workflow_release_id
         → workflow_release   .name, .notes, .user_id, .dtime_inserted

Which gives you, for any decision you can identify: the logic that produced it, the release it came from, who published that release, and when.

So the elimination step becomes mechanical. Take the loans in bucket 2, join their originating decisions on engine_request_id, and group. If they cluster on a release that came after the last known-good cohort, compare the two releases and read what changed.

If they do not cluster, that is a result too, and a more useful one than it looks. It means the loans arriving in bucket 2 were approved by the same logic that approved the cohort curing normally last quarter, so the change is downstream of the decision. That points the investigation at collections capacity, the treatment path, contact rates, or the acquisition mix, and it does so from records you already hold, before anyone designs a study. A negative answer that is definite is worth more than a positive answer that is arguable.

Two practical notes. The release identifier is not returned in the decision response body, so the linkage lives in the log and is joined on request_id. And releases are identified by their id and a generated name stamped with the publication time, in the form credit_scoring_2608201430, rather than by a semantic version number.

Comparing the two releases

Once you have two release ids, comparing them is the point of the exercise. The comparison renders the flow step by step, marking each as added, deleted, modified or unchanged, and expands the modified ones.

For a scorecard, that expansion is at the level you need for this diagnosis. Predictors are matched by identity rather than by position, so a renamed predictor is not reported as one deletion and one addition. Each predictor shows its weight before and after. Each bin shows its operator and boundaries before and after.

That distinguishes the two changes that look identical in a portfolio report and are not alike at all:

• A weight moved. The same applicants are being scored, and the contribution of one predictor to the total changed.
• The segmentation moved. A bin boundary shifted, so a band of applicants who previously scored one way now score another. The population is being cut differently.

A weight change tends to move the whole score distribution slightly. A segmentation change tends to move one band a lot, which is what produces a cohort that behaves unlike its neighbours, which is what shows up months later as a cure rate that will not sit still.

Where the experiment lives

Champion/challenger in Decisimo is a fork in the decision flow, splitting on a random value, with the challenger logic on one branch and the champion on the other.

That placement matters for this whole argument. Because the split is a step in the flow rather than configuration sitting beside it, the experiment is inside the release snapshot. Comparing releases shows the challenger branch appearing, the boundary it splits on, and the scorecard behind it. A platform that held champion/challenger in a separate configuration layer would leave the flow diff looking unchanged while a tenth of your applicants were being scored by something else.

One limitation to plan for, because the default will not serve you here. Whether a particular application took the champion or the challenger branch is recoverable only if you made it recoverable. Either the caller supplies the split value in the request payload, in which case it lands in input_json in your log and you can group on it later, or the caller sets the record_steps flag, which returns the list of steps that actually executed. Neither is on by default. If you are running an experiment whose results you intend to analyse by cohort months from now, put the split value in the payload.

What this does not give you

The record is a reconstruction aid, not a tamper-evident ledger. Releases and deployments can be deleted, and the foreign keys are set to null on delete, so removing a deployment leaves the historical references in your log pointing at nothing. If your retention obligation runs for years, do not delete deployments that historical decisions reference.

The release build also pins the exact entity revision it compiled for rule sets, decision tables, functions, scorecards and decision trees, but not for models, external data sources, OCR services or subflows. For those, you have the inlined definition in the snapshot, which is what executed, but not a pointer back to the design-time revision it came from.

And release checks warn rather than block. A release with lint warnings publishes; approval processes attach to a release after it exists rather than gating its creation.

What to do with this

Four things, in order of how much they pay back.

Join your decision log to your loan book on the request identifier, and keep the engine_request_id in your warehouse. This is the step people skip, and without it the rest is unavailable. It costs one column.

Slice deterioration by release before you slice it by month. Vintage is a proxy for policy; the release identifier is the policy. When they disagree, the release is right.

Run the policy check first and write down the answer, either way. It is the one dimension on the list that resolves to a yes or a no, and both answers are worth having: one tells you what to roll back, the other tells the collections and marketing conversations to proceed without a suspicion hanging over them.

Put your champion/challenger split value in the request payload. An experiment you cannot decompose afterwards produced a number, not a finding.

None of this requires a data science function. It requires that the thing which made the decision still exists, in the form it had when it made it, and that the decision points at it.

Frequently asked questions

How can I tell which decision logic approved loans in a delinquency bucket?

Join each loan’s originating decision to the lender’s decision log using the request identifier, then group the loans by release. Each execution record identifies the deployment that produced the decision, and the release preserves the compiled logic that ran. Compare the releases for loans in the deteriorating cohort with the release for a cohort that is curing normally.

Why isn’t origination month enough to identify which policy approved a loan?

Origination month is a proxy for policy, not proof of which decision logic approved an application. Two applications approved on the same morning can be decided by different logic if a release goes live between them, so a monthly vintage can contain decisions made under different logic. Use the release to identify the policy that ran.

What does a decision-flow release preserve?

A decision-flow release preserves a compiled snapshot of the logic at publication, including rule sets, decision tables, scorecards, decision trees, functions, models, external data sources and subflows. The engine runs that stored snapshot, not the current design tables. Editing the decision flow later does not change the logic that produced an earlier decision.

How can a lender track champion-challenger results by cohort?

Put the random split value in the request payload so it is recorded in the lender’s decision log and results can be grouped by that value. The branch an application took is not otherwise recoverable by default. A setting can return which parts of the decision flow actually ran, but neither method is enabled by default.