Test fraud rules on past applications
Drop in past applications with known outcomes and write a few rules. See what each rule and the whole set would have caught and whom they would have stopped.
Applications file
Applications read: 2,000, of which 77 known fraud and 1,718 known good.
The file is read in this browser and never uploaded; the rules travel in the link, the data never does, and only counts go out if you ask for the result by email.
The fields the rules can use are application_id, amount, email_domain, device_age_days, account_age_days, ip_country, phone_country.
These are 2,000 generated demo applications with four example rules.
Net for all rules
The rules catch 74 of 77 known fraud and flag 232 applications, 141 of them good customers.
Fraud caught against good customers flagged
Rules
An application is flagged when any rule fires; within a rule, every condition has to hold; for is one of, separate the values with commas.
Rule results
| Rule | Flagged | Fraud caught | Good flagged | Unknown flagged | Precision | Catch rate | False-positive rate | Unique catches | Net |
|---|---|---|---|---|---|---|---|---|---|
| New device | 104 | 48 | 44 | 12 | 52.2% | 62.3% | 2.6% | 5 | €70,240 |
| Throwaway email | 30 | 27 | 0 | 3 | 100.0% | 35.1% | 0.0% | 0 | €40,500 |
| Large first loan | 37 | 30 | 6 | 1 | 83.3% | 39.0% | 0.3% | 2 | €44,760 |
| Foreign IP | 152 | 51 | 95 | 6 | 34.9% | 66.2% | 5.5% | 8 | €72,700 |
| All rules | 232 | 74 | 141 | 17 | 34.4% | 96.1% | 8.2% | €105,360 |
Get the result by email
The next step is running the same rules on every new application and testing each change on past decisions before it goes live. Start free
Show the formulas
Precision = Fraud caught ÷ (Fraud caught + Good flagged) Catch rate = Fraud caught ÷ All known fraud False-positive rate = Good flagged ÷ All known good customers Unique catches = Fraud caught, Flagged by no other rule Net = Fraud caught × Loss per fraud − Good flagged × Cost per good customer
The figures come from past outcomes in the file only. Fraud found later or fraud a rule would have put off is not in it.
Test fraud rules on past data
The tool compares each fraud rule with past applications that have known outcomes. It shows fraud caught, good customers flagged, precision, catch rate, false-positive rate and each rule's unique catches.
These figures show what a rule would have done on the file. A lender can compare the fraud caught with the good customers flagged, then test each change on past decisions before it goes live.
Fraud rule questions
How do I test fraud rules on past data?
Load a CSV file or paste past applications with a column marked fraud or good. Write rules on the fields in the file, then see what each rule and the whole set would have flagged.
What do precision and catch rate mean for a fraud rule?
Precision is the share of flagged applications with a known outcome that is fraud. Catch rate is the share of all fraud that the rule catches.
How do I measure the false-positive rate of fraud rules?
The false-positive rate is good customers flagged divided by all good customers. This tool shows it for each rule and for all rules together.
Why do a rule's unique catches matter?
Unique catches are the fraud only that rule flags. They show the fraud contribution a rule makes beyond the other rules in the set.
How do I weigh fraud losses against good customers flagged?
The net is fraud caught multiplied by loss per fraud, minus good customers flagged multiplied by cost per good customer. The tool shows this amount for each rule and for all rules together.