Abstract: Selecting fraud-rule portfolios to satisfy a precision or recall floor can overfit the data used for selection. We study finite-family certification for bounded OR portfolios, comparing point-estimate selection, a fixed safety margin, and one-sided Clopper-Pearson lower bounds with Bonferroni adjustment. In a factorial follow-up with 6,400 datasets from two author-designed synthetic generators, full-validation certification reduced the held-out floor-violation rate by 29.1 percentage points on average versus full-validation point selection, while reducing deployment by 31.7 points. Compared with a fixed 0.05 margin, certification reduced violations by 22.2 points and deployment by 29.4 points; results varied across conditions. Using the full rather than half validation sample increased deployment by 17.1 points on average, without a uniform reduction in shifted-test violations. In a separate 400-dataset no-shift sweep, certification deployment rose from 61.3% to 90.2% as validation size increased from 1,500 to 15,000; no violations occurred among 2,887 deployed portfolios tested on 100,000-row holdouts. A temporal case study on the public synthetic Bank Account Fraud Base dataset used recall at a 5% false-positive-rate cap; the selected candidate pool captured less than 1% of test fraud, showing limited coverage in this setup. Together, these experiments characterize how finite-family certification changes the reliability-deployment tradeoff for bounded OR portfolios under specified synthetic conditions. The evidence covers the evaluated generators and one public synthetic BAF case study; it does not estimate performance on real transaction populations or robustness to arbitrary distribution shift. The authors revised and rewrote this abstract after receiving a draft from OpenAI Codex.
Read the original article:
