Skip to content
← All work

02 · Causal inference · ML

Causal Credit Uplift

Who changes because you treated them? T-learner vs CausalForestDML on a real email RCT, then on synthetic credit data where the true effect is known.

Code on GitHubLive demo
corr(pred, true CATE), causal forest
0.927
PEHE, causal forest
0.0015
Persuadables in its top decile
100%
Qini, T-learner on Hillstrom
7.74

Qini curves, T-learner vs CausalForestDML

Real curves from dashboard/public/data/phase1.json and phase2.json. Toggle the phase.

Qini coefficient: T-learner 91.34 · CausalForestDML 87.66

Problem

A credit-limit increase is a treatment. A default model ranks people by outcome, which puts Sure Things on top and misses both the Persuadables and the Sleeping Dogs, the customers the treatment actually harms.

Approach

  1. 01Phase 1 on real randomized data: the Hillstrom MineThatData email RCT (Mens E-Mail vs No E-Mail, outcome is a site visit). Lock down the method before inventing credit data.
  2. 02Phase 2 on a synthetic credit-limit RCT with four planted segments and a known true CATE for every row. Outcome is good standing at 3 months.
  3. 03Score both learners on Qini and uplift at 10/30/50%, then on PEHE and correlation with the true CATE, then on who actually lands in the top decile.

Results

Hillstrom: T-learner wins overall Qini (7.74 vs 6.26). CausalForestDML wins if you can only treat the top 10% (0.153 vs 0.116). The two models share only 58.1% of their top-decile customers (Jaccard 0.409), so CATE rankings are less stable than a risk score ranking.

Synthetic credit: CausalForestDML recovers the true effect better (PEHE 0.0015 vs 0.0037, correlation 0.927 vs 0.847). Its top decile is 100% true Persuadables and 0.0% Sleeping Dogs. The T-learner gets 95.5% and 0.1%.

No single winner. The T-learner ranks better, the forest is safer. Which one ships depends on whether you care more about the list or about not harming people on it.

Next

A real randomized limit-increase experiment, overlap diagnostics, and a policy-value curve that turns CATE into a limit rule a committee could sign.