QED Insight #0004: How is your shop handling the race-proxy problem in fair-lending testing?

This was so interesting to me when I first crossed it. Discrimination based on race is so important - and yet, we can use statistics to proxy it? That feels weird to me and is something that still feels slightly absurd when I explain it to people outside the field: for most of what we lend on outside mortgages - auto, cards, personal - we're not allowed to collect the applicant's race or ethnicity (ECOA/Reg B). But we're still on the hook to show we're not discriminating. So we have to test for a disparity by a variable we're legally barred from observing.

The workaround everyone ends up at is BISG - Bayesian Improved Surname Geocoding. Estimate P(race) from the surname (Census surname list), estimate it again from the census tract, Bayes them together, and now every applicant has a probability vector across groups instead of a label. Aggregate across the book and you can back out approval/pricing/denial gaps.

What bugs me is how much rides on a proxy that's noisy at the individual level and has uneven accuracy across groups. The measured disparity is partly a property of the estimator, not just the lending.

I actually set this up as a validation once. Took ~200k 2024 HMDA records across six states that carry self-reported race, assigned each one a surname from the Census surname file to match its known race, then ran BISG on just the surname + real census tract and re-measured the denial-rate gap on the proxy. It nailed the direction every time but it shrank the gap: Black/White denial ratio was 1.58 on the real labels, 1.39 once it went through BISG. The multi-minority bucket was worse - a real 1.64 came out at 1.08, basically erased. So the proxy wasn't just noisy, it pulled everything toward parity. I've treated a BISG disparity as a floor ever since, not the number itself.

For those of you doing this day to day: are you on plain BISG, BIFSG (adding first name), or a vendor flavor? And how do you defend the methodology when the number it produces is the thing an examiner or plaintiff leans on?