disparate impact
What It Means
Disparate impact measures whether an AI system's decisions affect different demographic groups at significantly different rates, even when the system wasn't intentionally designed to discriminate. It compares the approval or success rates between protected groups (like racial minorities) and non-protected groups to identify potential bias. The mathematical formula calculates this as a ratio - if one group gets approved 60% of the time while another gets approved 80% of the time, there's disparate impact.
Why Chief AI Officers Care
Disparate impact violations can trigger lawsuits, regulatory investigations, and significant financial penalties under civil rights laws, even if there was no intent to discriminate. Many industries like banking, hiring, and insurance are legally required to monitor for disparate impact in their AI systems. Beyond compliance, disparate impact can damage brand reputation, limit market reach, and indicate that AI models may be missing valuable business opportunities by systematically excluding qualified candidates or customers.
Real-World Example
A bank's AI loan approval system approves 75% of white applicants but only 55% of Black applicants with similar credit profiles. Even though race wasn't directly used as an input variable, the 20-percentage-point gap suggests disparate impact that could violate fair lending laws and result in regulatory action, regardless of whether the bank intended to discriminate.
Common Confusion
People often confuse disparate impact with disparate treatment - disparate impact occurs even when you treat everyone the same way, while disparate treatment involves intentionally treating groups differently. Many assume that removing demographic variables from AI models prevents disparate impact, but bias can still occur through proxy variables like zip codes or shopping patterns.
Industry-Specific Applications
See how this term applies to healthcare, finance, manufacturing, government, tech, and insurance.
Healthcare: In healthcare AI, disparate impact occurs when diagnostic algorithms, risk assessment tools, or treatment recommendation...
Finance: In finance, disparate impact is crucial for credit decisions, hiring, and insurance underwriting, where AI models must c...
Premium content locked
Includes:
- 6 industry-specific applications
- Relevant regulations by sector
- Real compliance scenarios
- Implementation guidance
Technical Definitions
NISTNational Institute of Standards and Technology
"For Predictor Y and Sensitive Impact S. Definition 6.2 Disparate Impact (DI) = P[Yˆ = 1 | S != 1]/P[Yˆ = 1 | S = 1] "Source: friedler_comparative_2019
Discuss This Term with Your AI Assistant
Ask how "disparate impact" applies to your specific use case and regulatory context.
Start Free Trial