Independent monitor finds approval disparities for Black applicants in Upstart's AI lending model

What happened?

Upstart underwrites and prices consumer loans with AI and machine-learning models that use non-traditional data, including education. After a 2020 Student Borrower Protection Center report raised concerns about "educational redlining", Upstart, the SBPC and the NAACP Legal Defense Fund agreed on 1 December 2020 to appoint Relman Colfax as a neutral, independent monitor to test the model for discrimination and publish its findings.[1][2]

Across the monitorship, Relman Colfax found no quantitative evidence that variables in the model worked as close proxies for race, national origin, sex or age. It did identify "statistically and practically significant" approval and denial disparities for Black applicants. It also found some interest-rate (APR) disparities for Black, Hispanic and female applicants, but none exceeded its practical-significance thresholds. The findings concerned a prior version of the model, and the monitor stressed that disparities alone "do not, standing alone, demonstrate a fair lending violation".[1]

In its third report, in September 2022, the monitor identified what it believed would likely have been a viable less discriminatory alternative model for Black applicants. Upstart said at the time that it was working to improve its models, that the audit found no pricing bias, and that approval disparities were "an industry-wide challenge driven by decades of racial inequities, including credit scores".[2][5]

The final report, released on 27 March 2024, says Upstart "has adopted nearly all of the Monitor’s recommendations" and strengthened its fair-lending testing. The parties stayed at an impasse over how to judge whether a less discriminatory alternative model performs well enough to adopt. Upstart argued that the monitor's method "would unacceptably compromise the accuracy of its models" and offered its own alternative; the joint release says the monitor did not validate that claim.[1][3][4]

When did it happen?

1. Dec 1, 2020

   Upstart, LDF and SBPC agree to appoint Relman Colfax as independent fair-lending monitor of Upstart's model.[1]

2. Nov 10, 2021

   The monitor's second report sets out its testing methodology.[2]

3. Sep 16, 2022

   The third report identifies approval disparities for Black applicants and a likely viable less discriminatory alternative model.[1][5]

4. Sep 26, 2022

   Upstart says it is improving its AI models and that the audit found no pricing bias.[5]

5. Mar 27, 2024

   The fourth and final report ends the monitorship at an impasse over the less-discriminatory-alternative standard.[1][3]

What was the impact?

• Customers: Black applicants had statistically and practically significant lower approval outcomes under the tested version of the model. The reports do not give a number of affected applicants, and no restitution was ordered or offered.[1][3]
• Other: The monitor found no pricing disparities above its practical-significance thresholds and no proxy variables.[1][3]

Severity 2 of 5: Limited harm: a voluntary monitor found statistically and practically significant approval disparities for Black applicants in a prior version of the model, without finding a fair-lending violation; no regulator acted and no penalty was imposed.

How did Upstart respond?

Upstart co-issued the release announcing the final report. Nat Hoopes, its head of government and regulatory affairs, said: "We share the advocates’ concern of the risk of bias in AI lending and are pleased that the Monitor confirmed our model does not lead to pricing disparities or include variables that implicate race, national origin, sex, or age. We believe that the measures incorporated into our fair lending program to remove bias achieve the same fairness objectives as those recommended by the Monitor."[3][4]

What was the outcome, and did regulators act?

The monitorship ended on 27 March 2024 without agreement on the less-discriminatory-alternative standard. The report closes with joint recommendations: lenders should routinely test credit models for discrimination, and regulators should issue guidance on finding and adopting less discriminatory alternative models. No regulator finding, penalty or lawsuit followed from the monitor's findings in the sources reviewed.[1][4]

1. Fair Lending Monitorship of Upstart Network’s Lending Model — Fourth and Final Report of the Independent Monitor news
   Relman Colfax · Mar 27, 2024
   “(1) finding no quantitative evidence that variables in Upstart’s Model functioned as close proxies for race, national origin, sex, or age; but (2) identifying approval disparities for Black applicants.”
2. Fair Lending Monitorship of Upstart Network’s Lending Model news
   Relman Colfax (case page) · Mar 27, 2024
   “Despite extended discussions and analyses, the Parties remain at an impasse over the appropriate and legally required methodology for assessing whether the performance of a potential less discriminatory alternative model would be comparable to the performance of an existing model.”
3. LDF, SBPC, and Upstart Announce Final Monitorship Report on AI and Fair Lending official
   Upstart (investor relations) · Mar 27, 2024
   “While the monitor did not find evidence that variables in Upstart’s AI model operated as close proxies for protected classes and did not find pricing disparities, it did identify approval disparities for Black applicants.”
4. LDF, SBPC, and Upstart Announce Final Monitorship Report on AI and Fair Lending news
   NAACP Legal Defense Fund · Mar 27, 2024
   “However, Upstart disagreed with “a key recommendation”—a less discriminatory alternative (LDA) model proposed by the Monitor—claiming that it would unacceptably compromise the accuracy of its models.”
5. Upstart says it's improving AI models after report finds race approval disparities news
   American Banker · Sep 26, 2022
   “But Upstart also said that loan approval disparities for Black applicants is "an industry-wide challenge driven by decades of racial inequities, including credit scores."”