The U.S. criminal legal system increasingly relies on software output to convict and incarcerate people. In a large number of cases each year, the government makes these consequential decisions based on evidence from statistical software — such as probabilistic genotyping, environmental audio detection and toolmark analysis tools — that the defense counsel cannot fully cross-examine or scrutinize. This imbalance undermines the commitments of the adversarial criminal legal system, which relies on the defense’s ability to probe and test the prosecution’s case to safeguard individual rights.
Responding to this need to adversarially scrutinize output from such software, in this talk, Rediet Abebe proposes a novel framework for examining the validity of evidentiary statistical software called robust adversarial testing. She will define and operationalize this notion of robust adversarial testing for defense use by drawing on a large body of recent work in robust machine learning and algorithmic fairness. Abebe will demonstrate how this framework both standardizes the process for scrutinizing such tools and empowers defense lawyers to examine their validity for instances most relevant to the case at hand. She will further discuss existing structural and institutional challenges within the U.S. criminal legal system, which may create barriers to implementing this framework, followed by a discussion on policy changes that could help address these concerns. She will close with an outline of research directions in this burgeoning area of adversarial machine learning and adversarial scrutiny in the law.
This talk is based on joint and ongoing work with Moritz Hardt, Angela Jin, John Miller, Ludwig Schmidt, and Rebecca Wexler, as well as conversations with numerous public defenders and forensic scientists, including Nathan Adams, Clinton Hughes, Daniel Krane, and Richard Torres.
Rediet Abebe Assistant Professor of Computer Science, University of California, Berkeley
