Machine Learning at the Flatiron Institute Seminar: Mahdi Soltanolkotabi

Name: Machine Learning at the Flatiron Institute Seminar: Mahdi Soltanolkotabi
Start: 2025-02-18T15:00:00-05:00
End: 2025-02-18T16:00:00-05:00
Location: 162 Fifth Avenue, 9th Floor Classroom

Tuesday February 18, 2025

Title: Towards More Reliable Generative AI: Probing Failure Modes, Harnessing Test-Time Inference, and Interpreting Diffusion Models

Abstract: Generative AI systems—especially Multimodal Large Language Models (MLLMs)—offer promising avenues across a wide range of tasks, from medical imaging to enhanced reasoning. In this talk, we explore strategies for making generative AI more dependable by examining its vulnerabilities and leveraging adaptive learning. First, we introduce MediConfusion, a benchmark that exposes systemic failure modes of state-of-the-art medical multimodal models in Visual Question Answering—a setting where reliability is paramount. Our findings reveal that even top models fail to distinguish visually dissimilar medical images, underscoring the challenges of deploying AI in clinical contexts. Next, we turn to test-time training (TTT), a gradient-based technique that updates model parameters using information from individual test instances. We provide a theoretical framework that explains how TTT can mitigate distribution shifts and significantly reduce the sample size needed for in-context learning. Finally, time permitting, we briefly address interpretability challenges in diffusion models, shedding light on how and why these powerful generative approaches produce their outputs. By bridging these threads, we show how identifying and addressing vulnerabilities—through challenges like MediConfusion and adaptive strategies like TTT—can enhance the reliability and impact of generative AI in healthcare and beyond.

About the Speaker

Mahdi Soltanolkotabi is the director of the center on AI Foundations for the Sciences (AIF4S) at the University of Southern California. He is also a professor in the Departments of Electrical and Computer Engineering, Computer Science, and Industrial and Systems engineering. Prior to joining USC, he completed his PhD in electrical engineering at Stanford in 2014. He was a postdoctoral researcher in the EECS department at UC Berkeley during the 2014-2015 academic year. Mahdi is the recipient of the Information Theory Society Best Paper Award, Packard Fellowship in Science and Engineering, an NIH Director’s new innovator award, a Sloan Research Fellowship, an NSF Career award, an Airforce Office of Research Young Investigator award (AFOSR-YIP), and faculty awards from Google and Amazon. His research focuses on developing the mathematical foundations of modern data science via characterizing the behavior and pitfalls of contemporary nonconvex learning and optimization algorithms with applications in AI, deep learning, large scale distributed training, federated learning, computational imaging, and AI for scientific and medical applications. Most recently his applied research focuses on developing and deploying reliable and trustworthy AI in healthcare.