AI for People:

Title:
Mind the gap: From predictions to ML-informed decisions

Abstract:
Machine learning (ML) is increasingly being used to support decision-making in critical settings, where predictions have potentially grave implications over human lives. In this talk, I will discuss the gap that exists between ML predictions and ML-informed decisions. The first part of the talk will highlight the role of humans-in-the-loop, and for the importance of evaluating decisions instead of predictions, through a study of the adoption of a risk assessment tool in child maltreatment hotline screenings. The second part of the talk will focus on the gap between the construct of interest and the proxy that the algorithm optimizes for. Using proposed methodology that leverages influence functions to extract knowledge from experts’ historical decisions, we show that in the context of child maltreatment hotline screenings (1) there are high-risk cases whose risk is considered by the experts but not wholly captured in the target labels used to train a deployed model, and (2) we can bridge this gap if we purposefully design with this goal in mind.

Bio:
Maria De-Arteaga is an Assistant Professor at the Information, Risk and Operation Management (IROM) Department at the University of Texas at Austin, where she is also a core faculty member in the Machine Learning Laboratory. She received a joint PhD in Machine Learning and Public Policy from Carnegie Mellon University. Her research on algorithmic fairness and human-centered machine learning studies the risks and opportunities of using machine learning to support experts’ decisions. De-Arteaga is a co-founder of the Machine Learning for the Developing World (ML4D) Workshop. Her work has been featured by UN Women and Global Pulse, and has received best paper awards at NAACL’19 and Data for Policy’16, and research awards from Google and Microsoft Research.