On 30 June, Umang Bhatt, a PhD candidate in the Machine Learning Group at the University of Cambridge and one of the REPHRAIN members behind the SOXAI project, conducted an online masterclass exploring the challenges and frontiers in algorithmic transparency.
An abstract of the event can be found below – a copy of Umang’s presentation can be found here.
Algorithmic transparency entails exposing system properties to various stakeholders for purposes that include understanding, improving, and contesting predictions. Until now, most research into algorithmic transparency has predominantly focused on explainability, which provides reasons for a machine learning model’s behavior to stakeholders. We first discuss research exploring how organizations view and use explainability. We report a gap between explainability in practice and the goal of external transparency, as we find that explanations are primarily serving internal stakeholders. Understanding a model’s specific behavior alone is not enough for stakeholders to gauge whether the model is wrong or lacks sufficient knowledge to solve the task at hand. We then argue for considering a complementary form of transparency by estimating and communicating the uncertainty associated with model predictions. We draw from literature spanning machine learning, visualization/HCI, design, decision-making, and fairness to encourage practitioners to measure, communicate, and use uncertainty as a form of transparency. We conclude with a discussion of algorithmic transparency in the context of human-machine teams.
Umang Bhatt is a PhD candidate in the Machine Learning Group at the University of Cambridge, where he is supervised by Adrian Weller. He is also an Enrichment Student at the Alan Turing Institute, an Advisor at the Responsible AI Institute, and recent winner of a J.P. Morgan AI PhD Fellowship. His research interests lie in machine learning, explainable artificial intelligence, and human-machine collaboration. His PhD research is funded by the Leverhulme Center for the Future of Intelligence with generous donations from DeepMind and Leverhulme Trust. Previously, he was a Fellow at the Mozilla Foundation and a Research Fellow at the Partnership on AI.
Please contact rephrain-centre@bristol.ac.uk with any questions – thank you to Umang for leading this session and to all attendees for their contributions!