User-centred Counterfactual Explanations for Explainable AI

The growing ubiquity of Artificial Intelligence (AI) systems across a wide range of high-stakes applications has heightened concerns about the interpretability and fairness of their decisions. These concerns have inspired a recent surge of research focused on eXplainable AI (XAI) techniques to elucidate the decisions of opaque automated systems. Counterfactual explanations have been proposed as a promising XAI technique with the potential to provide intuitive and actionable explanations that can help users understand how a model’s decision would change with alterations to input features. However, these proposals remain for the most part untested in controlled psychological experiments, and we know little about which aspects of counterfactual explanations help users understand AI system decisions. This thesis addresses this gap by drawing on insights from human explanation and counterfactual reasoning to examine how psychological findings can inform counterfactual explanation in XAI. Experiments 1 (N = 213) and 2 (N = 313) find that people generally prefer simple explanations, however this preference is contingent upon factors such as probability, domain representation, and whether the explanations address effects or causes. Experiments 3 (N = 127), 4 (N = 136) and 5 (N = 211) establish that counterfactual explanations enhance users’ understanding of AI decisions and slightly improve upon more traditional causal explanations, as well as being subjectively preferred by users. The experiments also show that people understand explanations referring to categorical features more readily than those referring to continuous features, a distinction that has been overlooked by existing computational counterfactual generation methods. Computational Experiments 1 and 2 investigate the extent to which state-of-the-art counterfactual methods meet the psychological requirements of users and evaluate a categorical post hoc processing technique. Computational Experiment 3 assesses the natural occurrence of categorical counterfactuals in machine learning datasets. We propose a case-based counterfactual explanation approach that facilitates user understanding of AI decisions by ensuring that generated counterfactual explanations refer to categorical features. Computational Experiments 4 and 5 evaluate two variants of this method relative to baseline and state-of-the-art methods, demonstrating that they generate counterfactuals that align with psychological constraints and perform well with respect to benchmark evaluation metrics. The results of this thesis highlight the pivotal importance of how explanations are structured and represented. It presents a user-centred approach to XAI that is driven by the psychological requirements of users rather than purely algorithmic considerations.