Human-in-the-Loop Learning Using Model Explanations and its Impact on Neuroradiologists in Detecting Alzheimer's Disease

Human-in-the-loop Learning (HITL) is an approach to machine learning that involves humans in the model training process. This allows more accurate, transparent, and trustworthy models to be trained by augmenting machine learning and deep learning approaches with human input. Beyond the typical user involvement, such as data labelling, model explanations can be integrated into HITL methods to gather more sophisticated user feedback. Users can view explanations of a trained model and provide feedback on whether the explanations show that a trained model uses relevant regions or features in a given instance, or if a model uses spurious correlations as shortcuts. This feedback can then be used to refine the trained model. This technique is referred to as eXplanation Based Learning (XBL). In this promising area, however, there are several important research gaps left unexplored: (1) selecting efficient user feedback types, (2) assessing the impact of XBL on model uncertainty, and (3) designing effective and efficient XBL approaches that can learn from limited user interaction. To address these gaps, this thesis presents the following contributions: (1) an empirical comparison of the effectiveness of different feedback types in XBL; (2) an image perturbation approach to identify spurious correlations and to demonstrate the positive impact of XBL on the stability of model uncertainty; (3) two novel XBL methods, one that considers the distance of spurious regions and another one that can fine-tune a model to ignore dataset biases using little user feedback; and (4) a multi-hospital neuroradiologists pilot user study for an assessment of the impact of XBL on users’ perception of individual model outputs, as well as their overall trust and satisfaction with trained models. The work in the thesis is grounded in clinical applications related to Alzheimer’s Disease, which is the most common form of dementia in the world. Alzheimer’s Disease is preceded by the Mild Cognitive Impairment (MCI) stage, and early identification of MCI patients’ likely progression to the severe stage of the disease is of great clinical value. In addition to the XBL contributions described above, this thesis also describes novel approaches for the early identification of Alzheimer’s disease using mono-modal brain scan datasets and a cost-effective uncertainty–based multi-modal learning approach. Lastly, we integrate XBL and Alzheimer’s disease detection and perform user experiments to show that XBL positively influences neuroradiologists’ perception of trained models, and to demonstrate that XBL can be used to improve model performance while providing an effective communication medium to interact with end users.