A Comparison of Bayesian Deep Learning for Out of Distribution Detection and Uncertainty Estimation

Deep neural networks have been successful in diverse discriminitive classification tasks. Despite their good prediction performance, they are poorly calibrated– i.e., often assigns high confidence to misclassified predictions. Potential consequences could lead to trustworthiness and accountability of models deployed in real applications, where predictions are evaluated based on their confidence scores. In this work we propose to validate and test the efficacy of likelihood based models in the task of out-of-distribution (OoD) detection. On different datasets and metrics we show that Bayesian deep learning models on certain occasions marginally outperform conventional neural networks and in the event of minimal overlap between in/out distribution classes, even the best models exhibit a reduction in AUC scores. Preliminary investigations indicate the potential inherent role of bias due to choices of initialisation, architecture or activation functions.