Bayesian Neural Networks for Out of Distribution Detection

Empirical studies have demonstrated that point estimate deep neural networks despite being expressive estimators capturing rich interactions between covariates, nevertheless, exhibit high sensitivity in their predictions leading to overconfident misclassifications due to changes in the underlying representation of data distributions. This implication lead us to study the problem of out-of-distribution detection in identifying and characterising out-of-distribution inputs. This phenomenon has real world implications especially in high-stake applications where it is undesirable and often prohibitive for an estimator to produce overconfident misclassified estimates. Alternatively, Bayesian models present a principled way of quantifying uncertainty over predictions represented in the estimator’s parameters but at the same time they pose challenges when applied to large high dimensional datasets due to computational constraints requiring estimating high dimensional integrals over a large para- meter space. Moreover, Bayesian models among others present properties leading to simple and intuitive formulation and interpretation of the underlying estimator. Therefore, we propose to exploit this synergy between Bayesian inference and deep neural networks for out-of-distribution detection. This synergy leads to Bayesian neural networks exhibiting the following benefits (i) providing efficient and flexible neural network architectures applicable to large high dimensional datasets, (ii) estimating the uncertainty over the predictions captured in the predictive posterior distribution via Bayesian inference. We validate our findings empirically across a number of datasets and performance metrics indicating the efficacy of the underlying methods and estimators presented in regard to calibration, uncertainty estimation, out-of-distribution detection, detection of corrupted adversarial inputs and finally the effectiveness of the proposed contrastive objectives for out-of-distribution detection. We hope that the methods and results presented here reflect the importance of how brittle an estimator can be due to discrepancies between train and test distribution leading to real world implications of particular interest to reliable and secure machine learning. The algorithmic advances and research questions presented in this dissertation extend the domains of out-of-distribution detection and robustness against ambiguous inputs, in addition to exploring auxiliary information that can be incorporated during training. The resulting estimators overall are high dimensional exhibiting efficient detection.