Now showing 1 - 10 of 31
- PublicationStacked-MLkNN: A stacking based improvement to Multi-Label k-Nearest NeighboursMulti-label classification deals with problems where each datapoint can be assigned to more than one class, or label, at the same time. The simplest approach for such problems is to train independent binary classification models for each label and use these models to independently predict a set of relevant labels for a datapoint. MLkNN is an instance-based lazy learning algorithm for multi-label classification that takes this approach. MLkNN, and similar algorithms, however, do not exploit associations which may exist between the set of potential labels. These methods also suffer from imbalance in the frequency of labels in a training dataset. This work attempts to improve the predictions of MLkNN by implementing a two-layer stack-like method, Stacked-MLkNN which exploits the label associations. Experiments show that Stacked-MLkNN produces better predictions than MLkNN and several other state-of-the-art instance-based learning algorithms.
- PublicationOn the Validity of Bayesian Neural Networks for Uncertainty EstimationDeep neural networks (DNN) are versatile parametric models utilised successfully in a diverse number of tasks and domains. However, they have limitations—particularly from their lack of robustness and over-sensitivity to out of distribution samples. Bayesian Neural Networks, due to their formulation under the Bayesian framework, provide a principled approach to building neural networks that address these limitations. This work provides an empirical study evaluating and comparing Bayesian Neural Networks to their equivalent point estimate Deep Neural Networks to quantify the predictive uncertainty induced by their parameters, as well as their performance in view of uncertainty. Specifically, we evaluated and compared three point estimate deep neural networks against their alternative comparable Bayesian neural network utilising well-known benchmark image classification datasets.
- PublicationEnsemble Topic Modeling via Matrix FactorizationTopic models can provide us with an insight into the underlying latent structure of a large corpus of documents, facilitating knowledge discovery and information summarization. A range of methods have been proposed in the literature, including probabilistic topic models and techniques based on matrix factorization. However, these methods tend to have stochastic elements in their initialization, which can lead to their output being unstable. That is, if a topic modeling algorithm is applied to the same data multiple times, the output will not necessarily always be the same. With this idea of stability in mind we ask the question – how can we produce a definitive topic model that is both stable and accurate? To address this, we propose a new ensemble topic modeling method, based on Non-negative Matrix Factorization (NMF), which combines a collection of unstable topic models to produce a definitive output. We evaluate this method on an annotated tweet corpus, where we show that this new approach is more accurate and stable than traditional NMF.
- PublicationDegree Centrality and the Probability of an Infectious Disease Outbreak in Towns within a Region Outbreak in Towns within a RegionAgent-based models can be used to help study the spread of infectious diseases within a population. As no individual town is in isolation, commuting patterns into and out of a town or city are a vital part of understanding the course of an outbreak within a town. Thus the centrality of a town in a network of towns, such as a county or an entire country, should be an important influence on an outbreak. We propose looking at the probability that an outbreak enters a given town in a region and comparing that probability to the centrality of the town. Our results show that as expected there is a relationship between centrality and outbreaks. Specifically, we found that the degree of centrality of a town affected the likelihood of an outbreak within the network spreading to the town. We also found that for towns where an outbreak begins the degree of centrality of the town affects how the outbreak spreads in the network.
- PublicationBenchmarking Multi-label Classification AlgorithmsMulti-label classification is an approach to classification prob- lems that allows each data point to be assigned to more than one class at the same time. Real life machine learning problems are often multi-label in nature—for example image labelling, topic identification in texts, and gene expression prediction. Many multi-label classification algorithms have been proposed in the literature and, although there have been some benchmarking experiments, many questions still remain about which ap- proaches perform best for certain kinds of multi-label datasets. This pa- per presents a comprehensive benchmark experiment of eleven multi- label classification algorithms on eleven different datasets. Unlike many existing studies, we perform detailed parameter tuning for each algorithm- dataset pair so as to allow a fair comparative analysis of the algorithms. Also, we report on a preliminary experiment which seeks to understand how the performance of different multi-label classification algorithms changes as the characteristics of multi-label datasets are adjusted.
- PublicationGraphical Perception of Value Distributions: An Evaluation of Non-Expert Viewers Data LiteracyAn ability to understand the outputs of data analysis is a key characteristic of data literacy and the inclusion of data visualisations is ubiquitous in the output of modern data analysis. Several aspects still remain unresolved, however, on the question of choosing data visualisations that lead viewers to an optimal interpretation of data. This is especially true when audiences have differing degrees of data literacy, and when the aim is to make sure that members of a community, who may differ on background and expertise, will make similar interpretations from data visualisations. In this paper we describe two user studies on perception from data visualisations, in which we measured the ability of participants to validate statements about the distributions of data samples visualised using different chart types. In the first user study, we find that histograms are the most suitable chart type for illustrating the distribution of values for a variable. We contrast our findings with previous research in the field, and posit three main issues identified from the study. Most notably, however,we show that viewers struggle to identify scenarios in which a chart simply does not contain enough information to validate a statement about the data that it represents. In the follow-up study, we ask viewers questions about quantification of frequencies, and identification of most frequent values from different types of histograms and density traces showing one or two distributions of values.This study reveals that viewers do better with histograms when they need to quantify the values displayed in a chart. Among the different types of histograms, interspersing the bars of two distributions in a histogram leads to the most accurate perception. Even though interspersing bars makes them thinner, the advantage of having both distributions clearly visible pays off. The findings of these user studies provide insight to assist designers in creating optimal charts that enable comparison of distributions, and emphasise the importance of using an understanding of the limits of viewers data literacy to design charts effectively.
- PublicationRadial Basis Function Data Descriptor (RBFDD) Network: An Anomaly Detection ApproachIn this paper, we propose a modification to the standard Ra-dial Basis Function (RBF) network that transforms it into a one-class classifier suitable for anomaly detection. We name this new approach the Radial Basis Function Data Descriptor (RBFDD) network. The RBFDD network is of interest as it has inherent adaptability in its architecture making it suitable for domains in which concept drift is a concern. Also, features learned by an RBFDD network (i.e., centers and spreads of Gaussian kernels and associated weights) provide us with a level of interpretability that has potential to be quite informative in terms of understanding the model learned and the reasoning behind flagging anomalies. In a set of evaluation experiments we compare the performance of the RBFDD network with some state of the art algorithms for anomaly detection over a collection of benchmark anomaly detection datasets. The results show that the RBFDD network is a promising approach and suggest potential for more investigations and promising directions for future work. We also investigate how RBFDD networks can be interpreted.
- PublicationReformulation Strategies of Repeated References in the Context of Robot Perception Errors in Situated DialogueWe performed an experiment in which human participants interacted through a natural language dialogue interface with a simulated robot to fulfil a series of object manipulation tasks. We introduced errors into the robot’s perception, and observed the resulting problems in the dialogues and their resolutions. We then introduced different methods for the user to request information about the robot’s understanding of the environment. In this work, we describe the effects that the robot’s perceptual errors and the information request options available to the participant had on the reformulation of the referring expressions the participants used when resolving a unsuccessful reference.
- PublicationExtending Jensen Shannon Divergence to Compare Multiple CorporaInvestigating public discourse on social media platforms has proven a viable way to reflect the impacts of political issues. In this paper we frame this as a corpus comparison problem in which the online discussion of different groups are treated as different corpora to be compared. We propose an extended version of the Jensen-Shannon divergence measure to compare multiple corpora and use the FP-growth algorithm to mix unigrams and bigrams in this comparison. We also propose a set of visualizations that can illustrate the results of this analysis. To demonstrate these approaches we compare the Twitter discourse surrounding Brexit in Ireland and Great Britain across a 14 week time period.
- PublicationA Categorisation of Post-hoc Explanations for Predictive ModelsThe ubiquity of machine learning based predictive models inmodern society naturally leads people to ask how trustworthythose models are? In predictive modeling, it is quite commonto induce a trade-off between accuracy and interpretability.For instance, doctors would like to know how effective sometreatment will be for a patient or why the model suggesteda particular medication for a patient exhibiting those symptoms? We acknowledge that the necessity for interpretabilityis a consequence of an incomplete formalisation of the prob-lem, or more precisely of multiple meanings adhered to a par-ticular concept. For certain problems, it is not enough to getthe answer (what), the model also has to provide an expla-nation of how it came to that conclusion (why), because acorrect prediction, only partially solves the original problem.In this article we extend existing categorisation of techniquesto aid model interpretability and test this categorisation