Efficient Large Scale Clustering based on Data Partitioning

DC FieldValueLanguage
dc.contributor.authorBendechache, Malika
dc.contributor.authorLe-Khac, Nhien-An
dc.contributor.authorKechadi, Tahar
dc.date.copyright2016 IEEEen
dc.description3rd IEEE International Conference on Data Science and Advanced Analytics (DSAA 2016), Montreal, Canada, 17-19 October, 2016en
dc.description.abstractClustering techniques are very attractive for extracting and identifying patterns in datasets. However, their application to very large spatial datasets presents numerous challenges such as high-dimensionality data, heterogeneity, and high complexity of some algorithms. For instance, some algorithms may have linear complexity but they require the domain knowledge in order to determine their input parameters. Distributed clustering techniques constitute a very good alternative to the big data challenges (e.g.,Volume, Variety, Veracity, and Velocity). Usually these techniques consist of two phases. The first phase generates local models or patterns and the second one tends to aggregate the local results to obtain global models. While the first phase can be executed in parallel on each site and, therefore, efficient, the aggregation phase is complex, time consuming and may produce incorrect and ambiguous global clusters and therefore incorrect models. In this paper we propose a new distributed clustering approach to deal efficiently with both phases; generation of local results and generation of global models by aggregation. For the first phase, our approach is capable of analysing the datasets located in each site using different clustering techniques. The aggregation phase is designed in such a way that the final clusters are compact and accurate while the overall process is efficient in time and memory allocation. For the evaluation, we use two well-known clustering algorithms; K-Means and DBSCAN. One of the key outputs of this distributed clustering technique is that the number of global clusters is dynamic; no need to be fixed in advance. Experimental results show that the approach is scalable and produces high quality results.en
dc.description.sponsorshipScience Foundation Irelanden
dc.rights© 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.en
dc.subjectMachine learningen
dc.subjectBig Dataen
dc.subjectSpatial dataen
dc.subjectDistributed miningen
dc.subjectData analysisen
dc.titleEfficient Large Scale Clustering based on Data Partitioningen
dc.typeConference Publicationen
dc.statusPeer revieweden
item.fulltextWith Fulltext-
Appears in Collections:Computer Science Research Collection
Insight Research Collection
Files in This Item:
 File SizeFormat
Downloadinsight_publication.pdf3.27 MBAdobe PDF
Show simple item record

Page view(s)

Last Week
Last month
checked on Jan 17, 2022

Download(s) 50

checked on Jan 17, 2022

Google ScholarTM


If you are a publisher or author and have copyright concerns for any item, please email research.repository@ucd.ie and the item will be withdrawn immediately. The author or person responsible for depositing the article will be contacted within one business day.