Designing and implementing data warehouse for agricultural big data
Files in This Item:
|1905.12411v1.pdf||3.73 MB||Adobe PDF||Request a copy|
|Title:||Designing and implementing data warehouse for agricultural big data||Authors:||Ngo, Vuong M.; Le-Khac, Nhien-An; Kechadi, Tahar||Permanent link:||http://hdl.handle.net/10197/11138||Date:||20-Jun-2019||Online since:||2019-10-10T08:46:23Z||Abstract:||In recent years, precision agriculture that uses modern information and communication technologies is becoming very popular. Raw and semi-processed agricultural data are usually collected through various sources, such as: Internet of Thing (IoT), sensors, satellites, weather stations, robots, farm equipment, farmers and agribusinesses, etc. Besides, agricultural datasets are very large, complex, unstructured, heterogeneous, non-standardized, and inconsistent. Hence, the agricultural data mining is considered as Big Data application in terms of volume, variety, velocity and veracity. It is a key foundation to establishing a crop intelligence platform, which will enable resource efficient agronomy decision making and recommendations. In this paper, we designed and implemented a continental level agricultural data warehouse by combining Hive, MongoDB and Cassandra. Our data warehouse capabilities: (1) flexible schema; (2) data integration from real agricultural multi datasets; (3) data science and business intelligent support; (4) high performance; (5) high storage; (6) security; (7) governance and monitoring; (8) consistency, availability and partition tolerant; (9) distributed and cloud deployment. We also evaluate the performance of our data warehouse.||Funding Details:||Science Foundation Ireland||metadata.dc.description.othersponsorship:||Origin Enterprises||Type of material:||Journal Article||Publisher:||Springer||Series/Report no.:||Lecture Notes in Computer Science (LNCS, volume 11514); Information Systems and Applications, incl. Internet/Web, and HCI book sub series (LNISA, volume 11514)||Copyright (published version):||2019 Springer||Keywords:||Data warehouse; Big data; Precision agriculture; Business intelligent; Constellation schema||DOI:||10.1007/978-3-030-23551-2_1||Language:||en||Status of Item:||Peer reviewed||Is part of:||Chan, K., Seshadri, S., Zhang, LJ. Big Data – BigData 2019 8th International Congress, Held as Part of the Services Conference Federation, SCF 2019, San Diego, CA, USA, June 25–30, 2019, Proceedings|
|Appears in Collections:||Computer Science Research Collection|
Show full item record
This item is available under the Attribution-NonCommercial-NoDerivs 3.0 Ireland. No item may be reproduced for commercial purposes. For other possible restrictions on use please refer to the publisher's URL where this is made available, or to notes contained in the item itself. Other terms may apply.