Privacy and Security of Large-Scale Multimodal Data Warehouses for Healthcare Systems

Clinical data, a complex collection of data in varying formats, is produced daily in immense volumes. This data becomes even more intricate as different healthcare facilities and testing platforms use unique or proprietary data formats. The inherent complexity of healthcare adds layers of complication with a variety of patients, dependencies between healthcare providers, regulatory variations, and the necessity of maintaining patient confidentiality. In our digital age, crimes can occur virtually, making security and privacy the top concern in the digital world. Especially in the healthcare sector, clinical data, which holds sensitive personal information, can have catastrophic consequences if misused or accessed by unauthorized individuals. Existing literature in the field of clinical data exhibits considerable research gaps, with most components, such as data distribution, sharing, privacy, and security, being researched individually. Aspects such as efficient data distribution, privacy-by-design, rapid approvals for data sharing at emergency treatment centers, and data authorization have received less attention. This research addresses these gaps by providing an efficient data warehousing solution with multiple privacy and security measures integrated into the design. This study's key concern is ensuring data security at all stages - data input, cleaning, transportation, analysis, and presentation. This research has taken a comprehensive approach, initiating an in-depth literature survey to understand the existing knowledge in this area. Subsequently, clinical data categorization and governance were studied, followed by examining the state-of-the-art in different data warehousing concepts, secure data storage concepts, and data warehouse architectures. A distributed data warehouse architecture was designed after identifying the requirements and stakeholders of healthcare data warehouses. Open-source clinical datasets and randomly generated datasets were acquired to facilitate an understanding of the complexity of clinical data and testing of the data warehouse. Data security and privacy components were developed for the data warehouse architecture. Following this, a prototype based on the architecture was developed and subjected to rigorous evaluation and testing phases. Chapter 7 extensively evaluates the data security and privacy measures of the data warehouse, using datasets ranging from patient data to clinical history. Findings underscore the balance between performance and security; for instance, it confirms that usage of advanced encryption does not affect performance significantly. Additionally, tested security features, from multi-factor authentication to anonymization techniques, effectively upheld patient data's integrity and privacy. In conclusion, this research has undertaken a holistic and thorough approach to studying data warehousing in healthcare, moving from theoretical understanding to practical application and evaluation, aiming to provide an effective and secure solution for clinical data management.