AI at the Edge: Enabling low latency computation in mobile and heterogeneous environments via Federated Learning

The ubiquitous and increasingly pervasive adoption of smart devices of various sizes and purposes is considered an enabler for the successful development of the services. The significantly increased number of edge devices plays a crucial role in generating substantial datasets and processing such data is an essential part of many smart applications. The rapid increase in data generated by edge devices, along with the need for low-latency processing, privacy, and scalability, has prompted the shift towards edge computing. However, in such smart applications, when deployed at the edge of the network, running Machine Learning (ML) models is not feasible as they require lots of data samples that it is not possible for a single edge device to store and train. Moreover, due to geo-distribution and partial observability, the data collected on an edge device can be very limited and biased. Thus, it is challenging to generate ML models that can adapt properly to unseen situations. To address these challenges, it is proposed to utilize Distributed Machine Learning (DML) methods and collect the data from edge devices in a server. The traditional cloud-centric DML can store large amounts of datasets and provides strong computational powers to train large ML models. However, this system fails in applications that require low latency, privacy, and scalability. For this reason, Federated Learning (FL) is proposed to benefit from all devices' datasets, without the need to collect data samples at a server. In FL, the model training part is left on the device side and local models' weights, which are trained by local data are shared with a central server, to combine all of those local models to generate a global model. Afterwards, this global model is distributed to edge devices for further training rounds. A global model is then generated using all local datasets while the data remains local to devices to preserve privacy. Using FL, the learning mechanism is collaborative, distributed and decentralised, and the system is more scalable and more efficient in communication. However, the transmission of local model weights at each training round still can cause communication overhead in the system. Many of the FL models also face a challenge when sharing the same global model architecture with all devices. Especially, when heterogeneous environments, distributing the same global model architecture is not working in FL. Literature shows that model compression methods can tackle some of the mentioned challenges. Quantization reduces the model size, positively impacting the model's communication cost. The model pruning decreases the computation cost and accelerates the inference at the edge devices by sparsification, leading to reduced memory and battery usage. However, existing studies aim to address each challenge separately, falling short of providing comprehensive representation for real-world applications, where each device can own unique data samples in a mobile environment. Moreover, factors such as low battery levels and connectivity issues make it impossible to share local models in each global round. This thesis explores how model compression methods can address the challenges of FL when dealing with heterogeneous scenarios in mobile systems. The main point argued is that, unlike the dominant approach which is a homogeneous global model distribution in FL, a model heterogeneous system should be designed for environments where devices differ in terms of data distribution, communication and computation abilities. To do this, this research introduces a transfer learning approach to produce a series of submodels for resource-constrained devices for mobile and heterogeneous FL settings considering real-world circumstances. This involves data-free model compressions of a pre-trained dense model on the server side through model pruning and quantization, to prioritize privacy and sources of edge devices.