Workload Orchestration in Extreme Edge Computing: A Reinforcement Learning-Based Approach

Edge computing is an emerging paradigm that brings computing resources closer to the network's edge, supporting compute-intensive and delay-sensitive applications. In multi-access Mobile Edge Computing (MEC), users can offload tasks to nearby servers for execution and then receive the results back. Nevertheless, the increasing popularity of IoT devices and data-intensive applications has driven a higher demand for computational resources, resulting in a rise in the number of edge servers at the network's edge. Simply expanding the number of high-capacity servers at the network's edge is not a sustainable solution for addressing the growing user demands. This situation has led to the emergence of the concept of Extreme Edge Computing (EEC), which aims to offload tasks to users' Extreme Edge Devices (EEDs), such as laptops and smartphones, located at the network's edge instead of relying solely on edge servers (e.g., Fog, Edge). However, such edge devices are heterogeneous and possess limited computing resources, which makes workload orchestration for task offloading in EEC challenging. Another challenge arises from the mobility of devices; in the EEC environments, both task generators and resources can be mobile and move at varying speeds and directions. Additionally, fluctuating workloads, characterized by varying task arrival rates and resource demands, as well as the complexity and interdependence of tasks with diverse requirements, further complicate the orchestration process. Existing research mainly focuses on workload orchestration for offloading tasks to high-capacity servers, such as MEC servers. Studies focusing on EEC mostly employ edge devices as helpers to offload tasks to the MEC or execute small tasks rather than as primary computing resources. These studies also have often been conducted under limited assumptions and within simplified environments. Challenges posed by environment dynamicity, such as resource limitations, device mobility, or task dependencies, are addressed independently. However, in the real world, these factors are interdependent and cannot be explored independently. Accordingly, this thesis proposes a novel resource, task, and mobility-aware Reinforcement Learning-based Workload Orchestration model called \textbf{ReWOrch} for task offloading in a dynamic EEC environment. The objectives of this work include a) maximizing the number of successful tasks, b) increasing EEDs resource utilization, c) reducing reliance on high-capacity servers, d) reducing energy consumption, and e) decreasing network usage. To achieve these objectives, 1) a multi-objective reinforcement learning-based dynamic resource allocation algorithm is developed to improve the utilization of EEDs and decrease the dependency on high-capacity servers while improving the number of successful tasks. 2) A fault-tolerant and emergency management scheme is proposed to enable the orchestrator to recover from device and task failures and operate effectively in emergencies. 3) A mobility analyzer scheme is proposed that leverages trajectory prediction to enable the orchestrator to select more reliable device resources for offloading tasks. 4) A task analyzer scheme is presented that introduces novel task partitioning models to orchestrate the workload of large and interconnected tasks. The EEC environment has been simulated using a state-of-the-art simulator, considering all the dynamic characteristics mentioned. The proposed model was evaluated across four real-world application areas by using various scenarios involving different challenges found in the EEC environment. The results indicate that the developed model outperformed the state-of-the-art works and managed the various challenges posed by complex environment features, including fluctuating workloads, high mobility, device heterogeneity, and task interdependencies.