Closed-Loop Learning for Smart Mobility and Smart Cities

Urban areas have recently become a realm for various significant environmental, economic, and societal transformations. The fast-growing population and exhaustive consumption of natural resources in cities have given rise to many environmental and ecological issues (e.g., increased levels of pollution and waste and other large-scale urbanization challenges). The problem of proper planning and managing the urbanization process is of great importance since it can notably improve the quality of living and make cities more intelligent and sustainable. Data sets used to build models of Smart City applications are often accompanied by closed-loop (feedback) effects. Smart City solutions often do not consider these feedback loops, which may deteriorate or even invalidate the model. Thus, there is a high demand to study and develop prediction and optimization techniques under feedback. Closedloop effects are usually not considered in classical machine learning techniques (e.g., regression, neural networks). In this work, we are ultimately interested in designing Smart City applications that account for closed-loop effects. Our preferred tool for designing such applications is reinforcement learning. Even though reinforcement learning has proven its applicability and success in many real-world scenarios, it still undergoes various challenges, including long training time, data management, environment perturbations, choice of an appropriate algorithm by the designer, and others. This thesis aims to address some of these challenges. Firstly, we want to develop methods to speed up reinforcement learning algorithms in large-scale nonstationary situations. Secondly, we aim to design a model of the reinforcement learning strategy in which the closed-loop effects (i.e., environment perturbations) can be minimized. Thirdly, we want to design tools to manage data collected by reinforcement learning agents in an optimal way. Finally, we design a framework to automatically select an appropriate reinforcement learning algorithm given no prior knowledge of the environment. The major contributions of this dissertation include 1) the design of an innovative multi-agent reinforcement learning framework SPToken based distributed ledger technologies and suitable for smart mobility applications, and 2) the design of likelihood-ratio based framework RLAPSE for an automatic selection of the appropriate reinforcement learning algorithm in a priori unknown environments. We believe that our results demonstrate the utility of reinforcement learning techniques for smart mobility and Smart City applications. The proposed approaches may significantly impact on sharing economy applications, where information needs to be shared to achieve an optimum outcome in an uncertain environment.