Towards Efficient Learning of Dexterous Robotic Manipulation using Deep Learning Techniques

Autonomous robots are yet to achieve human-like dexterity and cannot complete sophisticated manipulation tasks. The following work, undertaken during my doctoral studies, aims to develop technical solutions to address some significant limitations in the current state of the art in robotic manipulation: (1). A novel robot controller based on deep reinforcement learning (DRL) is developed to solve a dexterous robotic manipulation competition. In comparison to traditional control methods, the DRL achieves a higher level of autonomy and is capable of learning effective control policies with minimal human involvement. Furthermore, we propose a knowledge transfer method aimed at addressing the inefficient exploration of DRL agents in high-complexity, sparse-reward environments. Our method, tested on physical robotic systems, ultimately won the competition and demonstrates state-of-the-art performance, surpassing other participants by more than 20% in performance. (2). Online learning-based robotic control methods, such as the DRL mentioned above, often require a significant number of data acquisitions from the robotic environment to converge to a high-quality solution. This poses limitations on their application in the physical world, as data acquisition from certain environments can be costly and time-consuming. Therefore, we introduce an offline learning-based controller development method that can learn effective policies from pre-collected, fixed datasets; this method demonstrates its effectiveness and has been tested on several challenging robotic tasks. Moreover, since the effectiveness of robot controllers developed through offline learning methods relies significantly on the quality of the static training dataset, we propose various strategies to enhance the quality of the training dataset. Specifically, these strategies encompass data filtering, data clustering, and data augmentation. These methods have demonstrated remarkable efficacy, allowing even basic control algorithms to reach state-of-the-art performance levels, promising broad applicability across diverse set of offline policy learning scenarios. (3). Tactile sensation will undoubtedly become a key input for future robotic controllers; it can enhance robots' perception of the environment, thereby increasing the precision and efficiency when performing tasks. However, due to the poor interpretability and high complexity of tactile data, learning to control robots with tactile inputs remains a relatively understudied area; specifically, performing grip force control based on incipient slip detection. Incipient slip refers to the earliest stage of an object slipping from a robot's gripper during contact; detection of it allows corrective control actions to be taken before a potential hazard state arises during gripping, thereby enhancing the stability of the robot's grip. Therefore, we developed a learning-based method for incipient slip detection, with the trained model demonstrating both high detection accuracy and real-time capabilities. Moreover, our model showcases significant robustness, allowing models trained in one environment to be effectively deployed in various other environments with consistent performance. Ultimately, this model enhances the safety of a range of practical robotic manipulation tasks, which cannot be achieved without using tactile sensation.