Optimizing Workflow Scheduling for High-Performance Computing

High-Performance Computing has evolved into a large-scale system that providesubiquitousaccessto asharedpoolofresourcesfor BigDatatasks such as workflow applications. However, processing of workflow applications generates large numbers of tasks and huge amounts of data, making them very expensive and energy-consuming. In addition, workflow scheduling in the HPC environment must handle multiple-objectives optimization and scheduling dynamicity. Currently, there are several approaches for scheduling workflow applications, but these approaches have difficulties in (i) generating efficient schedules on diverse HPC resources, (ii) managing the dynamicity of resources and workflow applications in the HPC to fully exploit parallelism, and (iii) selecting the best computational resources to execute each workflow task. This is because most of these approaches are highly dependent on job priority without considering the capacity of processing nodes relative to the size of workflow tasks. This thesis addresses the workflow scheduling optimization problem in three ways: It has been shown how challenging it is to schedule large tasks on HPC resources, and presented a novel mono-objective optimization heuristic to find a higher number of solutions that can optimize the execution cost of scientific workflows to meet user requirements and improve the quality of services of HPC systems. Iproposedabi-objectiveschemethataddressesaprocessingnodeselection problem for workflow scheduling. This scheme considers both large and small workflow tasks and introduces a technique to split the large workflow tasks into subtasks to reduce their execution time so that all tasks can meet their deadlines at a lower cost and in a shorter time. Finally, I proposed a technique to the workflow scheduling dynamicity problem that reduces the three main, but conflicting scheduling objectives: energy consumption, makespan, and execution cost.