Modelling water disinfection process and sperm motility with deep reinforcement and imitation learning

Abstract

varying natural and engineering phenomena. This discipline plays a crucial role in optimizing and predicting the behavior of complex systems, thereby facilitating advancements in numerous scientific and industrial domains. Models often rely on parameters that must be fine-tuned to accurately represent the real-world systems they are intended to simulate. Reinforcement and imitation learning techniques offer a powerful path for addressing this challenge, providing an approach to optimize these parameters based on objective functions or expert demonstrations. This thesis presents two case studies to illustrate the utilization of these technique in process modelling. In the first case, the optimization of a parametric model of a water disinfection process is addressed. Using reinforcement learning, a search for the optimal parameters that align the model with the experimentally observed behavior of the process is performed. This approach allows for the fine-tuning of the model to accurately reflect the real-world dynamics of the system. In the second case, a method for generating synthetic videos of spermatozoa is proposed. Through imitation learning, a parametric model of a spermatozoon is animated to simulate its real behavior in a petri dish. This method is subsequently used to generate synthetic videos with fully labeled samples, to which a style transfer process is applied to give them a photo-realistic appearance. This research provides a valuable tool for generating datasets to validate and train deep learning based Computer Aided Sperm Analysis systems. The results of this research show that both reinforcement learning and imitation learning are feasible paradigms for process modelling. The choice between these methods depends on the specific process to be modelled and it is primarily determined by the availability of expert demonstrations or a suitable metric. Furthermore, it has been shown that these paradigms, in conjunction with modelling, can contribute to obtain explainable solutions, providing insights that are as understandable as the proposed model itself.