A Hybrid model for hot rolling pass schedule design using reinforcement learning

Abstract

Hot rolling of metals requires the design of pass schedules to accommodate different products and arrive at the desired target properties. Pass schedule design is often still performed manually, although automated approaches are emerging in the literature. One such approach is the use of reinforcement learning, which requires extensive training on simulations. To allow for efficient development of suitable reinforcement learning agents, it is important to reduce the computational cost of each simulation step as much as possible. While some simulation models for hot rolling exist, they have not been developed explicitly for reinforcement learning purposes. Today, there is a lack of fast approximate models to allow for efficient (pre-)training of reinforcement learning agents. To address this gap, we propose a hybrid model consisting of a data-driven model augmented by physical approximations. Our hybrid model produces better predictions than either of its components and is efficient enough to allow for quick training of reinforcement learning agents. We demonstrate that reinforcement learning agents pre-trained on our hybrid model can be transferred to a more accurate simulation and achieve good results after a short additional training phase. The two-stage training approach significantly reduces the overall training while achieving similar agent performances. The reduction in training time enables faster iteration cycles in agent and environment design, thereby creating a foundation for the development of reinforcement learning approaches for a wider range of hot rolling scenarios in the future.