A Hybrid model for hot rolling pass schedule design using reinforcement learning
Publication date
2024-03
Document type
Conference paper
Author
Kemmerling, Marco
Frahm, Nils
Idzik, Christian
Abdelrazeq, Anas
Bailly, David
Schmitt, Robert H.
Organisational unit
RWTH Aachen
Institute of Metal Forming, RWTH Aachen
Publisher
UB HSU
Book title
Machine learning for cyber physical systems
Part of the university bibliography
✅
Language
English
Keyword
Hot rolling
Reinforcement learning
Hybrid mode
Pass schedule
Abstract
Hot rolling of metals requires the design of pass schedules to accommodate different products and arrive at the desired target properties. Pass schedule design is often still performed manually, although automated approaches are emerging in the literature. One such approach is the use of reinforcement learning, which requires extensive training on simulations. To allow for efficient development of suitable reinforcement learning agents, it is important to reduce the computational cost of each simulation step as much as possible. While some simulation models for hot rolling exist, they have not been developed explicitly for reinforcement learning purposes. Today, there is a lack of fast approximate models to allow for efficient (pre-)training of reinforcement learning agents. To address this gap, we propose a hybrid model consisting of a data-driven model augmented by physical approximations. Our hybrid model produces better predictions than either of its components and is efficient enough to allow for quick training of reinforcement learning agents. We demonstrate that reinforcement learning agents pre-trained on our hybrid model can be transferred to a more accurate simulation and achieve good results after a short additional training phase. The two-stage training approach significantly reduces the overall training while achieving similar agent performances. The reduction in training time enables faster iteration cycles in agent and environment design, thereby creating a foundation for the development of reinforcement learning approaches for a wider range of hot rolling scenarios in the future.
Version
Published version
Access right on openHSU
Open access