ML4CPS – Machine Learning for Cyber-Physical Systems

Cyber Physical Systems are characterized by their ability to adapt and learn from their environment. Applications include advanced condition monitoring, predictive maintenance, diagnosis tasks, and many other areas. All these applications have in common that Machine Learning and Artificial Intelligence are the key technologies. However, applying ML and AI to CPS poses challenges such as limited data, less understood algorithms, and the need for high algorithm reliability. These topics were a focal point at the 7th ML4CPS - Machine Learning for Cyber-Physical Systems Conference in Berlin, held from the 20th to the 21st, where industry and research experts discussed current advancements and new developments.

This paper introduces an advanced AI-based automated surface inspection system for enhanced quality control in the manufacturing of white-coated sheet metal parts. The study emphasizes overcoming the challenges in AI-driven inspection, such as the need for large datasets corresponding to numerous physical components, which presents storage and logistical issues. By integrating Convolutional Neural Networks (CNNs) and a novel annotation process, the system can be trained effectively on various surface defects. The paper discusses three big data acquisition challenges and provides a solution approach, including large data volumes equivalent to numerous physical components, a novel division of the training process to reduce the workload for domain experts and the relevance of previously clearly defined defect classes.

Industrial separators play a pivotal role in production processes of various sectors such as chemical, pharmaceutical, biotechnology, oil extraction and food industries, with over 3000 distinct applications. Operating these separators involves managing several process parameters as well as discharge and cleaning cycles, which are hard to control mainly due to deficiencies of current physical sensor technology. Recent studies have shown that machine learning can be utilized to detect faults and particle presence in separators via vibration data. However, traditional machine learning methods require domain expertise or vast amounts of labeled data. We propose the use of self-supervised learning to resolve this issue by learning useful representations from unlabeled data, which is significantly easier and cheaper to obtain. An empirical validation on data from a disk stack separator shows that self-supervised learning can improve upon manual feature engineering and supervised approaches in terms of cost, accuracy and data efficiency.

Neural Networks have been successful in solving complex problems across various fields. However, they require significant data to learn effectively, and their decision-making process is often not transparent. To overcome these limitations, causal prior knowledge can be incorporated into neural network models. This knowledge improves the learning process and enhances the robustness and generalizability of the models. We propose a novel framework RCINN that involves calculating the inverse probability of treatment weights given a causal graph model alongside the training dataset. These weights are then concatenated as additional features in the neural network model. Then incorporating the estimated conditional average treatment effect as a regularization term to the model loss function, the potential influence of confounding variables can be mitigated, leading to bias minimization and improving the neural network model. Experiments conducted on synthetic and benchmark datasets using the framework show promising results.

The integration of machine learning (ML) into manufacturing processes is crucial for optimizing efficiency, reducing costs, and enhancing overall productivity. This paper proposes a comprehensive ML pipeline tailored for manufacturing applications, leveraging the widely recognized Cross-Industry Standard Process for Data Mining (CRISP-DM) as its foundational framework. The proposed pipeline consists of key phases, namely business understanding, use case selection and specification, data integration, data preparation, modelling, deployment, and certification. These are designed to meet the unique requirements and challenges associated with ML implementation in manufacturing settings. Within each phase, sub-topics are defined to provide a granular understanding of the workflow. Responsibilities are clearly outlined to ensure a structured and efficient execution, promoting collaboration among stakeholders. Further, the input and output of each phase are defined. The methodology outlined in this research not only enhances the applicability of CRISP-DM in the manufacturing domain but also serves as a guide for practitioners seeking to implement ML solutions in a systematic and well-defined manner. The proposed pipeline aims to streamline the integration of ML technologies into manufacturing processes, facilitating informed decision-making and fostering the development of intelligent and adaptive manufacturing systems.

Kubeflow integrates a suite of powerful tools for Machine Learning (ML) software development and deployment, typically showcased independently. In this study, we integrate these tools within an end- to-end workflow, a perspective not extensively explored previously. Our case study on anomaly detection using telemetry data from the International Space Station (ISS) investigates the integration of various tools—Dask, Katib, PyTorch Operator, and KServe—into a single Kubeflow Pipelines (KFP) workflow. This investigation reveals both the strengths and limitations of such integration in a real-world context. The insights gained from our study provide a comprehensive blueprint for practitioners and contribute valuable feedback for the open source community developing Kubeflow.

Manufacturing companies usually isolate their production networks from other networks to ensure security against external attacks and to guarantee a fail-safe 24/7 operational service. However, these measures make it technically and organizationally complex to install new sensors or deploy new software in the production process. As a result, machine learning is only used to a limited extent in manufacturing, as these models require regular adaptations. To tackle this challenge, one possible solution is to install an additional network that is not connected to the production network. This network can be utilized for rapid prototyping of new sensors, advanced data analysis, or the deployment of machine learning models. One possible solution is to install a radio-based low-power, long-range network, having the property to capture data over large distances with only little power consumption. This paper examines the potential of retrofitting cyberphysical systems with such a network in combination with machine learning methods. The results are evaluated through three practical use cases: monitoring a workspace with a molding machine, monitoring the cycles of a washing machine, and predicting the daily consumption profile of a main water pipeline.

This paper presents the Hybrid Online Timed Automaton Learning Algorithm (HyOTALA), a novel approach for anomaly detection in cyber-physical production systems (CPPS). It addresses the challenge of using hybrid data, combining sparse discrete and continuous signals, by learning a hybrid timed automaton model in an online setting. This model captures the dynamics of discrete manufacturing processes and provides significant advances in model identification for CPPS. The effectiveness of HyOTALA is demonstrated through a practical application, highlighting its potential to improve anomaly detection capabilities in industrial settings.

In many use cases cyber-physical systems are employed to produce products of small batch sizes as efficiently as possible. From an engineering standpoint, a major drawback of this flexibility is that the architecture of the cyber-physical system may change multiple times over its lifetime to accommodate new product variants. To keep a cyber-physical system working normally it has become common to employ fault diagnosis algorithms. These algorithms partly rely on physical first-principles models that need to be updated when the architecture of the system changes which usually has to be done manually. In this article we present a practical approach to obtain such a first-principles model through evaluating piping and instrumentation diagrams (P&IDs) with visual questions answering (VQA) models. We demonstrate that it is possible to leverage VQA models to construct physical equations which are a preliminary stage for the creation of models suitable for fault diagnosis. We evaluate our approach on OpenAIs GPT-4 Vision Preview model using a P&ID we created for a benchmark water tank system. Our results show that VQA models can be used to create physical first-principles models.