Rensmeyer, Tim

Achieving robust uncertainty quantification for deep neural networks represents an important requirement in many real-world applications of deep learning, such as medical imaging where it is necessary to assess the reliability of a neural network’s prediction. Bayesian neural networks are a promising approach for modeling uncertainties in deep neural networks. Unfortunately, generating samples from the posterior distribution of neural networks is a major challenge. One significant advance in that direction would be the incorporation of adaptive step sizes, similar to modern neural network optimizers, into Monte Carlo Markov chain sampling algorithms without significantly increasing computational demand. Over the past years, several papers have introduced sampling algorithms with corresponding theorems stating that they achieve this property. In this paper, we demonstrate that these methods can have a substantial bias in the distribution they sample, even in the limit of vanishing step sizes and at full batch size. Furthermore, for most of the algorithms, we show that convergence to the correct distribution can be restored with a simple fix at the cost of increasing computational demand.

Molecular dynamics simulations with first-principles methods, such as density functional theory, are a cornerstone in the development of new battery and fuel cell materials. However, due to their high computational demand, their application is mostly limited to small systems and short time horizons. AI-based methods are a promising approach for accelerating first-principles simulations while maintaining high simulation accuracy. A key challenge, however, is the efficient training of such AI-based methods for specific systems of interest. In this article, we provide an overview of the training approach being researched at the Professorship of Computer Science in Mechanical Engineering at Helmut-Schmidt University, Hamburg.

The dtec.bw project CoupleIT! – IT-based sector coupling: Digitally controlled fuel cell and electrolyzer technologies for stationary and mobile applications is an interdisciplinary approach to combine a wide range of competencies from disciplines as varied as electrical power systems, economic and social sciences, computer sciences and networks as well as sustainable development and social acceptance research. As such, this article is composed of individual contributions, constituting the main chapters that showcase general approaches and motivations but also concrete results. This compendium article starts in with a delineation of the motivation behind research in so-called microgrids composed of fuel cell and electrolyzer components and a presentation of the microgrid architecture opted for in this project. Chapter two goes into more detail on the side of electrical engineering and the feasibility of a parallel operation of inverters in microgrids to achieve the ability for an upscaling. Chapter three highlights economic and technological factors for an economically viable and grid-maintaining deployment of a hydrogen-based energy system. In addition, degradation of Li-ion batteries is discussed against the background of their flexible operation in a microgrid and other scenarios. Chapter four grants a glimpse into the field of computer science and the possibility to use artificial intelligence and neural networks for a new way to simulate the behaviour of matter on atomic and molecular scales. This approach holds potential to increase the efficiency of fuel cells by improving the molecular design of fuel cell membranes used within this project. Chapter five elucidates the intricacies of secure communication within one but also between multiple microgrids, an important aspect for achieving a resilient system. Chapter six concludes this compendium by highlighting the human perspective seen from the field of psychological acceptance research nested in the broader context of sustainable development. Among other things, areas of potential barriers to a public acceptance of hydrogen technology are identified and ways to overcome those barriers proposed. This interdisciplinary round trip starts with electrical engineering (chapters one and two), economic and social sciences (chapter three), followed by computer sciences (chapter four) and computer networks (chapter five) whence the baton is passed for one last time to the field of sustainable development and psychological acceptance research (chapter six).

Ab initio molecular dynamics simulations of material properties have become a cornerstone in the development of novel materials for a wide range of applications such as battery technology and catalysis. Unfortunately, their high computational demand can make them unsuitable in many applications. Consequently, surrogate modeling via neural networks has become an active field of research. Two of the major obstacles to their practical application in many cases are assessing the reliability of the neural network predictions and the difficulty of generating suitable datasets to train the neural network in the first place. Bayesian neural networks offer a promising framework for modeling uncertainty, active learning and improving data efficiency and robustness by incorporating prior physical knowledge. However, due to the high computational demand and slow convergence of the gold standard approach of Monte Carlo Markov Chain (MCMC) sampling methods, variational inference via Monte Carlo dropout is currently the only sampling method successfully applied in this domain. Since MCMC methods have often displayed a superior quality in their uncertainty quantification, developing a suitable MCMC method in this domain would be a significant advance in making neural network-based molecular dynamics simulations more practically viable. In this paper, we demonstrate that convergence for state-of-the-art models with high-quality MCMC methods can still be achieved in a practical amount of time by introducing a novel parameter-specific adaptive step size scheme. In addition, we introduce a new stochastic neural network model based on the NequIP architecture and demonstrate that, when combined with our novel sampling algorithm, we obtain predictions with state-of-the-art accuracy as well as a significantly improved measure of uncertainty over Monte Carlo dropout. Lastly, we show that the proposed algorithm can even outperform deep ensembles while sampling from a single Markov chain.

The chaotic and dynamic nature of heat transfer results is either time-consuming or inaccurate predictions of the temperature field in simulations. In particular, the simulation of burning buildings is complex and at the same time the key enabler in saving lives and keeping property damage to a minimum. Deep Learning Neural Networks are a possibility to speed up the simulation process and make real-time predictions of temperature behavior or even replace the simulation process as a whole in the long run. This paper proposes a novel procedure to create a convolutional neural network-based method CNN1D3D, that is capable of approximating the behavior of temperature in burning rooms. The data used for training is created with time intensive, numerical simulations. CNN1D3D's architecture consists of an convolution-based temporal and spatial encoding as well of a transposed convolution based decoder, that creates temperature predictions in real-time. The work shows a possibility for a distinct feature extraction for temporal and spatial features. It shows how solutions generated by simulations based on differential equations hold implicitly the necessary information needed for the method to recreate the context of the data set. This forms a basis for the abstraction onto further fluid dynamics applications. This work has several real-world applications and forms the basis for future rescue route calculations. The solution can be generalized on other applications with similar data structures. It provides the opportunity to capture the complexity of interdependencies and correlations in the field of fluid dynamics.

Machine Learning methods have achieved some impressive results over the past decade. However, this success was in large part a result of utilizing large amounts of data and growing computational resources efficiently. To extend this recent success to domains where large quantities of high-quality data are not readily available, the field of informed machine learning has emerged, which aims at integrating preexisting knowledge into machine learning models. The aim of this paper is to provide an overview of the major new developments in this field and to discuss important open problems.