On the convergence of locally adaptive and scalable diffusion-based sampling methods for deep Bayesian neural network posteriors
Publication date
2025-08
Document type
Konferenzbeitrag
Author
Organisational unit
Conference
28th International Conference on Artificial Intelligence and Statistics (AISTATS 2025) ; Mai Khao, Thailand ; May 3-5, 2025
Publisher
MLResearchPress
Series or journal
Proceedings of Machine Learning Research
ISSN
Periodical volume
258
Book title
Proceedings of AISTATS 2025
Article ID
258:145-153
Peer-reviewed
✅
Part of the university bibliography
✅
Language
English
Abstract
Achieving robust uncertainty quantification for deep neural networks represents an important requirement in many real-world applications of deep learning, such as medical imaging where it is necessary to assess the reliability of a neural network’s prediction. Bayesian neural networks are a promising approach for modeling uncertainties in deep neural networks. Unfortunately, generating samples from the posterior distribution of neural networks is a major challenge. One significant advance in that direction would be the incorporation of adaptive step sizes, similar to modern neural network optimizers, into Monte Carlo Markov chain sampling algorithms without significantly increasing computational demand. Over the past years, several papers have introduced sampling algorithms with corresponding theorems stating that they achieve this property. In this paper, we demonstrate that these methods can have a substantial bias in the distribution they sample, even in the limit of vanishing step sizes and at full batch size. Furthermore, for most of the algorithms, we show that convergence to the correct distribution can be restored with a simple fix at the cost of increasing computational demand.
Version
Published version
Access right on openHSU
Metadata only access