Researchers from the University of California, Berkeley, and the University of Texas at Austin have developed a new data scaling technique that significantly improves the performance of deep learning models in simulating complex diffusion-reaction systems. The team, led by Yuxiao Yi, Weizong Wang, Tianhan Zhang, and Zhi-Qin John Xu, presented their findings in a study published in the Journal of Computational Physics.
The research focuses on addressing the challenges posed by high-dimensional and stiff ordinary differential equations (ODEs) that often make simulations of complex systems computationally expensive. The team’s novel approach, called the Generalized Box-Cox Transformation (GBCT), is designed to mitigate the multiscale challenges inherent in these systems by rescaling state variables that span tens of orders of magnitude to a more consistent order of magnitude.
Deep learning has shown promise in modeling and sampling stiff systems, but data scaling techniques have not been extensively explored. The GBCT method aims to address this gap by reducing the frequency bias of deep neural networks when handling multi-magnitude or high-frequency data. By integrating GBCT into their previous data-driven framework, the researchers evaluated its performance against the original baseline surrogate model across six representative scenarios. These scenarios included a 21-species chemical reaction kinetics model, a 13-isotope nuclear reaction model, the Robertson problem coupled with diffusion, and simulations of two-dimensional turbulent reaction-diffusion systems as well as one- and two-dimensional nuclear reactive flows.
The numerical experiments demonstrated that GBCT significantly reduces prediction errors by up to two orders of magnitude compared to the baseline model, particularly in the long-term evolution of dynamical systems. Additionally, GBCT achieved comparable performance with only about one-sixth of the training epochs. Frequency analysis revealed that GBCT rescales high-frequency components of the objective function toward lower frequencies, aligning with the neural network’s natural low-frequency bias. This alignment enhances both training and generalization capabilities.
For the energy industry, this research offers practical applications in improving the accuracy and efficiency of simulations for complex chemical and nuclear reaction systems. By leveraging the GBCT method, energy researchers and engineers can better model and predict the behavior of these systems, leading to more informed decision-making and potentially significant cost savings. The source code to reproduce the results in this paper is available at https://github.com/Seauagain/GBCT.
This article is based on research available at arXiv.