Theory of AI for Scientific Computing

Abstract: Transport-based models, such as diffusion or flow-matching, have become a leading framework for generative modeling, by reducing the task to modeling conditional expectations under suitable "noise" semigroups. In this talk, we will describe how these models can be adapted beyond generative modeling to the setting of inverse problems. We will focus on two snippets: (i) performing provable posterior sampling in the context of linear inverse problems, and (ii) learning a generative model from corrupted measurements, akin to solving another linear inverse problem, this time in the space of distributions. Joint work with Jiequn Han, Chirag Modi and Eric Vanden-Eijnden.

Abstract: We introduce a novel kernel-based framework for learning differential equations and their solution maps, which is efficient in terms of data requirements (both the number of solution examples and the amount of measurements from each example), as well as computational cost and training procedures. Our approach is mathematically interpretable and supported by rigorous theoretical guarantees in the form of quantitative worst-case error bounds for the learned equations and solution operators. Numerical benchmarks demonstrate significant improvements in computational complexity and robustness, achieving one to two orders of magnitude improvement in accuracy compared to state-of-the-art algorithms. This presentation is based on joint work with Yasamin Jalalian, Juan Felipe Osorio Ramirez, Alexander Hsu, and Bamdad Hosseini. A preprint is available at: https://arxiv.org/abs/2503.01036.

Abstract: Physics-informed neural operators (PINOs) have emerged as powerful tools for learning solution operators of partial differential equations (PDEs). Recent research has demonstrated that incorporating Lie point symmetry information can significantly enhance the training efficiency of PINOs, primarily through techniques like data, architecture, and loss augmentation. In this work, we focus on the latter, highlighting that point symmetries oftentimes result in no training signal, limiting their effectiveness in many problems. To address this, we propose a novel loss augmentation strategy that leverages evolutionary representatives of point symmetries, a specific class of generalized symmetries of the underlying PDE. These generalized symmetries provide a richer set of generators compared to standard symmetries, leading to a more informative training signal. We demonstrate that leveraging evolutionary representatives enhances the performance of neural operators, resulting in improved data efficiency and accuracy during training.

Abstract: Physics-informed neural networks (PINNs) offer a flexible way to solve partial differential equations (PDEs) with machine learning, yet they still fall well short of the machine-precision accuracy many scientific tasks demand. This motivates an investigation into whether the precision ceiling comes from the ill-conditioning of the PDEs themselves or from the typical multi-layer perceptron (MLP) architecture. We introduce the Barycentric Weight Layer (BWLer), which models the PDE solution through barycentric polynomial interpolation. A BWLer can be added on top of an existing MLP (a BWLer-hat) or replace it completely (explicit BWLer), cleanly separating how we represent the solution from how we take its derivatives for the physics loss. Using BWLer, we identify fundamental precision limitations within the MLP: on a simple 1-D interpolation task, even MLPs with O(10⁵) parameters stall around 10^-8 relative error -- about eight orders above float64 machine precision -- before any PDE terms are added. In PDE learning, adding a BWLer lifts this ceiling and exposes a tradeoff between achievable accuracy and the conditioning of the PDE loss. For linear PDEs we fully characterize this tradeoff with an explicit error decomposition and navigate it during training with spectral derivatives and preconditioning. Across five benchmark PDEs, adding a BWLer on top of an MLP improves relative error by up to 30x for convection, 10x for reaction, and 1800x for wave equations while remaining compatible with first-order optimizers. Replacing the MLP entirely lets an explicit BWLer reach near-machine-precision on convection, reaction, and wave problems (up to 10 billion times better than prior results) and match the performance of standard PINNs on stiff Burgers’ and irregular-geometry Poisson problems. Together, these findings point to a practical path for combining the flexibility of PINNs with the precision of classical spectral solvers.

Abstract: Recent advances in deep learning have demonstrated an empirical trend: large-scale, general-purpose models trained on large datasets often outperform specialized models explicitly encoding domain-specific “constraints,” generally known as the “bitter lesson.” At the same time, in scientific modeling, we still require solutions that respect known physical principles. This raises a key question: Can we incorporate physics insights into machine learning methods that still benefit from advances in large-scale training and data?

In this talk, I will explore these questions through several examples. I will briefly discuss physics-informed neural networks (PINNs) to highlight challenges arising from naive incorporation of constraints. I will also discuss approaches that leverage large-scale training while ensuring physical consistency at deployment. Finally, I will discuss combining pre-trained score-based generative models (diffusion and flow matching) with statistical physics insights. By interpreting sampling from generative models as dynamics governed by the Onsager-Machlup action functional, we can efficiently generate physically realistic dynamical pathways (e.g., molecular transitions or protein folding events) without additional training. Throughout, I'll highlight open questions on how physics and mathematical insights can be effectively integrated into methods that naturally improve with increasing data and model scale.