← blog

an exploration of PINNs

03.03.2025 as i continue my physics undergraduate curriculum at illinois, i have found an interest in the intersection of theoretical physics and statistical methods, specifically machine learning. on a high level, physical systems are governed by partial differential equations (PDEs), which are notoriously difficult to solve analytically. machine learning, on the other hand, is a powerful tool for approximating functions and making predictions based on data.

a famous example of one of these systems is the navier-stokes equations, a set of coupled, nonlinear PDEs that govern the motion of viscous fluids. in many situations, no closed-form analytic solution exists, and standard numerical methods become computationally expensive. this is where physics-informed neural networks (PINNs) come in.

\( \nabla \cdot \mathbf{u} = 0 \quad \text{and} \quad \frac{\partial \mathbf{u}}{\partial t} + (\mathbf{u}\cdot\nabla)\mathbf{u} = -\frac{1}{\rho}\nabla p + \nu \nabla^2\mathbf{u} + \mathbf{f} \)

in turbulent flow, the velocity field \( \mathbf{u} \) exhibits chaotic behavior. PINNs are particularly useful in this context, allowing us to solve these complex problems while directly incorporating physical laws into the model.

but what is a PINN? first, let's define a standard neural network. in practice, neural networks are flexible functions of input data \( \mathbf{x} \) that aim to predict an output \( \mathbf{y} \).

\( NN(\mathbf{x}, \mathbf{\theta}) \approx \mathbf{y}(\mathbf{x}) \)

given training data, we fine-tune the parameters \( \mathbf{\theta} \) so that the network approximates the true function. this is done by minimizing a loss function, \( \mathcal{L}(\mathbf{\theta}) \), which quantifies the difference between predicted and true outputs.

\( \mathcal{L}(\mathbf{\theta}) = \frac{1}{N} \sum_{i=1}^N \left( NN(\mathbf{x}_i, \mathbf{\theta}) - \mathbf{y}_i \right)^2 \)

neural networks are a vast topic, and many important details are omitted here. however, the key idea is that neural networks are universal function approximators, capable of approximating any continuous function to arbitrary precision.

to illustrate how PINNs are used, let's consider a simpler system from undergraduate mechanics: a damped harmonic oscillator with mass \( m \), spring constant \( k \), and damping coefficient \( \mu \).

\( m \ddot{u} + \mu \dot{u} + ku = 0 \)

assume initial conditions \( u(0) = 1 \) and \( \dot{u}(0) = 0 \). although the system has an analytic solution, we will use a PINN to approximate it, modeling \( u(t) \) as \( NN(t, \mathbf{\theta}) \).

the key idea is to define a loss function with two components: a boundary loss \( \mathcal{L}_b \) to enforce initial conditions, and a physics loss \( \mathcal{L}_p \) to enforce the differential equation itself.

\( \mathcal{L}_b = \lambda_1 (NN(0, \mathbf{\theta}) - 1)^2 + \lambda_2 (\dot{NN}(0, \mathbf{\theta}) - 0)^2 \)

\( \mathcal{L}_p = \frac{1}{N} \sum_{i=1}^N \left( m\frac{d^2}{dt^2} NN(t_i, \mathbf{\theta}) + \mu\frac{d}{dt} NN(t_i, \mathbf{\theta}) + k NN(t_i, \mathbf{\theta}) \right)^2 \)

to simplify, we can rearrange into parameters \( \zeta \) (damping ratio) and \( \omega \) (natural frequency):

\( \ddot{u} + 2\zeta\omega\dot{u} + \omega^2 u = 0 \)

where \( \zeta = \frac{\mu}{2\sqrt{mk}} \) and \( \omega = \sqrt{\frac{k}{m}} \). the PINN outputs \( \hat{u}(t) \), which can then be compared to the true \( u(t) \).

code for the implementation in this post is available on my github.