The Role of Physics-Informed Neural Networks in Modeling Biological and Epidemiological Dynamical Systems

An example of a Gompertz curve where f(t) = ae ^ -be^(-ct)

Over the years, neural networks have become increasingly popular in Machine Learning, thanks to their ability to learn from data with complex patterns. To briefly simplify their architecture: they comprise layers (input, hidden, and output layers) of interconnected nodes called neurons, and during training, they adjust the learned parameter values to minimize the loss between predictions and actual outputs. Physics-informed neural networks are an improvement of neural networks that integrate domain knowledge and physics principles into their learning process to solve complex problems, often governed by ordinary differential equations (ODEs) or partial differential equations (PDEs).

ODEs and PDEs model many scientific and engineering problems that describe the evolution of quantities over time and space (spatial-temporal), such as fluid dynamics, structural mechanics, heat transfer, and wave propagation, which, unfortunately, traditional NNs do not understand their physical properties. PINNs have proven highly advantageous in these areas through their flexibility, accuracy, and generalizability in handling complex geometries and boundary conditions imposed by PDE and ODE systems.

PINNs have proven effective for physics-related problems, but they can also be applied to other complex, adaptive dynamic systems that are not fully captured by physics equations alone. A good example is the SIR (Susceptible, Infected, Recovered) model, which describes the spread of infectious diseases within a population. In this model, susceptible individuals can be infected via contact with infected individuals, and infected individuals recover over time. Picture them as individuals moving between compartments under certain conditions and assumptions. The recovered individuals are assumed to gain complete (herd) immunity and cannot be reinfected. Let us break it down:

i) dS/dt = -βSI

ii) dI/dt = βSI — ϒI

iii) dR/dt = ϒI;

where the L.H.S derivatives tell us the rate of change in the numbers of the susceptible(S(t)), infected (I(t)), and recovered individuals (R(t)), respectively, while β is the infection rate, and ϒ the recovery rate. The constraints of this model are such that the susceptible, infected, and recovered individuals add up to the total population N.

PINNs can be used to solve both forward and inverse problems. Forward problems in neural networks involve predicting a system’s output given known inputs or parameters. Ideally, if you were to approximate a solution for a dynamical system using PNNS, you’d follow the following steps:

1. Define the dynamical system by defining the differential equation, the system parameters, and the initial condition: dx/dt = f (x, t)
2. Define your network architecture, with your input (the independent variable) and your output (the dependent variable). Specify the number of hidden layers and the number of neurons per layer, and choose a suitable activation function to introduce non-linearity and enable your model to learn complex patterns and relationships in the data.
3. Define your loss functions, i.e., the data loss, the physics loss, and the total loss, which is the sum of the two.
4. Finally, train and evaluate the model by comparing its predictions with known values or observed data. This is a simple explanation of the steps needed, assuming the reader knows the nitty-gritty of neural network training and evaluation.

Estimating unknown parameters from observed data is an inverse problem. The ODEs are defined, and the loss function that minimizes the error between the observed data and the predicted values is formulated. The remaining steps are similar to those we described above, but during training, the model parameters are optimized using a suitable optimizer by minimizing the total loss, and then updated accordingly.

There are several approaches for solving ODEs and PDEs using physics-informed neural networks. Some of these methods include the Finite Difference method, which discretizes the spatial and temporal domains of these equations into a grid and approximates derivatives using finite differences. The boundary integral method represents the solution to PDEs as an integral over the domain’s boundary, reducing dimensionality and simplifying the numerical solution. PINNs learn the boundary integral and then solve the PDE with reduced computational costs.

In oncology, PINNs have been employed to analyze tumor growth and model cancer progression dynamics, thereby optimizing treatment strategies. They predict the pharmacokinetics and pharmacodynamics of antineoplastic agents, therefore helping to optimize dosage regimens and enabling personalized treatment by estimating key parameters.

The Gompertz curve, based on the Gompertz law of mortality, which states that the rate of absolute mortality decays exponentially with current size, has been used to model tumor growth.

where r is the rate of growth y’(t)/y(t) and k is an arbitrary constant.

The rationale behind the application in tumor growth is that tumors are cellular populations growing in a confined space with limited nutrients

Where X’ is the derivative of the initial tumor size X(0), K is the carrying capacity, and alpha represents the cell proliferation rate.

However, this model has been criticized for failing to accurately fit small tumors because the proliferation rate is unbounded in the model, whereas in real biological systems, it is bounded by the cell division time. The model also fails to account for immune interactions.

PINNs have a wide range of promising capabilities, including their application in digital twin research, but they are not without limitations. Large, high-dimensional datasets can lead toparameter explosion, resulting in computational complexity, overfitting, and slow convergence. In stiff ODE or PDE systems, numerical instabilities may arise, necessitating careful regularization. Some physical systems that exhibit stochastic behavior require specialized techniques, such as stochastic differential equations. At the time of writing this blog piece, the literature indicates that few models have integrated these techniques; hence, this area remains underexplored.

Domain-specific knowledge is essential in identifying equations, parameters, physical laws, and boundary conditions to develop accurate PINN models. This calls for interdisciplinary collaborations to foster idea exchange, enhance interpretability, and develop large-scale, real-time applications to address complex challenges and drive innovation.

References:

[embed]Understanding Physics-Informed Neural Networks: Techniques, Applications, Trends, and Challenges Physics-informed neural networks (PINNs) represent a significant advancement at the intersection of machine learning…doi.org

https://www.mdpi.com/2227-7390/13/10/1664