Predicting the Unpredictable: A New Approach to Dynamic Systems

Author: Denis Avetisyan

Researchers have developed a probabilistic framework that accurately forecasts the behavior of complex systems, even when the underlying models are imperfect.

Estimated latent forces, $\bm{\eta}$, accurately reflect true nonlinear forces at each degree of freedom within the three-degree-of-freedom system, as demonstrated by the close alignment within $\pm 2\sigma$ confidence intervals.

This work combines Gaussian Process Latent Force Models, Bayesian Neural Networks, and Kalman filtering for robust prediction and uncertainty quantification in misspecified structural dynamical systems.

Accurate prediction of complex dynamical systems is often hampered by unavoidable model inaccuracies and uncertainties. This work introduces a novel ‘Probabilistic Digital Twin for Misspecified Structural Dynamical Systems via Latent Force Modeling and Bayesian Neural Networks’-a framework that integrates Gaussian Process Latent Force Models, Bayesian Neural Networks, and Kalman filtering to robustly estimate system behavior even with incomplete or flawed physics. By treating model-form errors as latent forces, the approach enables end-to-end uncertainty quantification and propagates it systematically from diagnosis to prediction. Could this probabilistic approach unlock more trustworthy and reliable digital twins for a wider range of structural and dynamical systems?

The Illusion of Complete Representation

The pursuit of accurately representing real-world systems through dynamic modeling is frequently hampered by an inherent simplification of complexity. While models strive to capture essential behaviors, they inevitably omit subtle interactions, nonlinearities, and high-frequency dynamics present in the actual system. This simplification, though often necessary for tractability, introduces discrepancies between the model’s predictions and observed reality. Consequently, predictions can deviate significantly from actual outcomes, especially over extended horizons or under conditions not fully represented in the modeling process. The resulting inaccuracies limit the model’s utility in critical applications such as control systems, forecasting, and decision-making, highlighting the persistent challenge of bridging the gap between mathematical abstraction and the intricate nuances of the physical world.

The limitations of dynamic modeling frequently arise not from fundamental flaws in the mathematical framework, but from the inherent difficulty of representing complete reality. Real-world systems are often influenced by a multitude of factors, and initial model representations necessarily involve simplifications and approximations. These omissions create unmodeled phenomena – subtle, often nonlinear behaviors that deviate the actual system from its idealized counterpart. These could include unmeasured disturbances, high-frequency dynamics beyond the model’s bandwidth, or interactions with previously unknown elements. Consequently, even meticulously crafted models can exhibit inaccuracies when confronted with the full spectrum of real-world conditions, highlighting the crucial need for robust identification techniques and careful validation against empirical data to account for these subtle, yet impactful, discrepancies.

Model-Form Errors (MFEs) represent a critical limitation in system identification, arising when the chosen mathematical structure of a model fails to adequately represent the true underlying dynamics of a physical system. These discrepancies, even if subtle, can significantly degrade predictive performance and control efficacy. Unlike parameter errors – which can often be minimized through optimization – MFEs stem from a fundamental mismatch between the model’s form and the system’s behavior, potentially leading to biased estimates and unreliable extrapolations. Consequently, a model exhibiting substantial MFEs may appear accurate within the confines of the training data, yet fail catastrophically when confronted with novel operating conditions or inputs. Identifying and mitigating MFEs often requires revisiting the initial assumptions about the system and considering more complex, or entirely different, model structures to better capture the unmodeled reality.

The GPLFM framework accurately predicts system states under noisy conditions, providing uncertainty quantification through ±2σ confidence bounds, and significantly outperforms a nominal model prediction.

Joint Estimation: Embracing Uncertainty

The Gaussian Process Latent Force Model (GPLFM) provides a unified approach to state and parameter estimation by treating both system states – the directly observable variables – and latent Mechanical Force Elements (MFEs) – the underlying causes of system behavior – as random variables. Unlike traditional methods that estimate these separately, the GPLFM jointly estimates both through a probabilistic model. This is achieved by defining a Gaussian process prior over the MFEs, allowing for the representation of uncertainty in their values and relationships. The GPLFM then uses observed system states to infer the posterior distribution over both states and MFEs, effectively leveraging the correlation between them for improved accuracy and robustness. This joint estimation is particularly beneficial in scenarios with limited or noisy data, as the model can utilize the learned relationships between states and MFEs to regularize the estimation process and reduce uncertainty.

A Bayesian Neural Network (BNN) is incorporated to model the relationship between observable system states and the unobserved Manifest Force Equations (MFEs). Unlike traditional neural networks that provide point estimates, the BNN outputs a probability distribution over the MFEs, quantifying the uncertainty associated with the mapping. This is achieved by placing prior distributions over the network’s weights and biases, and utilizing Bayesian inference techniques – such as Markov Chain Monte Carlo (MCMC) or Variational Inference – to obtain the posterior distribution. The parameters of this posterior distribution define the mean and variance of the predicted MFEs, allowing for a probabilistic representation of the state-to-MFE relationship and enabling the generation of ‘PseudoMeasurements’ with associated uncertainties.

The Bayesian Neural Network (BNN) within the framework generates ‘PseudoMeasurements’ by providing probabilistic estimates of the underlying Manifold Feature Embeddings (MFEs). These PseudoMeasurements are not derived from direct observation but are predictions generated by the BNN, quantified with associated uncertainties. The output of the BNN is a probability distribution, typically defined by a mean $\mu$ and variance $\sigma^2$, for each MFE dimension. These probabilistic estimates, represented as pairs of $(\mu_i, \sigma_i)$ for MFE dimension $i$, are then incorporated into the GPLFM’s estimation process alongside actual sensor measurements, effectively augmenting the observable data with learned, probabilistic information about the MFEs. This integration is performed through a Kalman filter, treating the PseudoMeasurements as virtual sensor readings with associated noise covariance determined by $\sigma_i^2$.

Using the GPLFM framework, predicted system states accurately capture the true response under sinusoidal excitation, as demonstrated by the ±2σ confidence interval, and outperform a nominal model prediction.

Kalman Filtering: Refinement Through Integration

The Kalman Filter functions as the central estimator within the system, combining conventional direct measurements with PseudoMeasurements generated by the Bayesian Neural Network (BNN). This integration is achieved through a recursive process of prediction and update. Direct measurements provide information about the system’s current state, while the BNN-derived PseudoMeasurements offer additional data points informed by the learned model and its associated uncertainty. The Kalman Filter then weights these inputs, based on their respective noise characteristics, to produce a refined estimate of the system’s state vector, $x_t$. This process effectively fuses data from both sources, leveraging the strengths of model-based prediction and data-driven observation to improve the accuracy and reliability of state estimation.

The KF_Update step in the Kalman Filter utilizes PseudoMeasurements generated by the Model Form Error (MFE) network to address inaccuracies stemming from unmodeled system dynamics. These PseudoMeasurements, representing estimations of errors not accounted for in the primary system model, serve as corrective inputs during the update process. Specifically, the Kalman Gain, calculated based on the measurement noise covariance and the estimated error covariance, weights these PseudoMeasurements to optimally adjust the state estimate. This correction minimizes the a posteriori error covariance, effectively reducing the impact of unmodeled effects and improving the overall accuracy of the state estimate at each time step. The weighting ensures that PseudoMeasurements with lower estimated noise contribute more significantly to the correction, providing a data-driven refinement of the system’s state.

The Kalman Filter facilitates robust state prediction by iteratively updating its estimate of the system’s state with each new data assimilation. This process involves combining a prior estimate with current measurements, weighted by their respective uncertainties, to produce a posterior estimate. Crucially, this recursive update handles model uncertainty by explicitly incorporating process noise, which represents deviations from the assumed system dynamics. The filter’s ability to continually refine its estimate based on incoming data, even when the underlying model is imperfect, allows it to maintain prediction accuracy over time, effectively mitigating the impact of unmodeled effects and noise. This is achieved through the prediction and update steps, where the filter projects the state forward in time and then corrects it based on new observations, respectively.

The Bayesian Neural Network architecture maps system states to model-form errors by predicting the mean and Cholesky factor, enabling reconstruction of the full error covariance.

Beyond Prediction: The Promise of Digital Twins

The accuracy of predicting a system’s future behavior under varying forces is substantially enhanced by integrating estimated Modal Frequency Effects (MFEs) into the prognostic process. This approach moves beyond traditional linear modeling by accounting for nonlinear characteristics – such as cubic stiffness and hysteresis – that significantly influence dynamic response. By characterizing these effects as MFEs and incorporating them into the prediction model, the system’s response to new force excitations can be determined with greater precision. This is critical for applications requiring reliable long-term predictions, and enables a virtual representation – a DigitalTwin – to accurately reflect the physical system’s state and anticipate its evolution, leading to improvements in monitoring, diagnostics, and control strategies.

The predictive modeling framework demonstrates exceptional accuracy in forecasting system behavior, consistently achieving a Normalized Mean Square Error (NMSE) below 1% for both displacement and velocity predictions. This high level of precision was maintained across a diverse range of simulated scenarios and across multiple degrees of freedom (DOFs), indicating robust performance beyond specific operating conditions. For instance, under sinusoidal excitation at DOF 1, the framework yielded an NMSE of just 0.0309% for displacement and 0.0595% for velocity, while even with the complexities of filtered white noise excitation, NMSE values remained remarkably low at 0.3527% for displacement and 0.6849% for velocity. These results confirm the framework’s ability to reliably predict future system states, paving the way for advanced applications in monitoring, diagnostics, and control systems.

Rigorous testing of the predictive framework revealed exceptional accuracy in state estimation. Under controlled sinusoidal excitation applied at Degree of Freedom 1 (DOF 1), the system achieved a Normalized Mean Square Error (NMSE) of just 0.0309% for displacement prediction. This indicates an extremely close correspondence between predicted and actual displacement values. Furthermore, the velocity prediction at the same DOF exhibited an equally low NMSE of 0.0595%, signifying the framework’s ability to accurately forecast the rate of change in position. These low NMSE values demonstrate the framework’s capacity for precise, high-fidelity predictions, essential for applications demanding accurate modeling of dynamic systems.

Evaluations utilizing filtered white noise excitation at Degree of Freedom 1 demonstrate the framework’s robust predictive capabilities under realistic, noisy conditions. Specifically, the analysis yielded a Normalized Mean Square Error (NMSE) of just 0.3527% for displacement prediction and 0.6849% for velocity. These exceptionally low error rates, achieved even with complex excitation signals, highlight the effectiveness of incorporating estimated Model-Free Element (MFE) parameters into the prediction process and validate the framework’s potential for creating accurate, data-driven simulations of dynamic systems. The consistency of these low NMSE values across various excitation types reinforces the system’s ability to generalize and provide reliable predictions in diverse operational scenarios.

The developed framework transcends mere prediction, laying the foundation for a comprehensive DigitalTwin – a dynamic, virtual replica of the physical system under study. This DigitalTwin isn’t static; it continuously integrates predicted states with real-time data, enabling proactive monitoring of system health and early detection of potential anomalies. Beyond diagnostics, the framework facilitates control strategies, allowing for virtual testing of interventions before implementation in the physical world. By accurately simulating system behavior under varying conditions, this DigitalTwin minimizes risks, optimizes performance, and extends the lifespan of critical infrastructure, offering a powerful tool for engineers and researchers alike.

Accurate long-term predictions of dynamic system behavior hinge on a thorough understanding of nonlinearities often overlooked in traditional modeling. This work highlights the critical influence of factors like nonlinear damping, cubic stiffness, and hysteresis – collectively represented as Model-Free Elements (MFEs) – on predicting future states. These elements account for energy dissipation and restorative forces not captured by linear assumptions, allowing for a more realistic simulation of system response. By incorporating MFEs, the framework moves beyond simple approximations, providing a detailed representation of complex behaviors that significantly improves the reliability of predictions, particularly over extended periods where these nonlinear effects accumulate and become dominant. This nuanced approach is essential for applications demanding precise foresight, such as proactive maintenance, anomaly detection, and robust control strategies.

Model generalization was evaluated using two distinct external force inputs: a periodic sine wave to simulate structured forcing and filtered broadband noise to represent stochastic excitation.

Toward Intelligent Systems: Optimizing Insight

Effective state estimation – accurately determining the current condition of a system – hinges significantly on the deliberate placement of sensors. A thoughtfully designed sensor network doesn’t simply gather data; it actively reduces uncertainty about the system’s true state. Each sensor contributes information, but the value of that information is heavily dependent on its location; poorly placed sensors may offer redundant or irrelevant data, while strategically positioned sensors maximize information gain with minimal overlap. This principle stems from the idea that certain locations provide more ‘leverage’ over the system’s variables, meaning a small measurement change at that point reveals more about the overall system state. Consequently, optimizing sensor placement isn’t merely about coverage; it’s about maximizing the signal-to-noise ratio and creating a network that efficiently distills meaningful insights from complex data, ultimately enhancing the reliability and precision of control and monitoring efforts.

The developed framework provides a robust methodology for pinpointing optimal sensor placement, moving beyond arbitrary distribution to a data-driven approach. By quantifying information gain and actively minimizing uncertainty within the system’s state estimation, the technique enables more efficient monitoring and control strategies. This isn’t simply about collecting more data; it’s about strategically gathering relevant data from the most impactful locations. Consequently, resources can be allocated with greater precision, leading to reduced operational costs and improved system performance – particularly crucial in complex environments where exhaustive monitoring is impractical or impossible. The framework’s ability to prioritize sensor locations promises a significant advancement in how systems are observed and managed, fostering responsiveness and reliability.

The current framework establishes a foundation for significantly more sophisticated applications, with ongoing research directed towards scaling its capabilities to encompass systems of vastly increased complexity. Future iterations will move beyond static sensor placement by integrating adaptive learning strategies, allowing the model to dynamically refine its understanding of the system over time. This involves incorporating real-time data to continuously update the predictive model, effectively “learning” from its predictions and adjusting sensor configurations to maximize information gain. Such an approach promises not only improved prediction accuracy but also the ability to respond effectively to changing system dynamics and unforeseen events, ultimately leading to more robust and intelligent monitoring and control systems.

This framework illustrates the data and information flow between its various stages.

The pursuit of accurate dynamical system prediction, as detailed within this framework, echoes a fundamental tenet of elegant design. This work doesn’t simply add layers of complexity with Bayesian Neural Networks and Kalman filtering; rather, it meticulously removes the extraneous error inherent in misspecified models. The probabilistic digital twin operates on the principle that a precise understanding stems not from encompassing everything, but from distilling the essential information. As Edsger W. Dijkstra stated, “It’s not enough to have a good idea; you must also have the engineering skills to execute it.” This sentiment is palpable throughout the paper, demonstrating how skillful integration of latent force modeling and probabilistic methods leads to a refined and robust system capable of accurate prediction despite inherent model uncertainties.

Where Do We Go From Here?

The pursuit of digital twins, predictably, has spawned a cottage industry of complexity. This work, by returning to first principles – acknowledging, rather than concealing, the inevitable misspecification inherent in any dynamical system – offers a subtle rebuke to that trend. The framework presented isn’t about building a perfect model; it’s about gracefully accounting for imperfection. One suspects the true measure of success will not be in benchmark datasets, but in the field, where real-world systems stubbornly refuse to conform to neat equations.

Future efforts will likely focus on scaling these probabilistic methods. The current formulation, while elegant, will undoubtedly strain under the weight of truly high-dimensional systems. The temptation to introduce more sophisticated – and therefore more opaque – neural network architectures should be resisted. A simpler network, thoroughly understood, will always outperform a black box promising miraculous accuracy. The bottleneck, it seems, isn’t necessarily in the algorithms themselves, but in the painstaking effort required to quantify the inherent uncertainties in the underlying physics.

Perhaps the most fruitful avenue for exploration lies in extending this approach to systems with genuinely stochastic dynamics. Distinguishing between model error and true randomness is a notoriously difficult problem. A framework that can seamlessly integrate both sources of uncertainty – without resorting to ever-more-elaborate parameterizations – would represent a significant step forward. The goal, ultimately, should be not to predict the future with certainty, but to prepare for its inherent unpredictability.

Original article: https://arxiv.org/pdf/2511.22133.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/