Subspace Learning: An optimal way to synthesize and analyze large scale dynamical networks

The optimal digital twin.

by Rupert Ullmann and Stefan Sicklinger

Designing large-scale systems that span different physical domains remains one of the most challenging aspects of product design. This challenge becomes even more pronounced when products involve the interplay of software, hardware, and environmental factors.

Over the past five decades, numerous technologies have emerged to tackle this challenge. For analysis, various numerical methods have been developed, including Finite Element Method (FEM), Time Integration Methods for Differential Algebraic Equation Systems (DAEs), and Meshfree methods (such as Smoothed-Particle Hydrodynamics, or SPH). More recent innovations like Physics-informed Neural Networks (PINNs) also aim to solve large-scale partial differential equations (PDEs), although they have yet to gain widespread industry adoption. Despite their diversity, these technologies share a common goal: finding solutions for DAEs and/or PDEs that govern specific physical or logical product behaviors.

Another class of technologies has emerged to not only analyze but also synthesize designs automatically. These include parameter optimization, shape optimization, topology optimization, and combinations thereof, often referred to as generative design.

However, an important aspect has been overlooked: the inherent probabilistic nature of the world we inhabit. The synthesis of products must also account for uncertainty quantification.

Subspace Learning

Subspace Learning aims to combine all three aspects in an optimal manner. This involves extracting only the essential information from large-scale PDEs and DAEs systems while still enabling modification of the system design, which is crucial for accounting for uncertainties. The condensation of information can be achieved through Krylov projection methods, as detailed mathematically in Ullmann (2023). While this article does not delve into the detailed explanation of the method, its focus is on motivating its application in large-scale industrial contexts that span numerous disciplines such as mechanical, acoustical, electrical, thermal, fluidal, and logical domains, as well as state analysis, network design, and many more.

Let us look in some simple examples:

Engineering Example 1: Steady State Dynamics

Consider a simple network of two coupled beams, which has one force input and one displacement output. Each subsystem is a finite element model represented by solid brick elements. As the mesh is rather fine, each of the subsystems has 13164 degrees of unknowns. For more details of the beam modeling refer to [A 3D solid beam benchmark for parametric model order reduction]

Figure 1: The two subsystem network

A forward uncertainty quantification (UQ) of the network output should be evaluated for uncertain dimensional and material parameters of each beam subsystem, resulting in 9 uncertain parameters per subsystem. This results in a massive amount of repetitive system evaluations for varied input parameters. Considering the system as a network but not monolithic is the first remedy, as it allows to parallelize the evaluation of subsystems followed by an interface synchronization, as described in [R. Ullmann 2023: Highly Efficient Energetic Synthesis of Coupled Steady-State Dynamic Systems] . Anyhow, the large subsystem sizes still define a computational bottleneck.

Subspace learning is the second remedy aiming at finding a cheaper-to-evaluate approximation of each subsystem input-to-output behavior. At the same time, it is possible to preserve parameters for variation during UQ. A Krylov subspace approach is considered, in which the approximation to the hypersurface of the input-to-output is iteratively trained in a learning phase, for details refer to [R. Ullmann & S. Sicklinger & G. Müller 2021: Optimization-Based Parametric Model Order Reduction for the Application to the Frequency-Domain Analysis of Complex Systems]. The idea of the approach is illustrated in Figure 2 showing for one dimension, how the approximation is iteratively trained by adding information subsequently at chosen parameter points.

Figure 2: Subspace learning for one input along the frequency dimension

Performing subspace learning for all twelve times twelve nine-dimensional surfaces of the subsystem input-to-output behavior, one can reduce the subsystem size from 13164 to 241 degrees of freedom with relative errors smaller than 0.1%.

This reduction of necessary information allows one to easily compute the forward uncertainty analysis of the whole network as provided in Figure 3.

Figure 3: The network output

Industrial Large-Scale Applications

The two simple examples provided earlier effectively illustrate the problem formulation and objectives. Now, let’s consider a large-scale industrial example from Ullmann (2023), where an acoustic network is synthesized. This network is the outcome of a project step involving a large steady-state dynamic Finite Element Method (FEM) vehicle model.

Figure 4: Interpretation of a complex, large-scale vehicle model as network

The problem formulation is highly non-convex, which requires a globalized optimization utilizing several hundred thousand system evaluations at different parameter configurations. Again, the combination of network representation and subsystem subspace learning is the enabler. It reformulations the monolithic system with millions of unknowns into a network of some dozen parallel-to-evaluate subsystems, each approximated by not even 1000 unknowns. This approach results in the massive speed-up as required and allows for new engineering insights from the globally optimized parametrization of the whole system. Figure 5 shows the new value set for 17 parameters — bushing stiffness values, mass densities and metal sheet thickness values — which reduce the transmission of structure-borne sound to a global minimum.

Figure 5: improved vehicle system design obtained by globalized optimization. The initial setup is marked grey, the optimized one red

Summary

This article aims to showcase the immense potential of the subspace learning method for addressing large, complex system design challenges. The method facilitates highly efficient searches in the design space by generating gradients as a byproduct of problem solutions. When combined with the significant acceleration of problem analysis enabled by the optimality properties of Krylov projection, it becomes feasible to integrate sampling-based and gradient-based optimization algorithms. This integration, often referred to as globalized optimization methods, is pivotal for tackling highly non-convex design problems, which are prevalent across various engineering disciplines.

Highly non-convex design problems manifest in diverse application areas, including:

• Autonomous Systems: Identifying edge-case scenarios.
• Acoustic Design: Achieving optimal designs for silent vehicles.
• Circuit Design: Optimizing robust analog and digital circuit designs.
• … the list continues indefinitely…

Given the probabilistic nature of the universe we inhabit, incorporating uncertainty quantification into the method outlined in Ullmann (2023) is straightforward. This represents a significant advantage for real-world problems, as designs generated by the method inherently possess robustness. Moreover, the mathematical operators generated by the Subspace Learning algorithm exhibit a dense structure, enabling efficient acceleration with hardware commonly used for training and inference in modern deep learning models.

If you have any comments or questions, please feel free to reach out.

References

• R. Ullmann & S. Sicklinger & G. Müller 2021: Optimization-Based Parametric Model Order Reduction for the Application to the Frequency-Domain Analysis of Complex Systems
• R. Ullmann 2023: Highly Efficient Energetic Synthesis of Coupled Steady-State Dynamic Systems
• S. Sicklinger 2014: Stabilized Co-Simulation of Coupled Problems Including Fields and Signals
• Benchmark R. Ullmann 2022: A 3D solid beam benchmark for parametric model order reduction
• Lecture Notes TUM 2024: Design and Partitioning of Dynamic Systems