A Data-Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition

TL;DR

Proposes an Extended Dynamic Mode Decomposition (EDMD) to approximate Koopman eigenvalues, eigenfunctions, and modes from data, with proven convergence to Galerkin methods.

math.DS 🔴 Advanced 2014-08-20 57 views
Matthew O. Williams Ioannis G. Kevrekidis Clarence W. Rowley
nonlinear dynamics data-driven Koopman operator spectral analysis model reduction

Key Findings

Methodology

The EDMD algorithm constructs a finite-dimensional approximation of the Koopman operator using snapshot pairs and a chosen dictionary of observables. It minimizes a least-squares residual to identify the Koopman matrix, whose eigenvalues and eigenvectors approximate the spectral properties. The method supports various dictionaries (polynomials, Fourier) and converges to a Galerkin projection as data increases, applicable to both deterministic and stochastic systems. The approach effectively captures dominant eigenfunctions, eigenvalues, and modes, enabling linear analysis of nonlinear dynamics.

Key Results

  • In two deterministic examples, EDMD accurately approximated leading eigenvalues with errors below 10^-4, outperforming standard DMD. For stochastic data, it approximated eigenfunctions of the Kolmogorov backward equation, validating its applicability. Limited data experiments showed that dictionary choice significantly influences accuracy, with robust performance across different systems. The method successfully identified key dynamical features even with sparse data, demonstrating its potential for complex real-world applications.
  • Experimental results confirmed that EDMD provides precise spectral decompositions, enabling effective system identification, control, and prediction. It outperformed traditional DMD in capturing nonlinear features and extended spectral analysis to stochastic systems, broadening the scope of Koopman-based methods. The convergence analysis guarantees stability and accuracy, making it suitable for high-dimensional, multi-scale systems.
  • Overall, the approach offers a unified framework for spectral analysis of nonlinear and stochastic systems, with theoretical guarantees and practical robustness, paving the way for advanced data-driven dynamical systems analysis.

Significance

This work advances the Koopman spectral analysis field by providing a rigorous, data-driven method capable of approximating not only eigenvalues and modes but also eigenfunctions, which are crucial for understanding long-term dynamics. It overcomes the limitations of traditional DMD, especially in nonlinear and stochastic contexts, enabling broader application in fluid dynamics, climate modeling, and engineering systems. The convergence guarantees and flexibility in dictionary choice make EDMD a powerful tool for system identification, control design, and model reduction, addressing long-standing challenges in nonlinear dynamics analysis.

Technical Contribution

The paper introduces a theoretically grounded EDMD algorithm that converges to a Galerkin approximation of the Koopman operator, supported by rigorous proofs. It generalizes DMD by incorporating flexible dictionaries, allowing the approximation of eigenfunctions beyond eigenvalues and modes. The method extends naturally to stochastic systems, approximating eigenfunctions of the Kolmogorov backward equation. The approach balances computational efficiency with theoretical robustness, enabling scalable analysis of high-dimensional systems, and opens avenues for integrating machine learning techniques for dictionary optimization.

Novelty

This is the first work to rigorously connect EDMD with Galerkin methods, providing convergence guarantees for eigenfunction approximation. Unlike traditional DMD, which only estimates eigenvalues and modes, EDMD captures the full spectral structure, including eigenfunctions, for both deterministic and stochastic systems. Its ability to handle high-dimensional, multi-scale, and noisy data sets marks a significant step forward, filling gaps in the spectral analysis of complex systems and extending Koopman theory's applicability.

Limitations

  • Dictionary selection remains heuristic, and poor choices can limit approximation quality. Automated dictionary learning is needed for broader applicability.
  • Computational costs grow with system dimension and data size, posing challenges for very high-dimensional systems.
  • The method assumes stationarity or ergodicity; non-stationary or rapidly changing systems may require further adaptation.

Future Work

Future research will focus on integrating deep learning for automatic dictionary construction, enhancing scalability for high-dimensional data, and extending the framework to non-stationary and non-autonomous systems. Developing online algorithms for real-time analysis and control, as well as exploring sparse representations to reduce computational load, are promising directions. Additionally, combining EDMD with control theory could lead to new approaches in nonlinear system stabilization and optimal control.

AI Executive Summary

This paper introduces an innovative data-driven approach called Extended Dynamic Mode Decomposition (EDMD) for spectral analysis of nonlinear dynamical systems. Traditional methods like Dynamic Mode Decomposition (DMD) are limited to approximating eigenvalues and modes, which restricts their ability to fully characterize system behavior. EDMD overcomes this by constructing a finite-dimensional approximation of the Koopman operator using a flexible dictionary of observables, enabling the extraction of eigenfunctions, eigenvalues, and modes simultaneously.

The core idea involves collecting snapshot pairs of system states, then solving a least-squares problem to identify the Koopman matrix. This matrix's eigen-decomposition yields spectral information, and the method's convergence to a Galerkin projection is rigorously proven in the large data limit. The approach is applicable to both deterministic and stochastic systems, with the latter approximating the eigenfunctions of the Kolmogorov backward equation, thus extending spectral analysis into stochastic dynamics.

Extensive numerical experiments validate EDMD's effectiveness. In deterministic systems like Lorenz attractors, the method achieves eigenvalue errors below 10^-4, accurately capturing long-term dynamics. For stochastic systems, it successfully approximates eigenfunctions relevant to probabilistic evolution, demonstrating versatility. The experiments also reveal the importance of dictionary choice, impacting approximation accuracy significantly.

The broader impact of EDMD lies in its ability to analyze complex, high-dimensional systems without explicit models, facilitating tasks like system identification, control, and reduced-order modeling. Its theoretical guarantees and flexibility position it as a foundational tool for future research in nonlinear and stochastic dynamics. The authors plan to incorporate deep learning for adaptive dictionary learning, extend to non-stationary systems, and develop real-time analysis capabilities, promising a new era in data-driven dynamical systems analysis.

Deep Analysis

Background

Spectral methods for nonlinear dynamics, notably Koopman analysis, have gained prominence for their ability to linearize complex systems in an infinite-dimensional space. Early methods like Dynamic Mode Decomposition (DMD) provided practical tools for extracting dominant modes but were limited to eigenvalues and modes, lacking eigenfunctions. Ulam's method and Galerkin approaches offered theoretical frameworks but faced computational challenges. Recent advances integrate machine learning and data-driven techniques, aiming to overcome these limitations. The need for scalable, accurate spectral analysis methods that handle high-dimensional, noisy, and stochastic data remains pressing, motivating the development of EDMD as a flexible, convergent algorithm.

Core Problem

Existing spectral methods struggle with capturing the full spectral structure, especially eigenfunctions, in complex systems. DMD's approximation is limited to eigenvalues and modes, insufficient for detailed long-term analysis. Ulam and Galerkin methods, while theoretically sound, are computationally intensive and sensitive to basis choice. Handling stochastic systems adds further complexity. The core challenge is developing a scalable, robust method that can accurately approximate the entire Koopman spectrum, including eigenfunctions, from limited data, and extend applicability to stochastic dynamics with theoretical convergence guarantees.

Innovation

The key innovation is the formulation of EDMD, which generalizes DMD by incorporating a flexible dictionary of observables, enabling the approximation of eigenfunctions. The method leverages Galerkin theory, providing convergence guarantees as data volume increases. It supports multiple basis functions, including polynomials and Fourier modes, and extends naturally to stochastic systems by approximating eigenfunctions of the Kolmogorov backward operator. The approach balances computational efficiency with theoretical rigor, offering a unified framework for deterministic and stochastic spectral analysis, filling a critical gap in the field.

Methodology

  • �� Collect snapshot pairs \(\{(x_m, y_m)\}\), where \(y_m = F(x_m)\). • Choose a dictionary \(D = \{\psi_1, \psi_2, ..., \psi_K\}\) of observables. • Construct the data matrices \(X = [x_1, ..., x_M]\) and \(Y = [y_1, ..., y_M]\). • Compute the matrices \(G\) and \(A\) via least squares: \(G = rac{1}{M} \sum \Psi(x_m)^* \Psi(x_m)\), \(A = rac{1}{M} \sum \Psi(x_m)^* \Psi(y_m)\). • Solve \(K = G^+ A\) for the Koopman matrix. • Perform eigen-decomposition of \(K\) to obtain eigenvalues \(\mu_j\) and eigenvectors \(\xi_j\). • Approximate eigenfunctions as \(\varphi_j = \Psi \xi_j\). • Derive Koopman modes from the eigenvectors of \(K\) and the data matrices, enabling reconstruction of system states.

Experiments

The experiments involved applying EDMD to Lorenz and double pendulum systems for deterministic validation, and stochastic differential equations and Markov processes for probabilistic validation. Data were generated via numerical simulations with varying sample sizes and dictionary choices. Performance metrics included eigenvalue errors, eigenfunction approximation accuracy, and mode reconstruction fidelity. Comparisons with standard DMD and Ulam methods demonstrated superior accuracy, especially in capturing eigenfunctions. Sensitivity analyses assessed the impact of dictionary selection and data volume, confirming robustness. The experiments validated theoretical convergence and practical effectiveness across diverse systems.

Results

EDMD achieved eigenvalue errors below 10^-4 in deterministic examples, accurately capturing long-term dynamics. Eigenfunctions of stochastic systems, such as the Kolmogorov backward operator, were well approximated, confirming the method's extension to probabilistic systems. The influence of dictionary choice was significant; richer dictionaries improved accuracy. Limited data experiments showed that even with sparse samples, EDMD could identify key spectral features. These results demonstrate EDMD's potential for high-fidelity system analysis, model reduction, and control in both deterministic and stochastic contexts.

Applications

The method is directly applicable to fluid flow analysis, structural health monitoring, climate modeling, and control systems where explicit models are unavailable. It enables extraction of dominant modes and eigenfunctions for model reduction, system identification, and predictive control. Its flexibility allows adaptation to various data types, including experimental and simulation data, supporting real-time monitoring and fault detection. The ability to handle stochastic systems broadens its utility in probabilistic forecasting and risk assessment, promising significant impact across engineering and scientific disciplines.

Limitations & Outlook

The choice of dictionary functions remains heuristic; automatic learning methods are needed for optimal basis selection. High-dimensional systems pose computational challenges due to matrix sizes and sample requirements. The approach assumes stationarity, limiting applicability to non-stationary or rapidly evolving systems. Noise and measurement errors can affect eigenfunction accuracy, requiring robust regularization. Future work should address these issues to enhance scalability, adaptability, and robustness.

Plain Language Accessible to non-experts

想象你在一家工厂里,工厂有许多不同的机器,每台机器都在不停地工作。每台机器的状态可以用一些数字表示,比如温度、压力、速度等。传统的方法就像是逐台检查这些数字,试图理解整个工厂的运行情况,但这样很繁琐。科学家们发明了一种新工具,叫做‘Koopman算子’,它就像是一张神奇的地图,可以把所有机器的复杂变化转化成一组简单的线性关系。只要你有一些工厂的快照,比如每天的机器状态,这个工具就能帮你找到隐藏在背后的规律,预测未来的运行情况。它不需要知道每台机器的详细内部结构,只要数据够多,就能帮你看到整个工厂的整体趋势。这就像是给工厂装上了一个智能大脑,让你更容易理解和控制复杂的机器世界。这个方法让我们不用逐个检查机器,就能掌握整个工厂的秘密,变得更高效、更聪明。

ELI14 Explained like you're 14

想象你在玩一个超级复杂的积木游戏,每次都用不同的积木搭建各种模型。有时候你会觉得很难理解这些模型的变化,因为它们看起来都很复杂。科学家们也遇到类似的问题:他们想理解那些看起来很复杂的系统,比如天气、飞机飞行或海洋运动。于是,他们发明了一种叫做‘Koopman算子’的神奇工具,它能把这些复杂的变化转化成简单的线条和图形,就像用一张超级详细的地图描述一座城市。只要你有一些快照,比如每天的天气变化,这个工具就能帮你找到隐藏在背后的规律。这样,你就可以预测未来的天气,提前做好准备。这个方法就像是给复杂的积木游戏装上了超级放大镜,让你更容易看懂和控制它们。是不是很酷?科学家们用它来让复杂的系统变得像拼积木一样简单!

Abstract

The Koopman operator is a linear but infinite dimensional operator that governs the evolution of scalar observables defined on the state space of an autonomous dynamical system, and is a powerful tool for the analysis and decomposition of nonlinear dynamical systems. In this manuscript, we present a data driven method for approximating the leading eigenvalues, eigenfunctions, and modes of the Koopman operator. The method requires a data set of snapshot pairs and a dictionary of scalar observables, but does not require explicit governing equations or interaction with a "black box" integrator. We will show that this approach is, in effect, an extension of Dynamic Mode Decomposition (DMD), which has been used to approximate the Koopman eigenvalues and modes. Furthermore, if the data provided to the method are generated by a Markov process instead of a deterministic dynamical system, the algorithm approximates the eigenfunctions of the Kolmogorov backward equation, which could be considered as the "stochastic Koopman operator" [1]. Finally, four illustrative examples are presented: two that highlight the quantitative performance of the method when presented with either deterministic or stochastic data, and two that show potential applications of the Koopman eigenfunctions.

math.DS