Exploiting Sparsity In Parameter Estimation And Input Design

QC 20240828

Abstract

Sparse parameter estimation is a key aspect of system identification, as it allows for reducing a model's order and because some models inherently exhibit sparsity in their parameters. The accuracy of an estimated sparse model depends directly on the performance of the sparse estimation method. It is well-known that the accuracy of a sparse estimation method is influenced by the correlations between the regressors of the model being estimated. Mutual coherence represents the maximum of these correlations, and when the parameter vector is sparse, accurate estimation requires low mutual coherence.

However, in system identification, a major challenge arises from constructing the regressor based on time series data, which often leads to high mutual coherence. This high coherence hinders accurate sparse estimation. To address this issue, the first part of this thesis introduces novel methods that reduce mutual coherence through linear coordinate transformations. These methods can be integrated with any sparse estimation technique. Our numerical studies show significant improvements in performance compared to state-of-the-art sparse estimation algorithms.

In the second part of the thesis, the focus shifts to optimal input design for linear models in system identification, which aims to achieve maximum model accuracy based on specific criteria. Traditional linear model input design techniques lack coherence constraints between the input sequences, often resulting in high mutual coherence and, consequently, increased sparse estimation errors for sparse linear models. Therefore, the second part of the thesis concentrates on designing optimal inputs for sparse linear models. We formulate the proposed methods and develop numerical algorithms using alternating minimization. Additionally, we compare the performance of our proposed methods with state-of-the-art input design algorithms and provide theoretical analysis of the proposed methods in both parts of the thesis.

In the third part of the thesis, which consists of two sub-parts, we focus on input design for nonlinear models. Firstly, we address the computational complexity required in nonlinear model input design, which involves solving an optimization problem with a large number of free variables, resulting in high computational complexity. We demonstrate that most of these free variables do not impact the optimization problem because they are zero. We then propose a new method to estimate the positions of the non-zero free variables among all variables, significantly reducing computational complexity. This proposed method is evaluated both theoretically and numerically compared to state-of-the-art input design techniques for nonlinear models.Secondly, many nonlinear models, such as Nonlinear Auto Regressive eXogenous (NARX) and Volterra series models, exhibit sparsity in their parameters, meaning mutual coherence directly affects the accuracy of these model estimations. Due to the lack of constraints on the correlation between input sequences in nonlinear model input design, the resulting optimal regressor often exhibits high mutual coherence. To address this challenge, we propose a new method for nonlinear model input design that promotes sparse model estimation. This method aligns the input design objectives with those of standard input design, effectively reducing mutual coherence and improving estimation accuracy. Additionally, we theoretically analyze the cost of adding the coherence constraint to the optimal input design problem and compare the numerical performance of the proposed method to state-of-the-art nonlinear model input design methods.

The last part delves into data informativity in system identification, which determines if the collected data is sufficient for accurate model building. Specifically, this part focuses on the effect of external excitations on data informativity. We propose a method to find the minimum number of external excitations required to uniquely identify linear combinations of elements in transfer function models. This method involves formulating an optimization problem with binary decision variables and applying unitary transformations to reduce excitations. Numerical evaluations demonstrate the proposed method's effectiveness compared to state-of-the-art algorithms, highlighting its potential to minimize the number of excitation sources for data informativity.

urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-352238