State-dependent vector hybrid linear and nonlinear ARMA modeling: Theory

A new model is proposed to represent a general vector nonstationary and nonlinear process by setting up a state-dependent vector hybrid linear and nonlinear autoregressive moving average (SVH-ARMA) model. The linear part of the process is represented by a vector ARMA model, the nonlinear part is represented by a vector nonlinear ARMA model employing a multilayer feedforward neural network, and the nonstationary characteristics are captured with a hidden Markov chain. Based on a unified Q-likelihood function, an expectation-maximization algorithm for model identification is derived, and the model parameters are estimated by applying a state-dependent training and nonlinear optimization technique iteratively, which finally yields maximum likelihood estimation of model parameters. This model can track the nonstationary varying of a vector linear and/or nonlinear process adaptively and represent a vector linear and/or nonlinear system with low order. Moreover, it is able to characterize and track the long-range, second-order correlation features of many time series and thus can be used for reliable multiple step ahead prediction. Some impressive applications of the SVH-ARMA model are being presented in the companion paper by Zheng et al., pp. 575-597, this issue.