An Analytical Model for Predicting the Convergence Behavior of the Least Mean Mixed-Norm (LMMN) Algorithm

Stochastic gradient-based adaptation algorithms have received a great attention in various applications. The most well-known algorithm in this category is the Least Mean Squares (LMS) algorithm that tries to minimize the second-order criterion of mean squares of the error signal. On the other hand, it has been shown that higher-order adaptive filtering algorithms based on higher-order statistics can perform better in many applications, particularly in the presence of intense noises. However, these algorithms are more prone to instability and also their convergence rates decline in the vicinity of their optimum solutions. In attempt to make use of the useful aspects of these algorithms, it has been proposed to combine the second-order criterion with higher-order ones, e.g. that of the Least Mean Fourth (LMF) algorithm. The Least Mean Mixed-Norm (LMMN) algorithm is a stochastic gradient-based algorithm which aim is to minimize an affine combination of the cost functions of the LMS and LMF algorithms. This algorithm has inherited many properties and advantages of the LMS and the LMF algorithms and mitigated their weaknesses in some ways. These advantages are achieved at the cost of the additional computation burden of just one addition and four multiplications per iteration. The main issue of the LMMN algorithm is the lack of an analytical model for predicting its behaviour, the fact that has restricted its practical application. To address this issue, an analytical model is presented in the current paper that is able to predict the mean-square-error and the mean-weights-error behaviour with a high accuracy. This model is derived using the Isserlis’ theorem, based on two mild and practically valid assumptions; namely the input signal is stationary, zero-mean Gaussian and the measurement noise are additive zero-mean with an even probability distribution function (pdf). The accuracy of the derived model is verified using several simulation tests. These results show that the model is of a high accuracy in various settings for the noise’s power level and distribution as well as the unknown filter characteristics. Furthermore, since the LMF and the LMS algorithms are special cases of the more general LMMN algorithm, the proposed model can also be used for predicting the behaviour of these algorithms.