Semi-supervised Sparse Feature Selection based on Hessian Regularization and Fisher Discriminant Analysis

Feature selection is one of the most important techniques in machine learning and pattern recognition, which eliminates redudant features and selects a suitable subset of features. This avoids overfitting when building the model and improves the model performance. In many applications, obtaining labeled data is costly and time consuming, while unlabeled data are readily available. Therefore, semi-supervised feature selection methods can be used to consider both labeled and unlabeled data in the feature selection process. In this paper, a semi-supervised sparse feature selection method is proposed based on hessian regularization and Fisher discriminant analysis which selects the appropriate features using the labeled data and the local structure of both labeled and unlabeled data. In the proposed method, an objective function based on semi-supervised scatter matrix and l2,1-norm is presented for feature selection which considers the correlation among features. To solve the proposed objective function, an iterative algorithm is used and its convergence is experimentally and theoretically proved. The results of the experiments on five data sets indicate that the proposed method improves the selection of relevant features compared to other methods used in this paper.