instance selection

 In the era of big data, automatic data analysis techniques such as data mining have been widely used for decision-making and have become very effective. Among data mining techniques, classification is a common method for decision making and prediction. Classification algorithms usually work well on balanced datasets. However, one of the challenges of the classification algorithms is how to correctly predicting the label of new samples based on learning on imbalanced datasets. In this type of dataset, the heterogeneous distribution of the data in different classes causes examples of the minority class to be ignored in the learning process, while this class is more important in some prediction problems. To deal with this issue, in this paper, an efficient method for balancing the imbalanced dataset is presented, which improves the accuracy of the machine learning algorithms to correct prediction of the class label of new samples. According to the evaluations, the proposed method has a better performance compared to other methods based on two common criteria in evaluating the classification of imbalanced datasets, namely "Balanced Accuracy" and "Specificity".

Linear discriminant analysis is a well-known matrix-based dimensionality reduction method. It is a supervised feature extraction method used in two-class classification problems. However, it is incapable of dealing with data in which classes have unequal covariance matrices. Taking this issue, the Chernoff distance is an appropriate criterion to measure distances between distributions. In the proposed method, for data classification, LDA is used to extract most discriminative features but instead of its Fisher criterion, the Chernoff distance is employed to preserve the discriminatory information for the several classes with heteroscedastic data. However, the Chernoff distance cannot handle the situations where the component means of distributions are close and leads to the component distribution overlap and underperforming classification. To overcome this issue, the proposed method designs an instance selection method that provides the appropriate covariance matrices. Aiming to improve LDA-based feature selection, the proposed method includes two phases: (1) it removes non-border instances and keeps border ones by introducing a maximum margin sampling method. The basic idea of this phase is based on keeping the hyperplane that separates a two-class data and provides large margin separation. In this way, the most representative instances are selected. (2) It extracts features on selected instances by the proposed extension of LDA which generates a desirable scatter matrix to increase the efficiency of LDA. In the proposed method, the instance selection process is considered a constrained binary optimization problem with two contradicting objects, and the problem solutions are obtained by using a heuristic method named filled function. This optimization method does not easily get stuck in local minima; meanwhile, it is not affected by improper initial points. The performance of the proposed method on data collected from the UCI database is evaluated by 10-fold validation. The results of experiments are compared to several competing methods, which show the superiority of the proposed method in terms of classification accuracy percentage and computational time.

Increasing the use of Internet and some phenomena such as sensor networks has led to an unnecessary increasing the volume of information. Though it has many benefits, it causes problems such as storage space requirements and better processors, as well as data refinement to remove unnecessary data. Data reduction methods provide ways to select useful data from a large amount of duplicate, incomplete and redundant data. These methods are often applied in the pre-processing phase of machine learning algorithms. Three types of data reduction methods can be applied to data: 1. Feature reduction.2. Instance reduction: 3. Discretizing feature values. In this paper, a new algorithm, based on ReliefF, is introduced to decrease both instances and features. The proposed algorithm can run on nominal and numeric features and on data sets with missing values. In addition, in this algorithm, the selection of instances from each class is proportional to the prior probability of classes. The proposed algorithm can run parallel on a multi-core CPU, which decreases the runtime significantly and has the ability to run on big data sets. One type of instance reduction is instance selection. There are many issues in designing instance selection algorithms such as representing the reduced set, how to make a subset of instances, choosing distance function, evaluating designed reduction algorithm, the size of reduced data set and determining the critical and border instances. There are three ways of creating a subset of instances. 1) Incremental. 2) Decremental. 3) Batch. In this paper, we use the batch way for selecting instances. Another important issue is measuring the similarity of instances by a distance function. We use Jaccard index and Manhattan distance for measuring. Also, the decision on how many and what kind of instances should be removed and which must remain is another important issue. The goal of this paper is reducing the size of the stored set of instances while maintaining the quality of dataset. So, we remove very similar and non-border instances in terms of the specified reduction rate. The other type of data reduction that is performed in our algorithm is feature selection. Feature selection methods divide into three categories: wrapper methods, filter methods, and hybrid methods. Many feature selection algorithms are introduced. According to many parameters, these algorithms are divided into different categories; For example, based on the search type for the optimal subset of the features, they can be categorized into three categories: Exponential Search, Sequential Search, and Random Search. Also, an assessment of a feature or a subset of features is done to measure its usefulness and relevance by the evaluation measures that are categorized into various metrics such as distance, accuracy, consistency, information, etc. ReliefF is a feature selection algorithm used for calculating a weight for each feature and ranking features. But this paper is used ReliefF for ranking instances and features. This algorithm works as follows: First, the nearest neighbors of each instances are found. Then, based on the evaluation function, for each instance and feature, a weight is calculated, and eventually, the features and instances that are more weighed are retained and the rest are eliminated. IFSB-ReliefF (Instance and Feature Selection Based on ReliefF) algorithm is tested on two datasets and then C4.5 algorithm classifies the reduced data. Finally, the obtained results from the classification of reduced data sets are compared with the results of some instance and feature selection algorithms that are run separately.

Microarray datasets have an important role in identification and classification of the cancer tissues. In cancer researches, having a few samples of microarrays in cancer researches is one of the most concerns which lead to some problems in designing the classifiers. Moreover, due to the large number of features in microarrays, feature selection and classification are even more challenging for such datasets. Not all of these numerous features contribute to the classification task, and some even impede performance. Hence, appropriate gene selection method can significantly improve the performance of cancer classification. In this paper, a modified multi-objective cuckoo search algorithm is used to feature selection and sample selection to find the best available solutions. For accelerating the optimization process and preventing local optimum trapping, new heuristic approaches are included to the original algorithm. The proposed algorithm is applied on six cancer datasets and its results are compared with other existing methods. The results show that the proposed method has higher accuracy and validity in comparison to other existing approaches and is able to select the small subset of informative genes in order to increase the classification accuracy.