A Promising Method for Correcting Class Noise in the Presence of Attribute Noise

Noise is a critical concern for practical machine learning, especially medical applications. There exist two kinds of noise, including attributes and class noises. Class noise is potentially more dangerous, so various filtering techniques, particularly prediction-based, have been proposed to control it. Great attention to class noise has made the researchers ignorant that attribute noise, in turn, is harmful. Hence, it is improper to utilize prediction-based filtering to correct class noise without regarding attribute noise.

To tackle this problem, we developed a method to fix class noise in the presence of attribute noise. This method excludes noisy components of attributes, based on the information bottleneck principle, by compressing attributes locally and gradually in successive iterations. It uses heterogeneous ensemble filtering to correct class noise. In the initial iteration, filtering is conservative and progressively, in succeeding iterations, tends to majority vote.

We compared the proposed method's predictive performance with the RF majority-vote filter on three real binary classification problems from the UCI repository, including Breast, Transfusion, and Ionosphere. Random forest, adaptive boosting, support vector machines, and naïve Bayes were used for assessing methods from different viewpoints. Results show that the proposed method performed better than the RF majority-vote filter and seems to open a promising research scope for noise filtering.

Our study revealed that correcting class noise by controlling attribute noise enhances the predictive performance of classifiers.