Replacement of Missing Values and its Effect on the Classification Accuracy in Medical Data Mining

The missing values in medical data may impact the data mining process and any kind of interpretation. Thus the treatment of these missing values is a necessary task. In this research, the effect of various methods of dealing with missing values on medical data classification accuracy is evaluated. This paper studied the effect of missing data replacement methods including Mean/Mode, Hot Deck, K-Nearest Neighbor, Maximum Possible Value, All Possible Value, Case Deletion, and Regression on classification accuracy for two popular classifiers namely K-nearest-neighbor and Neural Networks from Weka Data mining tool on 10 medical datasets including Breast Cancer, Cardiac Problems, Dermatology, Hepatitis, Thyroid, Diabetes, Primary Tumor, Liver Patient, Lung Cancer and Post-Operative Patient. These were selected from the six amounts of missing values. For classification accuracy estimation, the 10-fold cross validation method is used. The results show that although the mean/mode method almost had better classification improvement that, none of the replacement methods for all amounts of missing values, is not always the most accurate classification with increasing amounts of missing values for the K-nearest-neighbor classifier. There was no supremacy for all the replacement methods against the various amounts of missing values for any of the replacement methods for all data sets with different amounts of missing values. The current study shows that the replacement methods that have been evaluated for all the different rates of missing values do not necessarily improve the accuracy of classification and none of the investigated replacement methods is not absolutely the best one.