A Comparative study of the traditional accuracy assessment and disagreement measures of the classification of remote sensing imagery

Evaluation of the image classification results is very important in the remote sensing projects. So far, many indices have been presented to assess the accuracy of image classification, though Kappa coefficient and Overall accuracy are the most famous ones. Some researchers have criticized these two parameters, and have presented new parameters for evaluation of the classification results. In this paper, the relation between two new accuracy assessment parameters (presented by Pontius & Millones) and traditional accuracy assessment parameters (Overall accuracy and kappa coefficient) is studied. These two new parameters are called “Quantity disagreement” and “Allocation disagreement” which report disagreement between ground truth and classification data. In order to apply the comparative study on the traditional and new disagreement measures, supervised maximum likelihood classification was applied on 31 satellite images with different spatial resolutions. Then, Kappa and Overall accuracy as traditional accuracy parameters and Quantity disagreement and Allocation disagreement as new measures were computed for each classified image and then the correlation coefficients of the both measures were calculated. The results show a high correlation between new parameters and traditional ones in negative direction irrespective the spatial resolution. In this way, the disagreement do not provide new information about the classification results to the user, and only if there is any request for classification error, the new disagreement parameters can be used along with the traditional ones.