A Comparison between Benchmarking and Bookmarking to Classification of Performance Levels in Large-scale Study of Mathematics Assessment

Standard setting is one of the assessment techniques to create valid classifications of examinees. In present study, the effect of two standard setting methods, benchmark and bookmarking, was examined in results of a large-scale study, which was planned for assessing mathematics learning in sixth grade students of Tehran city.

Two methods were compared using data of a provincial large-scale assessment which carried out on 9720 sixth grade students in Tehran city. They asked 264 mathematics items and their response were analyzed by plausible values. 

Results of applying benchmark showed that 75, 48, 18, and 2 percent of students attained minimum scores in low, mediate, high, and advanced levels; respectively. In addition, 23.9 percent of items located in the same level that identified by content experts. In contrast, quality of classification by content experts in bookmarking was critiqued due to comparing of successive averages with standard deviations of location parameters. Moreover, effect of using five response probabilities: 0.52, .057, 0.62, 0.67, and 0.75 in classification of students indicated that, in spite of recommendation of response probability 0.67 in literature, the lowest response probability (0.52) produced the most realistic results rather than other response probabilities, however, this is still a strictly standard comparing benchmarking methods.

Standard setting should be considered as a technical issue in all assessments that grading or pass/fail is consequent of the test.