Comparing Reliability and Item Difficulty of Multiple-Choice and Translation Questions on a Test of Grammatical Knowledge

Multiple-choice questions are commonly employed for testing language knowledge. However، since in the recently proposed systematic approach to item writing، using various item formats is desirable and there is very few researches on the possibility of using translation items for giving variety to items on a test. The present study aims at comparing reliability and item difficulty of translation and multiple-choice item formats on a test of grammatical knowledge. First، the specifications of the tests were composed of two sections: one with translation items and the other with multiple-choice items، and then reviewed by eight English teachers with some experience in language testing. Then، the specifications were revised according to the feedback. The test for the present study was prepared، and administered to 158 English learners with mixed proficiency levels. The data was analyzed using both classical test theory and Rasch model. The results indicated that both item formats had good reliability (r translation section = 0. 88، r multiple-choice section = 0. 84). Also، generally، the items showed good fit in Rasch model. Furthermore، an independent t-test did not find a significant difference between two formats in terms of item difficulty (t= 1. 696، df= 58، P>0. 95). Therefore، based on the finding of the study، it is suggested that، in a systematic approach to item writing، translation item format can be employed alongside multiple – choice item format for measuring grammatical knowledge.