Randomness among Teacher Raters Rating Language Learner's Essays: A FACETS Approach

Inconsistency of the ratings of raters may invalidate test results, adversely affecting the decisions made about the placement of language learners to a higher level of education. In the present study, the researcher used the many-facet Rasch measurement model to examine how consistently teacher raters rated the essays written by language learners in their writing classes at Imam Khomeini International University. The teacher raters each rated 56 essays, using a researcher-made, 5-point analytic rating scale. Using FACETS, the Rasch-based computer programme for rating data, the researcher analysed the data. The results of FACETS analysis, including separation indices and fit values, showed that teacher raters were self-consistent in rating the essays language learners wrote. The results of single rater-rest of the raters revealed that each teacher rater’s ratings were consistent with those of other raters. These findings may carry implications for research and pedagogy, shedding light on rater training.