Effect of Test Item Analysis of Summative Exams on Quality of Test Designing

Item analysis is a process in which both test items (questions) and student's answers are examined in order to assess the quality and quantity of the items and test as a whole. The purpose of this study was to investigate the effect of analysis of multiple choice test items of summative exams on quality of the test design by faculty members of Tehran Nursing and Midwifery School. Methods & Materials: A quasi experimental method (pre-test and post-test) without control group was used in this study. After a pilot study, 33 nursing faculty members of school of nursing and midwifery at Tehran University of Medical Sciences were chosen through census sampling. Then one of their exams designed in the second semester (83-84) were chosen to be analyzed. The analysis results were reported to the faculty members. Then their designed tests for the next semester were analyzed again. The analysis was carried out using a checklist which included item structure, whole structure of exam, content validity, and levels of thinking skills reflected in questions and criteria for holding an exam. Moreover, for the quantitative analysis of questions, item difficulty and discrimination index were calculated. Item distracter analysis was examined by calculating the percentage of examinees who selected incorrect alternatives. Integrated t-test, Pearson and Spearman correlation coefficients, and Fisher's exact test were used for the statistical analysis.

1056 questions before presenting the feedback and 803 questions at the end were analyzed and then the results were compared. According to the results, there was a significant difference between before and after intervention in variables item structure (P<0.001), levels of thinking skills (P<0.05), and item distracter analysis (P<0.001).While there was not significant difference between item difficulty, discrimination index, whole structure of exam, content validity, and criteria that should be considered on holding of an exam. Although, Pearson correlation coefficient showed that variables such as age (r=-0.535, P=0.004), and years of services (r=-0.546, P=0.003) with difficulty index were statistically significant.

The results emphasized that item analysis, providing feedback to the faculty members and offering educational booklets to assist them were effective means on improving some qualitative and quantitative items analysis measures.