difficulty index

Multiple-choice questions (MCQs) are a common assessment method, and it is crucial to design them carefully. Therefore, this study aimed to determine the item analysis of MCQ exams in clerkship tests for general medicine students.

Following a cross-sectional study, a total of 1202 MCQs designed for fourth-year clerkship medical students in the second semester of 2019 were analyzed. Difficulty and discrimination indices of student scores and taxonomy levels were then computed. Furthermore, the prepared standard structural Millman checklist was utilized.

Of the 1202 MCQs, according to difficulty indices, most questions (666) were considered acceptable (55.39%). In terms of the discrimination index (DI), 530 (44.09%) questions had an average discrimination coefficient. Additionally, 215 (17.88%) had a negative or poor DI and required revision or elimination from the tests bank. Of the 1202 MCQs, 669 (50.7 %) were designed at a lower cognitive level (taxonomy I), 174 (14.5 %) belonged to taxonomy II, and 419 (34.8%) of the questions had taxonomy III. Moreover, according to the structural flaws of the Millman checklist, the most common structural flaw was a lack of negative choices for Stems 1127 (93.8 %), while vertical options 376 (31.3%) were the least common.

Based on the results, it is recommended that easy questions and negative/poor DI of items, a high level of Bloom’s taxonomy type I, and questions with unstructured flaws be reviewed and reconstructed to improve the quality of the question banks. Holding training courses on designing test questions could effectively improve the quality of the questions.

 Multiple-choice questions (MCQs) are used for measuring the student’s progress, and they should be analyzed properly to guarantee the item’s appropriateness. The analysis usually determines three indices of an item; difficulty or passing index (PI), discrimination index (DI), and distractor efficiency (DE).

 This study was aimed to analyze the multiple-choice questions in the preclinical and clinical examinations with different numbers of options in medical program of Universiti Putra Malaysia.

 This is a cross-sectional study. Forty multiple-choice questions with four options from the preclinical examination and 80 multiple-choice questions with five options from the clinical examination in 2017 and 2018 were analyzed using optical mark recognition machine and Ms. Excel. The parameters included PI, DI, and DE.

 The average difficulty level of multiple-choice questions for preclinical and clinical phase examinations were similar in 2017 and 2018 that were considered ‘acceptable’ and ‘ideal’ ranged from 0.55 to 0.60, respectively. The average DIs were similar in all examinations that were considered ‘good’ (ranged from 0.25 to 0.31) except in 2018 clinical phase examination that showed ‘poor’ items (DI = 0.20 ± 0.11). The questions for preclinical phase showed an increase in the number of ‘excellent’ and ‘good’ items in 2018 from 37.5% to 70.0%. There was an increase of 10.0% for preclinical phase, and 6.25% for clinical phase, in the number of items with no non-functioning distractors in 2018. Among all, preclinical multiple-choice questions in 2018 showed the highest mean of DE (71.67%).

 Our findings suggested that there was an improvement in the questions from preclinical phase while more training on questions preparation and continuous feedback should be given to clinical phase teachers. A higher number of options did not affect the level of difficulty of a question; however, the discrimination power and distractors efficiency might differ.

Clinical reasoning is one of the core features of clinical competency. Training and assessing clinical reasoning is vital in post-graduate training. Extended matching questions(EMQs) are effective in assessing problem-solving and clinical reasoning abilities, but not commonly used in Postgraduate training. Covid-19 pandemic, which prevented both patient encounter and regular academic activities, warranted the introduction of innovative Teaching-Learning methods to sustain clinical reasoning skills. Hence, we aimed to introduce EMQs in formative assessment among anaesthesia residents and analyzed its psychometric properties. 

The study was conducted at the Department of Anaesthesiology, MGMCRI, Pondicherry. Four modules of EMQs as part of a formative assessment was conducted among residents (n=20). A total of 40 clinical vignettes and 60 options were administered. Post-validation of the EMQs was done by item analysis. Test reliability was estimated by the Kuder-Richardson 20 formula. Difficulty index(DIF-I), discrimination index(DIS-I), and distractor functionality were analysed.

The KR-20 reliability coefficient was 0.72. The mean DIF-I was 0.43±0.17, where 72.5%(29) were in the acceptable range, 20%(8) difficult, and 7.5%(3) easy. The mean DIS-I was 0.28±0.24, where 40%(16) had acceptable, 27.5%(11) excellent, and 20% had poor discrimination. Ninety percent of distractors were functional. The DIS-I exhibited a positive correlation with DIF-I (r= 0.2155, P=0.0185).

The results of the present study indicates that EMQs have acceptable test reliability. The majority of the items (80%) followed the principles of MCQs. We conclude that EMQs can be effectively used as part of the postgraduate assessment to test higher-order knowledge and clinical competency.

Multiple-choice questions (MCQs) are the most frequently accepted tool for the evaluation of comprehension, knowledge, and application among medical students. In single best response MCQs (items), a high order of cognition of students can be assessed. It is essential to develop valid and reliable MCQs, as flawed items will interfere with the unbiased assessment. The present paper gives an attempt to discuss the art of framing well-structured items taking kind help from the provided references. This article puts forth a practice for committed medical educators to uplift the skill of forming quality MCQs by enhanced Faculty Development programs (FDPs).

The objective of the study is also to test the quality of MCQs by item analysis.

In this study, 100 MCQs of set I or set II were distributed to 200 MBBS students of Late Shri Lakhiram Agrawal Memorial Govt. Medical College Raigarh (CG) for item analysis for quality MCQs. Set I and Set II were MCQs which were formed by 60 medical faculty before and after FDP, respectively. All MCQs had a single stem with three wrong and one correct answers. The data were entered in Microsoft excel 2016 software to analyze. The difficulty index (Dif I), discrimination index (DI), and distractor efficiency (DE) were the item analysis parameters used to evaluate the impact on adhering to the guidelines for framing MCQs.

The mean calculated difficulty index, discrimination index, and distractor efficiency were 56.54%, 0.26, and 89.93%, respectively. Among 100 items, 14 items were of higher difficulty level (DIF I < 30%), 70 were of moderate category, and 16 items were of easy level (DIF I > 60%). A total of 10 items had very good DI (0.40), 32 had recommended values (0.30 - 0.39), and 25 were acceptable with changes (0.20 - 0.29). Of the 100 MCQs, there were 27 MCQs with DE of 66.66% and 11 MCQs with DE of 33.33%.

In this study, higher cognitive-domain MCQs increased after training, recurrent-type MCQ decreased, and MCQ with item writing flaws reduced, therefore making our results much more statistically significant. We had nine MCQs that satisfied all the criteria of item analysis.