Validity and reliability of a checklist for critical appraisal of the original research abstracts by student peer reviewers in medical student research congresses

Abstract Nowadays، it is of utmost importance to critically appraise the research evidences presented in scientific congresses by the audiences. In addition to improvement in scientific and practical skills of critical appraisal، it is important to use a standard framework as the major tool for peer reviewing. We aimed to assess the validity and reliability of one proposed checklist for critical appraisal of the original research abstracts by student peer reviewers. Methods and Materials: This study was a part of an educational interventional project that was performed in a setting of a workshop where 40 medical students from the medical faculties of the universities in Tehran were recruited. Participants were selected using a non-probability purposive sampling method. Educational curriculum of the workshop included clarifying explanations on the 31-item checklist for peer reviewing of the abstracts as well as several tips about each item using lectures، simulations and group discussions during 10 hours. Medical students used the checklist twice، at beginning and the end of the workshop to score three sample abstracts. Data were collected and analyzed using Spearman correlation (internal consistency) and Cronbach’s’ alpha methods to calculate the reliability of different items and domains of the introduced peer reviewing checklist by SPSS software. Moreover، Delphi method was applied to confirm the validity of the instrument by experts’ opinion. A group of experts finally confirmed the validity of this checklist by means of Delphi method. Moreover، internal consistency of the main domains of the checklist consisting of “Introduction”، “Methods”، “Results” and “Conclusion” were statistically significant (P<0. 05). The coefficient of Cronbach’s alpha was calculated as excellent (0. 926) in “Methods”، and acceptable in two other domains of “Results” (0. 650) and “Conclusion” (0. 739) in pre-test data. The introduced checklist was shown to be a structured، valid and reliable tool، which could be used in research congresses to improve the quality of peer reviewing.