Please use this identifier to cite or link to this item:
Title: Examination of the inter-rater agreement among faculty marking a research proposal on an undergraduate health course
Authors: Rone-Adams, S
Naylor, S
Keywords: Inter-rater agreement;Double marking;Assessment;Research proposal;Health course
Issue Date: 2009
Publisher: Nova Southeastern University
Citation: The Internet Journal of Allied Health Sciences and Practice, 7(4), 2009
Abstract: Purpose: This research looked at inter-rater agreement among faculty marking a research proposal on an undergraduate health course. The purpose of this study was to investigate if there were significant differences between marks as well as the comments given on research proposals that were double marked by two faculty members, where faculty were blind to the other marker’s marks and comments. Subjects: The subjects for this study consisted of 80 final year health students who submitted a research proposal and 13 full-time and 10 part-time faculty members who double marked the proposals. Methods: Faculty pairs marked the proposals independently and were required to meet with each other and discuss the marks given, reasons for the mark and the comments. Markers were asked to come to an agreement about the mark and the comments. Analyses: Agreement in marks was assessed using a Bland-Altman plot. Weighted Cohen's Kappa was used to estimate the agreement between the classifications given by markers. Comments were analyzed for differences using thematic analysis. Results: There was a wide discrepancy in the classification of students between markers. The weighted proportions found to agree on classification was 46%. Analysis of the comments indicated a wide discrepancy between markers. Conclusion: The outcomes of this study are similar to previous studies that have looked at inter-rater agreement when double marking was used to mark various types of written assignments. Further exploration of the inter-rater agreement in the marking process and other marking processes that results in a transparent system is needed.
Description: Copyright @ 2009 The Internet Journal of Allied Health Sciences and Practice.
ISSN: 1540-580X
Appears in Collections:Physiotherapy
Dept of Clinical Sciences Research Papers

Files in This Item:
File Description SizeFormat 
Fulltext.pdf225.29 kBAdobe PDFView/Open

Items in BURA are protected by copyright, with all rights reserved, unless otherwise indicated.