چكيده به لاتين
Language performance assessment is a notoriously difficult task where test-takers’ score might be influenced by such extraneous factors as rater characteristics. In order to ensure validity issues, it is crucially important to identify these factors and control for their effects. In this respect, the overriding purpose of this study was to investigate whether rater characteristics such as age, gender, language proficiency level, discipline, degree, and teaching experience have an impact on the severity/leniency of the raters in the IELTS writing performance assessments. The contribution of each characteristic to severity/leniency was also investigated. To this aim, 30 raters participated in this study. They were first given a demographic questionnaire and then 20 IELTS task-2 writing samples to rate both holistically and analytically based on four major criteria including task achievement, coherence and cohesion, lexical resource, and grammatical range and accuracy on a six-point Likert scale. The non-parametric chi-square test for independence was used to explore whether the characteristics of raters and their severity/leniency in the IELTS writing performance assessment were related. In addition, the standard multiple regression was utilized to determine which characteristics more predicted raters’ severity/leniency and how much was the contribution of each. Finally, wilcoxon signed rank test was calculated to see if there was significant difference between the holistic and analytic scores awarded by raters. The results revealed that although there was no significant relationship between the raters’ characteristics and their severity/leniency, a significant difference was observed between the holistic and analytic scores. This study recommended test-developers to plan on a rater training program to produce attentive raters who rate more appropriately and efficiently.
