Prompt and rater effects in second language writing performance assessment

Keywords

Inter-rater reliability, Psychology, Writing assessment, Linguistics, Mathematics education

References

• Student Characteristics and Essay Test Writing PerformanceSarah Warshauer Freedman — 1983
• Approaches to Language Testing. Advances in Language Testing Series: 2. Papers in Applied Linguistics.Bernard Spolsky — 1978
• Systems Development in Adult Language Learning: The Threshold Level in a European Unit/Credit System for Modern Language Learning by Adults.van Ek — 1975
• The Validity of Using Holistic Scoring to Evaluate Writing: A Critical OverviewDavida Charney — 1984
• Reflections on Research and Assessment in Written CompositionAlan C. Purves — 1992
• The Effects of Systematic Variations in Essay Topics on the Writing Performance of College FreshmenJames Hoetker, Gordon Brossell — 1989
• Essay Examination Topics and Students’ WritingJames Hoetker — 1982
• Syntactic Complexity and Teachers’ Quality Ratings of Narrations and ArgumentsMarion Crowhurst — 1980
• An Experiment with the Wording of Essay TopicsGordon Brossell, Barbara Hoetker Ash — 1984
• Of Ships and Sealing Wax: Drafting and Revising Processes in Grade Twelve Students' Examination Writing.Tom W. Gee — 1985
• Rhetorical Specification in Essay Examination TopicsGordon Brossell — 1983
• On the Viability of Some Untestable Assumptions in Equating Exams That Allow Examinee Choice. Program Statistics Research Technical Report No. 93-31.Xiangbo Wang — 1993
• Native and Nonnative Reactions to ESL CompositionsToshihiko Kobayashi — 1991
• Examining the Invariance of Rater and Project Calibrations Using a Multi-facet Rasch Model.Thomas R. O’Neill, Mary E. Lunz — 1996
• Fossilization in Adult Second Language AcquisitionZhaoHong Han — 2004
• Approaches to language testingBernard Spolsky — 1978
• Designing Writing Tasks for the Assessment of WritingLeo Ruth, Sandra Murphy — 1988
• Fairness and Validation in Language Assessment: Selected Papers from the 19th Language Testing Research Colloquium, Orlando, FloridaAntony John Kunnan — 2000
• Language Testing and Validation: An Evidence-Based ApproachCyril J. Weir — 2004
• Validating holistic scoring for writing assessment : theoretical and empirical foundationsMichael M. Williamson, Brian Huot — 1993

Cited by

• Mapping the Relationships among the Cognitive Complexity of Independent Writing Tasks, L2 Writing Quality, and Complexity, Accuracy and Fluency of L2 WritingWeiwei Yang — 2014
• A validation framework for automated essay scoring systemsLucy Fang Lu — 2012
• Shedding light on the predictive validity of English proficiency tests in predicting academic successLaila Whitman Rumsey — 2013
• An investigation into the effects of topic and background knowledge of topic on second language speaking performance assessment in language proficiency interviewsNahal Khabbazbashi — 2013
• Enhancing rater reliability in the assessment of academic writing: An SFL analysis of raters’ rating and perceptionsGantina Sitepu — 2014
• Double Entropy Inter-Rater Agreement IndicesAndriy Olenko, Vitaliy Tsyganok — 2016
• Topic and background knowledge effects on performance in speaking assessmentNahal Khabbazbashi — 2017
• The performance of Culturally and Linguistically Diverse (CALD) students on a high-stakes writing testJ. Moore — 2016
• Integration of a web-based rating system with an oral proficiency interview test: argument-based approach to validationHyejin Yang — 2016
• Scores Assigned by Inexpert EFL Raters to Different Quality EFL Compositions, and the Raters' Decision-Making Behaviors.Turgay Han — 2017
• The longitudinal stability of rating characteristics in an EFL examination: Methodological and substantive considerationsIasonas Lamprianou, Dina Tsagari, Nansia Kyriakou — 2020
• The Effect of Additional Time on the Quality of Argumentation in L2 Writing Assessment: A Mixed-methods StudySenyung Lee, Gad S. Lim, Rachel Basse — 2021
• Une comparaison de l’étendue intra- et interindividuelle du niveau de sévérité d’examinateurs en français langue étrangèreChristophe Chénier — 2021
• Rater Severity/Leniency and Bias in EFL Students' Composition Using Many-Facet Rasch Measurement (MFRM)Yenni Arif Rahman, Fitri Apriyanti, Rahmi Aulia Nurdini — 2023
• Communal factors in rater severity and consistency over time in high-stakes oral assessmentReeta Neittaanmäki, Iasonas Lamprianou — 2024
• All types of experience are equal, but some are more equal: The effect of different types of experience on rater severity and rater consistencyReeta Neittaanmäki, Iasonas Lamprianou — 2024
• Argument-based validation frameworks in language testing: What are we expecting?Don Dong Yao — 2025
• Understanding Teacher Competence in Multiple-Choice Test Item Writing for English Reading and Listening Skill Tests: A Case of English as a Foreign Language (EFL) Teachers in Vietnamese Higher Education SettingsPhuong Viet Ha Ngo — 2020
• The influence of rater empathy, age and experience on writing performance assessmentPilvi Alp, Anu Epner, Hille Pajupuu — 2017
• The development and maintenance of rating quality in performance writing assessment: A longitudinal study of new and experienced ratersGad S. Lim — 2011

Related papers

• Diagnostic assessment of writing: A comparison of two rating scalesUte Knoch — 2009
• Performance Profiles for Academic Writing.Liz Hamp‐Lyons — 1987
• The Rater Performance Categorization System (RPCS) for Intensive English ProgramsAlper Şahin — 2021
• The Validity of Timed Essay Tests in the Assessment of Writing Skills.Tim Caudery — 1990
• Investigating variability in tasks and rater judgements in a performance test of foreign language speakingLyle F. Bachman, Brian K. Lynch, Maureen Mason — 1995
• For a Greater Good: Bias Analysis in Writing AssessmentMasoumeh Ahmadi Shirazi — 2019
• Rater expertise in a second language speaking assessment : the influence of training and experienceLawrence E. Davis — 2012
• The influence of rater language background on writing performance assessmentJeff S. Johnson, Gad S. Lim — 2009