The capacity of ChatGPT-4 for L2 writing assessment: A closer look at accuracy, specificity, and relevance

Saricaoglu A., Bilki Z.

Annual Review of Applied Linguistics, cilt.45, ss.253-273, 2025 (AHCI, SSCI, Scopus) identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 45
  • Basım Tarihi: 2025
  • Doi Numarası: 10.1017/s0267190525100160
  • Dergi Adı: Annual Review of Applied Linguistics
  • Derginin Tarandığı İndeksler: Arts and Humanities Citation Index (AHCI), Social Sciences Citation Index (SSCI), Scopus, Periodicals Index Online, Communication & Mass Media Index, EBSCO Education Source, Linguistic Bibliography, Linguistics & Language Behavior Abstracts, MLA - Modern Language Association Database, Psycinfo
  • Sayfa Sayıları: ss.253-273
  • Anahtar Kelimeler: accuracy, ChatGPT-4, L2 writing assessment, relevance, specificity
  • TED Üniversitesi Adresli: Evet

Özet

This study examined the capacity of ChatGPT-4 to assess L2 writing in an accurate, specific, and relevant way. Based on 35 argumentative essays written by upper-intermediate L2 writers in higher education, we evaluated ChatGPT-4’s assessment capacity across four L2 writing dimensions: (1) Task Response, (2) Coherence and Cohesion, (3) Lexical Resource, and (4) Grammatical Range and Accuracy. The main findings were (a) ChatGPT-4 was exceptionally accurate in identifying the issues across the four dimensions; (b) ChatGPT-4 demonstrated more variability in feedback specificity, with more specific feedback in Grammatical Range and Accuracy and Lexical Resource, but more general feedback in Task Response and Coherence and Cohesion; and (c) ChatGPT-4’s feedback was highly relevant to the criteria in the Task Response and Coherence and Cohesion dimensions, but it occasionally misclassified errors in the Grammatical Range and Accuracy and Lexical Resource dimensions. Our findings contribute to a better understanding of ChatGPT-4 as an assessment tool, informing future research and practical applications in L2 writing assessment.