Validation of Writing Scales for Turkish as a Second Language through Many-Facet Rasch Measurement

Fatma Küçük Üçpınar , Aylin Ünaldı

Research output: Contribution to journalArticle

Abstract

Rating scales and the extent to which raters use them effectively are two important factors that influence scoring validity of language tests when open-ended writing tasks are concerned. Research regarding rating scale development and validation in the assessment of English is ample; however, there has been no research on scale validation in the assessment of Turkish as a Second Language (TSL) to this date. This study reports on the development of two analytical rating scales used to assess academic writing skills of test takers in TSL, and presents quantitative evidence on the rating scale validation. For this purpose, texts written by 39 TSL students were scored by three raters. The analyses were conducted using Many-facet Rasch Measurement. Results indicate that empirically-developed analytical rating scales were used consistently and appropriately by the raters providing evidence for the reliability and effectiveness of the rating scales.
Original languageEnglish
Article number627600
Pages (from-to)23-48
Number of pages26
JournalBogazici University Journal of Education
Volume34
Issue number1
Publication statusPublished - 17 Dec 2018
Externally publishedYes

Fingerprint

rating scale
language
evidence
student

Cite this

@article{d8263bd4f270405ea4bf6c30c82f6359,
title = "Validation of Writing Scales for Turkish as a Second Language through Many-Facet Rasch Measurement",
abstract = "Rating scales and the extent to which raters use them effectively are two important factors that influence scoring validity of language tests when open-ended writing tasks are concerned. Research regarding rating scale development and validation in the assessment of English is ample; however, there has been no research on scale validation in the assessment of Turkish as a Second Language (TSL) to this date. This study reports on the development of two analytical rating scales used to assess academic writing skills of test takers in TSL, and presents quantitative evidence on the rating scale validation. For this purpose, texts written by 39 TSL students were scored by three raters. The analyses were conducted using Many-facet Rasch Measurement. Results indicate that empirically-developed analytical rating scales were used consistently and appropriately by the raters providing evidence for the reliability and effectiveness of the rating scales.",
keywords = "Scale development, writing assessment of Turkish as a second language, many-facet Rasch measurement",
author = "{K{\"u}{\cc}{\"u}k {\"U}{\cc}pınar}, Fatma and Aylin {\"U}naldı",
year = "2018",
month = "12",
day = "17",
language = "English",
volume = "34",
pages = "23--48",
journal = "Bogazici University Journal of Education",
issn = "1300-9567",
number = "1",

}

Validation of Writing Scales for Turkish as a Second Language through Many-Facet Rasch Measurement. / Küçük Üçpınar , Fatma; Ünaldı , Aylin.

In: Bogazici University Journal of Education, Vol. 34, No. 1, 627600, 17.12.2018, p. 23-48.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Validation of Writing Scales for Turkish as a Second Language through Many-Facet Rasch Measurement

AU - Küçük Üçpınar , Fatma

AU - Ünaldı , Aylin

PY - 2018/12/17

Y1 - 2018/12/17

N2 - Rating scales and the extent to which raters use them effectively are two important factors that influence scoring validity of language tests when open-ended writing tasks are concerned. Research regarding rating scale development and validation in the assessment of English is ample; however, there has been no research on scale validation in the assessment of Turkish as a Second Language (TSL) to this date. This study reports on the development of two analytical rating scales used to assess academic writing skills of test takers in TSL, and presents quantitative evidence on the rating scale validation. For this purpose, texts written by 39 TSL students were scored by three raters. The analyses were conducted using Many-facet Rasch Measurement. Results indicate that empirically-developed analytical rating scales were used consistently and appropriately by the raters providing evidence for the reliability and effectiveness of the rating scales.

AB - Rating scales and the extent to which raters use them effectively are two important factors that influence scoring validity of language tests when open-ended writing tasks are concerned. Research regarding rating scale development and validation in the assessment of English is ample; however, there has been no research on scale validation in the assessment of Turkish as a Second Language (TSL) to this date. This study reports on the development of two analytical rating scales used to assess academic writing skills of test takers in TSL, and presents quantitative evidence on the rating scale validation. For this purpose, texts written by 39 TSL students were scored by three raters. The analyses were conducted using Many-facet Rasch Measurement. Results indicate that empirically-developed analytical rating scales were used consistently and appropriately by the raters providing evidence for the reliability and effectiveness of the rating scales.

KW - Scale development

KW - writing assessment of Turkish as a second language

KW - many-facet Rasch measurement

M3 - Article

VL - 34

SP - 23

EP - 48

JO - Bogazici University Journal of Education

JF - Bogazici University Journal of Education

SN - 1300-9567

IS - 1

M1 - 627600

ER -