Constructing a Scale to Assess L2 Written Speech Act Performance: WDCT and E-mail Tasks
Abstract
This study reports the development of a scale to evaluate the speech act performance by intermediate-level Chinese learners of English. A qualitative analysis of the American raters’ comments was conducted on learner scripts in response to a total of 16 apology and request written discourse completion task (WDCT) situations. The results showed that in content, these raters attended to features, including amount of information, politeness, clarity, and relevance. For form, they focused on grammar, phrasing, and word choice. These features were then incorporated into the scale design, resulting in a scoring rubric consisting of two 10-point subscales, each of which contains five levels with descriptors for each performance level. For scale validation, seven experienced American writing teachers in Taiwan evaluated 84 e-mail requests written by the Chinese learners to their professors. Reliability and validity of the rating scale were explored through the FACETS analysis. The results showed that the scale can be used in a reliable and valid way. The study concludes with a discussion of the usefulness of this rating scale and proposes directions for future research.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Features such as amount of information, politeness, clarity, and relevance, as well as forms such as grammar, phrasing, and word choice, are prominent in raters' evaluation of written speech act performance.
“The upshots of their research showed that features such as the amount of the information, politeness, clarity, and relevance as well as the forms such as grammar, phrasing, and word choice are the prominent forms in the raters' viewpoint.”
From A comparative study of the effect of teacher’s group and individual feedback on Iranian EFL learners’ learning of speech acts in apology letter writing (2020) · cites this paper as Chen and Liu (2016)
Cites
Grice, H. P. (1975) · Speech Acts · doi:10.1163/9789004368811_003 · Cited by 23,221 · see citing works
Politeness : Some Universals in Language Usage
Brown, P. (1987) · Cited by 11,901 · see citing works
Leech, G. (2016) · Routledge eBooks · doi:10.4324/9781315835976 · Cited by 7,406 · see citing works
Fundamental Considerations in Language Testing
Spolsky, B., & Bachman, L. F. (1991) · Modern Language Journal · doi:10.2307/329499 · Cited by 4,387 · see citing works
3. English as a Global Language
Pinner, R. S. (2016) · Multilingual Matters eBooks · doi:10.21832/9781783095681-004 · Cited by 3,344 · see citing works
Cross-Cultural Pragmatic Failure
THOMAS, J. (1983) · Applied Linguistics · doi:10.1093/applin/4.2.91 · Cited by 2,001 · see citing works
Cross-cultural pragmatics: Requests and apologies
Sbisà, M. (1992) · Journal of Pragmatics · doi:10.1016/0378-2166(92)90006-w · Cited by 1,695 · see citing works
Language learning and technology
Healey, D. (2016) · doi:10.4324/9781315657899.ch1 · Cited by 1,420 · see citing works
Optimizing rating scale category effectiveness.
Linacre, J. M. (2002) · PubMed · Cited by 1,287 · see citing works
(1998) · Choice Reviews Online · doi:10.5860/choice.35-3102 · Cited by 1,268 · see citing works
Measuring Second Language Performance
Kleinsasser, R. C., & McNamara, T. F. (1998) · Modern Language Journal · doi:10.2307/330236 · Cited by 1,255 · see citing works
Collaborative writing: Product, process, and students’ reflections
Storch, N. (2005) · Journal of Second Language Writing · doi:10.1016/j.jslw.2005.05.002 · Cited by 1,052 · see citing works
Weigle, S. C. (2002) · doi:10.1017/cbo9780511732997 · Cited by 779 · see citing works
Acquisition in Interlanguage Pragmatics
Barron, A. (2003) · Pragmatics & beyond. New series · doi:10.1075/pbns.108 · Cited by 652 · see citing works
Testing Second Language Speaking
Fulcher, G. (2014) · Routledge eBooks · doi:10.4324/9781315837376 · Cited by 603 · see citing works
Bardovi‐Harlig, K., Dörnyei, Z., & Dörnyei, Z. (1998) · TESOL Quarterly · doi:10.2307/3587583 · Cited by 582 · see citing works
Diagnosing Foreign Language Proficiency : The Interface between Learning and Assessment
(2005) · Bloomsbury Academic eBooks · doi:10.5040/9781474212151 · Cited by 572 · see citing works
The Phonology of English as an International Language: New Models, New Norms, New Goals
Jenkins, J. (2000) · Cited by 518 · see citing works
Studying Compliment Responses: A Comparison of DCTs and Recordings of Naturally Occurring Talk
Golato, A. (2003) · Applied Linguistics · doi:10.1093/applin/24.1.90 · Cited by 499 · see citing works
Bardovi‐Harlig, K. (1999) · Language Learning · doi:10.1111/0023-8333.00105 · Cited by 446 · see citing works
Communicative language testing
Weir, C. J. (1991) · Cited by 427 · see citing works
Using FACETS to model rater training effects
Weigle, S. C. (1998) · Language Testing · doi:10.1177/026553229801500205 · Cited by 385 · see citing works
Detecting and measuring rater effects using many-facet Rasch measurement: Part II.
Myford, C. M., & Wolfe, E. W. (2003) · PubMed · Cited by 384 · see citing works
Pragmatics and Language Learning.
Bouton, L. F. (1996) · Cited by 348 · see citing works
Can pragmatic competence be taught
Kasper, G. (1997) · Cited by 332 · see citing works
Yuan, Y. (2001) · Journal of Pragmatics · doi:10.1016/s0378-2166(00)00031-x · Cited by 301 · see citing works
Economidou‐Kogetsidis, M. (2011) · Journal of Pragmatics · doi:10.1016/j.pragma.2011.06.006 · Cited by 291 · see citing works
Beebe, L. M., & Cummings, M. C. (2006) · doi:10.1515/9783110219289.1.65 · Cited by 287 · see citing works
Blount, B. G. (1995) · John Benjamins Publishing Company eBooks · doi:10.1075/hop.m.ant1 · Cited by 278 · see citing works
Biesenbach-Lucas, S. (2007) · Language learning & technology · doi:10.64152/10125/44104 · Cited by 272 · see citing works
Purpura, J. E. (2004) · Cambridge University Press eBooks · doi:10.1017/cbo9780511733086 · Cited by 235 · see citing works
A many-facet Rasch analysis of the second language group oral discussion task
Bonk, W. J., & Ockey, G. J. (2003) · Language Testing · doi:10.1191/0265532203lt245oa · Cited by 234 · see citing works
(2006) · doi:10.1515/9783110219289 · Cited by 221 · see citing works
Diagnostic assessment of writing: A comparison of two rating scales
Knoch, U. (2009) · Language Testing · doi:10.1177/0265532208101008 · Cited by 216 · see citing works
Effects of training on raters of ESL compositions
Weigle, S. C. (1994) · Language Testing · doi:10.1177/026553229401100206 · Cited by 208 · see citing works
L2 pragmatic awareness: Evidence from the ESL classroom
Bardovi‐Harlig, K., & Griffin, R. (2005) · System · doi:10.1016/j.system.2005.06.004 · Cited by 203 · see citing works
Testing of second language pragmatics: Past and future
Roever, C. (2011) · Language Testing · doi:10.1177/0265532210394633 · Cited by 193 · see citing works
The development of e-mail literacy: From writing to peers to writing to authority figures
Chen, C.-F. E. (2006) · Language learning & technology · doi:10.64152/10125/44060 · Cited by 192 · see citing works
Constructing rating scales for second language tests
Upshur, J. A., & Turner, C. E. (1995) · ELT Journal · doi:10.1093/elt/49.1.3 · Cited by 174 · see citing works
Sasaki, M. (1998) · Journal of Pragmatics · doi:10.1016/s0378-2166(98)00013-7 · Cited by 169 · see citing works
Assessing Second Language Writing: The Rater’s Perspective
Lumley, T. (2005) · Cited by 161 · see citing works
Effective rating scale development for speaking tests: Performance decision trees
Fulcher, G., Davidson, F., & Kemp, J. (2010) · Language Testing · doi:10.1177/0265532209359514 · Cited by 155 · see citing works
Pragmatic and grammatical awareness: A function of the learning environment?
Niezgoda, K., & Röver, C. (2001) · Cambridge University Press eBooks · doi:10.1017/cbo9781139524797.008 · Cited by 150 · see citing works
Pragmatic competence and foreign language teaching
Martínez‐Flor, A., Usó-Juan, E., & Guerra, A. B. F. (2003) · Universitat Jaume I eBooks · Cited by 148 · see citing works
Validity argument for assessing L2 pragmatics in interaction using mixed methods
Youn, S. J. (2014) · Language Testing · doi:10.1177/0265532214557113 · Cited by 147 · see citing works
A CATEGORICAL INSTRUMENT FOR SCORING SECOND LANGUAGE WRITING SKILLS
Brown, J. D., & Bailey, K. M. (1984) · Language Learning · doi:10.1111/j.1467-1770.1984.tb00350.x · Cited by 119 · see citing works
Upshur, J. A., & Turner, C. E. (1999) · Language Testing · doi:10.1177/026553229901600105 · Cited by 113 · see citing works
"At Your Earliest Convenience:" A Study of Written Student Requests to Faculty.
Hartford, B. S., & Bardovi‐Harlig, K. (1996) · Cited by 109 · see citing works
The influence of training and experience on rater performance in scoring spoken language
Davis, L. (2015) · Language Testing · doi:10.1177/0265532215582282 · Cited by 107 · see citing works
Developing Chinese EFL learners’ email literacy through requests to faculty
Chen, Y. (2014) · Journal of Pragmatics · doi:10.1016/j.pragma.2014.05.009 · Cited by 101 · see citing works
Self-citation(2011) · Netbiblo eBooks · doi:10.4272/0-9729892-0-x.ch2 · Cited by 100 · see citing works
Pragmatics: Teaching Speech Acts.
Tatsuki, D. H., & Houck, N. (2010) · Cited by 98 · see citing works
Defining and Operationalizing the Construct of Pragmatic Competence: Review and Recommendations
Timpe‐Laughlin, V., Wain, J., & Schmidgall, J. E. (2015) · ETS Research Report Series · doi:10.1002/ets2.12053 · Cited by 97 · see citing works
Ke, I.-C., & Cahyani, H. (2014) · System · doi:10.1016/j.system.2014.07.008 · Cited by 97 · see citing works
Developing a common scale for the assessment of writing
Hawkey, R., & Barker, F. (2004) · Assessing Writing · doi:10.1016/j.asw.2004.06.001 · Cited by 89 · see citing works
Individual Feedback to Enhance Rater Training: Does It Work?
Elder, C., Knoch, U., Barkhuizen, G., & von Randow, J. (2005) · Language Assessment Quarterly · doi:10.1207/s15434311laq0203_1 · Cited by 88 · see citing works
Brown, P., Levinson, S., & Gumperz, J. (2021) · Cross-Cultural Pragmatics · doi:10.1017/cbo9780511813085 · Cited by 87 · see citing works
A conversation-analytic hermeneutic rating protocol to assess L2 oral pragmatic competence
Walters, F. S. (2007) · Language Testing · doi:10.1177/0265532207076362 · Cited by 79 · see citing works
Small talk in England, Ireland, and the USA
Schneider, K. P. (2008) · Pragmatics & beyond. New series · doi:10.1075/pbns.178.06sch · Cited by 73 · see citing works
Pragmatics tests: Different purposes, different tests
Brown, J. D. (2001) · Cambridge University Press eBooks · doi:10.1017/cbo9781139524797.020 · Cited by 72 · see citing works
Indicators for pragmatic instruction: Some quantitative tools
Hudson, T. D. (2001) · Cambridge University Press eBooks · doi:10.1017/cbo9781139524797.019 · Cited by 66 · see citing works
Tests of oral performance: the need for data-based criteria
Fulcher, G. (1987) · ELT Journal · doi:10.1093/elt/41.4.287 · Cited by 65 · see citing works
Pragmatic Knowledge: Rules and Procedures
Færch, C., & Kasper, G. (1984) · Applied Linguistics · doi:10.1093/applin/5.3.214 · Cited by 64 · see citing works
Panayiotis Panayides, A., & Walker, M. J. (2013) · Europe’s Journal of Psychology · doi:10.5964/ejop.v9i3.611 · Cited by 47 · see citing works
Written Requests in Emails Sent by Adult Chinese Learners of English
Lee, C. F. K. (2004) · Language Culture and Curriculum · doi:10.1080/07908310408666682 · Cited by 47 · see citing works
Assessing second language pragmatics: An overview and introductions
Kasper, G., & Ross, S. J. (2013) · Palgrave Macmillan UK eBooks · doi:10.1057/9781137003522_1 · Cited by 40 · see citing works
Information sequencing in Modern Standard Chinese in a genre of extended spoken discourse
Kirkpatrick, A. (1993) · Text and Talk · doi:10.1515/text.1.1993.13.3.423 · Cited by 28 · see citing works
Small Talk in England, Ireland, and the U.S.A
Schneider, K. P. (2008) · Pragmatics and beyond. New series · Cited by 18 · see citing works
Interlanguage pragmatics: A reply to Pilar Garces-Conejos Blitvich
Cohen, A. D. (2006) · Intercultural Pragmatics · doi:10.1515/ip.2006.021 · Cited by 10 · see citing works
Investigating the Complimenting Behaviors of Chinese Speakers of American English
Chen, Y., & Rau, D.-H. V. (2011) · Language and Linguistics 語言暨語言學 · Cited by 9 · see citing works
Self-citation
Flagged first, then by citations
1 of the 72 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Ren, W. (2022) · Cambridge University Press eBooks · doi:10.1017/9781009082709 · Cited by 84 · see citing works
Su, H., & Lu, X. (2022) · Assessing Writing · doi:10.1016/j.asw.2022.100668 · Cited by 38 · see citing works
Taguchi, N., & Ли, Ш. (2020) · Contrastive Pragmatics · doi:10.1163/26660393-bja10014 · Cited by 31 · see citing works
Variations in Rating Scale Functioning in Assessing Speech Act Production in L2 Chinese
Ли, Ш., Taguchi, N., & Xiao, F. (2019) · Language Assessment Quarterly · doi:10.1080/15434303.2019.1648473 · Cited by 20 · see citing works
Pourmousavi, Z., & Mohamadi Zenouzagh, Z. (2020) · Asian-Pacific Journal of Second and Foreign Language Education · doi:10.1186/s40862-020-00088-w · Cited by 17 · see citing works
Effects of linguistic proficiency on speech act development in L2 Chinese during study abroad
Ли, Ш., Tang, X., Taguchi, N., & Xiao, F. (2022) · Study Abroad Research in Second Language Acquisition and International Education · doi:10.1075/sar.21014.li · Cited by 12 · see citing works
McDonald, K. (2018) · Language Testing in Asia · doi:10.1186/s40468-018-0074-3 · Cited by 12 · see citing works
The difficulty of oral speech act production tasks in second language pragmatics testing
Huang, W., & Lu, X. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1096399 · Cited by 10 · see citing works
Comparing holistic and analytic marking methods in assessing speech act production in L2 Chinese
Ли, Ш., Wen, T., Li, X., Feng, Y., & Chuan, L. (2022) · Language Testing · doi:10.1177/02655322221113917 · Cited by 9 · see citing works
Establishing an Operational Model of Rating Scale Construction for English Writing Assessment
Wu, X. (2021) · English Language Teaching · doi:10.5539/elt.v15n1p16 · Cited by 7 · see citing works
Su, Y., & Shin, S. (2023) · Language Testing · doi:10.1177/02655322231210217 · Cited by 5 · see citing works
Zhang, E. D., & Yu, S. (2021) · Applied Linguistics Review · doi:10.1515/applirev-2021-0014 · Cited by 4 · see citing works
Eskin, D. (2017) · DOAJ (DOAJ: Directory of Open Access Journals) · doi:10.7916/d8b00gs7 · Cited by 4 · see citing works
Rating scales in L2 speech act research: A systematic review
Gao, X., & Deng, L. (2025) · Research Methods in Applied Linguistics · doi:10.1016/j.rmal.2025.100204 · Cited by 2 · see citing works
Individual versus pair work on L2 speech acts: production and cognitive processes
Lin, M., Chen, Y., & Wu, H. (2022) · Applied Linguistics Review · doi:10.1515/applirev-2021-0088 · Cited by 2 · see citing works
Self-citationAssessing Pragmatic Skills in Speech Act Production and Perception
Gesuato, S., & Trubnikova, V. (2021) · Advances in educational marketing, administration, and leadership book series · doi:10.4018/978-1-7998-8579-5.ch010 · Cited by 1 · see citing works
Assessing pragmatic aspects of L2 communication: Why, how and what for
Gesuato, S., & Castello, E. (2020) · Lodz Papers in Pragmatics · doi:10.1515/lpp-2020-0001
Non-Expert Raters’ Scoring Behavior and Cognition in Assessing Pragmatic Production in L2 Chinese
Ли, Ш., Li, X., Feng, Y., & Wen, T. (2023) · Educational linguistics · doi:10.1007/978-3-031-24078-2_4
Language Acquisition in Diverse Linguistic, Social and Cognitive Circumstances, volume II
Garraffa, M., Morgan, G., Marinis, T., & Guasti, M. T. (2023) · Frontiers research topics · doi:10.3389/978-2-8325-3573-8
Pragmatics in Second Language Acquisition: Overview of Selected Issues
Róg, T. (2025) · Second language learning and teaching · doi:10.1007/978-3-031-86566-4_1
The Qualitative Phase in Individual Differences Scale Development in SLA: A Methodological Synthesis
Lyu, M., Wu, C., Jia, R., & Sudina, E. (2026) · Research Synthesis in Applied Linguistics · doi:10.1080/29984475.2026.2713533
- Authors
- Chen, Y.-S., & Liu, J.
- Affiliations
- National Chin-Yi University of Technology (TW)
- Guangdong University of Foreign Studies (CN)
- Corresponding author
- Chen, Yuan-shan
- Venue
- Language Assessment Quarterly
- Publisher
- Informa UK Limited
- Published
- 2016-07-02
- Volume
- 13
- Issue
- 3
- Pages
- 231-250
- ISSN
- 1543-4303 (ISSN-L)
- 1543-4311
- OpenAlex
- W2510639838
- Cited by
- 20
- References
- 72
- Citations by year
- 2026: 1 · 2025: 2 · 2023: 3 · 2022: 5 · 2021: 3 · 2020: 3 · 2019: 1 · 2018: 1 · 2017: 1
- FWCI
- 6.37
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Language Assessment Quarterly (publishedVersion)
- Funders
- National Science Council (NSC 100-2410-H-167-009)
- Topics
- Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
- Social Sciences › Arts and Humanities › Language and Linguistics › Linguistics and Language Analysis
- Social Sciences › Psychology › Developmental and Educational Psychology › Second Language Acquisition and Learning
- Keywords
- second language writing, speech act performance, rating scale development, rubrics, email writing, Chinese learners of English, politeness, scale validation, inter-rater reliability, writing assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
