Discourse measures for Basque summary grading
Abstract
In the context of Learning Technologies, the need to be able to assess the learning and domain comprehension in open-ended learner responses has been present in artificial intelligence and education since its beginnings. The advantage of using summaries is that they allow teachers to diagnose comprehension and the amount of information remembered from text in the learning process. This study addresses the issue of automatically obtaining overall discourse scores from surface discourse measures for Basque language. Global measures have been studied for cohesion, adequacy and use of language. The approach taken was to estimate the presence of the automatically gathered surface discourse measures in expert grading decisions in cohesion, adequacy and use of language. As a consequence, three grading decision-making regression models were obtained to estimate overall grades from text written in Basque. Next, the obtained regression models were tested in corpus-containing summaries written by learners with different degrees of summarisation maturity. The results show that the obtained grading frameworks significantly reflect human decisions and are able to discriminate summarisation maturity differences.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Immediate feedback promotes active training in interactive learning environments that would benefit learner's comprehension diagnosis.
“Immediate feedback promotes active training in interactive learning environments that would benefit learner's comprehension diagnosis [19].”
From A Review of Artificial Intelligence (AI) in Education from 2010 to 2020 (2021) · cites this paper as [19]
The development procedure was usually conducted with an induction-deduction approach, in which prior experiments and data were analyzed to predict variables followed by algorithm testing to obtain the final modelling equation.
“e development procedure was usually conducted with an induction-deduction approach, in which prior experiments and data were analyzed to predict the variables followed by the algorithm testing to obtain the final modelling equation [19].”
From A Review of Artificial Intelligence (AI) in Education from 2010 to 2020 (2021) · cites this paper as [19]
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. E., & Cohen, J. (1989) · Technometrics · doi:10.2307/1270020 · Cited by 99,588 · see citing works
Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,406 · see citing works
REMEMBERING: A STUDY IN EXPERIMENTAL AND SOCIAL PSYCHOLOGY
Bartlett, F. C., & Burt, C. (1933) · British Journal of Educational Psychology · doi:10.1111/j.2044-8279.1933.tb02913.x · Cited by 7,475 · see citing works
Halliday, M. A. K., & Hasan, R. (2014) · Routledge eBooks · doi:10.4324/9781315836010 · Cited by 7,392 · see citing works
Toward a model of text comprehension and production.
Kintsch, W., & van Dijk, T. A. (1978) · Psychological Review · doi:10.1037/0033-295x.85.5.363 · Cited by 5,298 · see citing works
Bartlett, F. C., & Kintsch, W. (1995) · Cambridge University Press eBooks · doi:10.1017/cbo9780511759185 · Cited by 2,408 · see citing works
Coh-Metrix: Analysis of text on cohesion and language
Graesser, A. C., McNamara, D. S., Louwerse, M. M., & Cai, Z. (2004) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03195564 · Cited by 1,556 · see citing works
Handbook of Discourse Analysis
Dittmar, N., & Forsthoffer, I. (2006) · Journal of Pragmatics · doi:10.1016/j.pragma.2006.03.004 · Cited by 1,549 · see citing works
The Handbook of Discourse Analysis
(2003) · Blackwell Publishing Ltd eBooks · doi:10.1111/b.9780631205968.2003.00012.x · Cited by 929 · see citing works
Macrorules for summarizing texts: the development of expertise
Brown, A. L., & Day, J. D. (1983) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(83)80002-4 · Cited by 714 · see citing works
Hualde, J. I., & de Urbina, J. O. (2003) · Mouton eBooks · doi:10.1515/9783110895285 · Cited by 706 · see citing works
(1992) · doi:10.1142/1588 · Cited by 550 · see citing works
Coh-Metrix: Capturing Linguistic Features of Cohesion
McNamara, D. S., Louwerse, M. M., McCarthy, P. M., & Graesser, A. C. (2010) · Discourse Processes · doi:10.1080/01638530902959943 · Cited by 309 · see citing works
Syntactic Simplification and Text Cohesion
Siddharthan, A. (2006) · Research on Language and Computation · doi:10.1007/s11168-006-9011-1 · Cited by 275 · see citing works
Text structure and children's comprehension and memory for expository material.
Taylor, B. M. (1982) · Journal of Educational Psychology · doi:10.1037/0022-0663.74.3.323 · Cited by 150 · see citing works
The role of long-term working memory in text comprehension.
Kintsch, W., Patel, V. L., & Ericsson, K. A. (1999) · Psychologica · Cited by 124 · see citing works
Developing Summarization Skills through the Use of LSA-Based Feedback
Kintsch, E., Steinhart, D., Stahl, G., LSA Research Group LSA Research Group, Matthews, C., & Lamb, R. (2000) · Interactive Learning Environments · doi:10.1076/1049-4820(200008)8:2;1-b;ft087 · Cited by 123 · see citing works
Fripp, A. L., Fripp, J. B., & Fripp, M. (2003) · Elsevier eBooks · doi:10.1016/b978-075067535-2/50010-3 · Cited by 89 · see citing works
Efficient Text Summarization Costs and Benefits
Garner, R. (1982) · The Journal of Educational Research · doi:10.1080/00220671.1982.10885394 · Cited by 68 · see citing works
Summarizing Emails with Conversational Cohesion and Subjectivity
Carenini, G., Ng, R. T., & Zhou, X. (2008) · Cited by 66 · see citing works
Coherence, cohesion and comments on students’ academic essays
Todd, R. W., Khongput, S., & Darasawang, P. (2007) · Assessing Writing · doi:10.1016/j.asw.2007.02.002 · Cited by 61 · see citing works
First vs second language comprehension: some evidence from text summarizing
Kozminsky, E., & Graetz, N. (1986) · Journal of Research in Reading · doi:10.1111/j.1467-9817.1986.tb00107.x · Cited by 39 · see citing works
A Cascaded Syntactic Analyser for Basque
Aduriz, I., Aranzabe, M. J., Egurrola, J. M. A., et al. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-24630-5_15 · Cited by 34 · see citing works
Lexical cohesion and term proximity in document ranking
Vechtomova, O., & Karamuftuoglu, M. (2008) · Information Processing & Management · doi:10.1016/j.ipm.2008.01.003 · Cited by 31 · see citing works
Recent Advances in Natural Language Processing
(1997) · Amsterdam studies in the theory and history of linguistic science. Series 4, Current issues in linguistic theory · doi:10.1075/cilt.136 · Cited by 24 · see citing works
A spelling corrector for Basque based on morphology
Aduriz, I., Urkia, M., Alegria, I., et al. (1997) · Literary and Linguistic Computing · doi:10.1093/llc/12.1.31 · Cited by 21 · see citing works
Long, J. B., & Harding-Esch, E. (1978) · Springer eBooks · doi:10.1007/978-1-4615-9077-4_24 · Cited by 14 · see citing works
Observing Lemmatization Effect in LSA Coherence and Comprehension Grading of Learner Summaries
Zipitria, I., Arruarte, A., & Elorriaga, J. A. (2006) · Lecture notes in computer science · doi:10.1007/11774303_59 · Cited by 12 · see citing works
Self-citationExploring Treebank Transformations in Dependency Parsing
Bengoetxea, K., & Gojenola, K. (2009) · Cited by 10 · see citing works
What is behind a summary-evaluation decision?
Zipitria, I., Larrañaga, P., Armañanzas, R., et al. (2008) · Behavior Research Methods · doi:10.3758/brm.40.2.597 · Cited by 7 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
A Review of Artificial Intelligence (AI) in Education from 2010 to 2020
Zhai, X., Chu, X., Chai, C. S., Jong, M. S. Y., Istenic, A., Spector, M., Liu, J.-B., Yuan, J., & Li, Y. (2021) · Complexity · doi:10.1155/2021/8812542 · Cited by 1,364 · see citing works
“…as intelligent tutoring system (ITS) and electronic assessment. development procedure was usually conducted with an induction-deduction approach, in which prior experiments and data were analyzed to predict the variables followed by the algorithm testing to obtain the final modelling equation [19].”
BackgroundMethod
- Authors
- Zipitria, I., Arruarte, A., & Elorriaga, J.
- Affiliations
- University of the Basque Country (ES)
- Corresponding author
- Zipitria, I.
- Venue
- Interactive Learning Environments
- Publisher
- Informa UK Limited
- Published
- 2013-12
- Volume
- 21
- Issue
- 6
- Pages
- 528-547
- ISSN
- 1049-4820 (ISSN-L)
- 1744-5191
- OpenAlex
- W2070818305
- Cited by
- 7
- References
- 30
- Citations by year
- 2025: 1 · 2024: 2 · 2023: 1 · 2021: 1 · 2017: 1 · 2012: 1
- FWCI
- 0.29
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Interactive Learning Environments (publishedVersion)
- Funders
- Eusko Jaurlaritza
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- Basque language, automated grading, cohesion, adequacy, language use, regression models, discourse analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
