CoChat

How to improve argumentative syntheses written by undergraduates using guides and instructional rubrics

Open access
Cuevas, Isabel, Mateos, Mar, Casado-Ledesma, Lidia, Olmos, Ricardo, Granado-Peinado, Miriam, Luna, María, Núñez, Juan Antonio, Martín, Elena · 2024 · European Journal of Psychology of Education

The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index; 9 of the 21 sources its claims cite were checked for support.

Abstract

Undergraduates often struggle writing argumentative syntheses from conflicting sources. Written guides can help in the different phases of the process involved in these tasks and are more effective when accompanied by explicit instruction. Nevertheless, there are few studies on instructional rubrics as an aid to argumentative writing and none are focused on synthesis tasks. Our objectives were to compare (1) the effectiveness of a guide and a rubric as aids to the processes of selection and integration in writing an argumentative synthesis; (2) whether explicit instruction in synthesis writing strategies enhances the effects of both aids and (3) the effectiveness of the aids offered during the practice sessions performed with the support of aids and after removing those aids. The study was conducted with 120 undergraduate psychology students. An experimental inter/intra-subject factorial design 2 (Instruction) x 2 (Type of aid) x 4 (Time) was employed. We used mixed linear models to assess the intervention effects. The guide facilitated the selection of arguments. Both guide and rubric promoted integration. When students also received explicit instruction, the learning rate of integration strategies was accelerated, and the impact of guide and rubric was greater.

Overview

CoChat Summary

TL;DR

This experimental study with 120 undergraduates tested whether a written guide versus an instructional rubric improves argumentative synthesis writing from conflicting sources, and whether explicit instruction enhances those aids. Both aids improved integration, but only the guide improved argument selection; explicit instruction produced greater and more durable integration gains after the aids were removed.

Key findings

  • Both the guide and the rubric promoted integration: students moved from a mean score of 3 (refutation) to 5 or 6 (weighing), though most did not reach high integration levels (8–10).
  • The guide facilitated selection of arguments, whereas the rubric did not: coverage of arguments was 0.907 points higher on average with the guide than with the rubric (95% CI [.289–1.525]); after adjusting for pre-test scores, corrected differences were 0.781 and 0.929 in favor of the guide at second practice and post-test.
  • Explicit instruction increased integration: EI yielded a mean 0.686 points higher than WEI (95% CI [0.115–1.256]); post-test versus pre-test gains were 2.42 points in EI (d = 0.839) and 1.554 points in WEI (d = 0.492).
  • After aids were withdrawn, EI students maintained integration (second practice vs. post-test p = .870), while WEI students showed a marginally significant decline (p = .092); at post-test, EI still outperformed WEI by 0.915 points (p = .048, d = 0.360).

Methods

  • Participants were 120 second-year psychology undergraduates (17.5% male, 82.5% female; age 19–21, mean 19.3, SD = 0.49), all native Spanish speakers, volunteering for academic credit.
  • An experimental inter/intra-subject factorial design 2 (Instruction: explicit vs. without explicit) × 2 (Aid: guide vs. rubric) × 4 (Time) was used; sessions ran online via Teams over 4 consecutive weeks and included pre-test, instruction plus practice 1, practice 2, and post-test, with counterbalanced text order.
  • Mixed linear models assessed effects on two dependent variables: integration level (10-point scale) and coverage of arguments; inter-rater reliability was ICC = .918 for integration and ICC = .885 for coverage, and models were also adjusted for pre-test scores.

Limitations

  • The paper states that it did not analyze how students used the aids and suggests in-depth interviews or discussion groups would help understand their usefulness; it also notes that the aids did not lead most students to high integration (levels 8–10) and that integrative conclusions remain particularly complex.
  • Evident from the text: the sample was limited to second-year psychology undergraduates, mostly female, native Spanish speakers, and volunteers, which may restrict generalizability.
  • Evident from the text: all sessions were implemented online because of COVID-19, and the study did not examine longer-term retention beyond the post-test.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s10212-024-00890-x

Cites

45 of 45 references verified0 citation errors · 0 not checkable

Statistics

9 tests recomputed · 0 with the reported and recomputed p on opposite sides of .05

Verification

Claims the paper makes

6 Valid · 1 Not valid · 6 Not checked · 76 Own results and interpretations

55 more claims

Create a free account to see all 89 claims and to verify the 12 references still open.

See all claims — free account
  • Probably not a substantial claim. 16
  • It may state two separate claims as one. 19

Create a free account to see these claims.

Statistics that don’t add up

Each reported p-value recomputed from its test statistic and degrees of freedom.

1 of 1

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Doesn’t add upF(1, 240) = 2.391reported p = .094 · recomputed p = .123

    “F(1,240) = 2.391, p = .094”

  • ConsistentF(3, 211) = 46.8reported p < .001 · recomputed p < .001

    “F(3,211)=46.800,p < .001”

  • ConsistentF(1, 121) = 5.993reported p = .016 · recomputed p = .016

    “F(1,121)=5.993,p = .016”

  • ConsistentF(3, 211) = 2.34reported p = .074 · recomputed p = .074

    “F(3,211)=2.340,p = .074”

  • ConsistentF(3, 360) = 4.179reported p = .006 · recomputed p = .006

    “F(3,360) = 4.179, p = .006”

5 of 9 · the rest aren’t loaded

4 more recomputed tests

Create a free account to see every test the Stats check recomputed.

See all tests — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 45 references

Research works printed but not linked: 4

  1. European Commission, Directorate-General for Education, Youth, Sport and Culture. (2019). Key competences for lifelong learning, Publications Office. https://doi.org/10.2766/569540 · doi:10.2766/569540
  2. Klein, P. D., Arcon, N., & Baker, S. (2016). Writing to learn. In C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (pp. 243–256). Guilford Press.
  3. Sundeen, T. H. (2014). Essay development for secondary students with learning disabilities: Graphic organizers for visualizing organizational patterns. Rural Special Education Quarterly, 33(3), 29–36. https://doi.org/10.1177/8756870514033003 · doi:10.1177/8756870514033003
  4. Szigriszt Pazos, F. (2001). Sistemas predictivos de legilibilidad del mensaje escrito: fórmula de perspicuidad [Predictive systems of readability of the written message: perspicuity formula]. Universidad Complutense de Madrid, Servicio de Publicaciones.

Cited by

See all 1

Metadata

Authors
Cuevas, I., Mateos, M., Casado-Ledesma, L., Olmos, R., Granado-Peinado, M., Luna, M., Núñez, J. A., & Martín, E.
Affiliations
  • Universidad Autónoma de Madrid (ES)
  • University of Florence (IT)
  • Universidad Francisco de Vitoria (ES)
  • Universidad a Distancia de Madrid (ES)
Corresponding author
  • Cuevas, Isabel
  • Mateos, Mar
  • Casado-Ledesma, Lidia
  • Olmos, Ricardo
  • Granado-Peinado, Miriam
  • Luna, María
Venue
European Journal of Psychology of Education
Published
2024-07-29
Volume
39
Issue
4
Pages
4573-4596
ISSN
  • 0256-2928 (ISSN-L)
  • 1878-5174
OpenAlex
W4401075492
Cited by
1
References
42
Citations by year
2026: 1
FWCI
1.26
Type
journal article
Language
en
Open access
Licence
Locations
  • European Journal of Psychology of Education (publishedVersion, cc-by)
  • Biblos-e Archivo (Universidad Autónoma de Madrid) (submittedVersion, cc-by)
  • Hispana (submittedVersion, cc-by)
Funders
  • Ministerio de Ciencia e Innovación (PID2019-105250RB-I00)
  • Università degli Studi di Firenze
Topics
  • Social Sciences › Education › Student Assessment and Feedback
  • Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
  • Social Sciences › Psychology › Developmental and Educational Psychology › Educational Strategies and Epistemologies
Keywords
writing guidelines, explicit instruction, argument selection, undergraduate writing, writing strategies, argumentative writing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-09-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-09-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: How to improve argumentative syntheses written by undergraduates using guides and instructional rubrics. https://app.cochat.ai/papers/10.1007/s10212-024-00890-x
The paper
Cuevas, I., Mateos, M., Casado-Ledesma, L., Olmos, R., Granado-Peinado, M., Luna, M., Núñez, J. A., & Martín, E. (2024). How to improve argumentative syntheses written by undergraduates using guides and instructional rubrics. European Journal of Psychology of Education. https://doi.org/10.1007/s10212-024-00890-x

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.