How to improve argumentative syntheses written by undergraduates using guides and instructional rubrics
Open accessThe DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index; 9 of the 21 sources its claims cite were checked for support.
Abstract
Undergraduates often struggle writing argumentative syntheses from conflicting sources. Written guides can help in the different phases of the process involved in these tasks and are more effective when accompanied by explicit instruction. Nevertheless, there are few studies on instructional rubrics as an aid to argumentative writing and none are focused on synthesis tasks. Our objectives were to compare (1) the effectiveness of a guide and a rubric as aids to the processes of selection and integration in writing an argumentative synthesis; (2) whether explicit instruction in synthesis writing strategies enhances the effects of both aids and (3) the effectiveness of the aids offered during the practice sessions performed with the support of aids and after removing those aids. The study was conducted with 120 undergraduate psychology students. An experimental inter/intra-subject factorial design 2 (Instruction) x 2 (Type of aid) x 4 (Time) was employed. We used mixed linear models to assess the intervention effects. The guide facilitated the selection of arguments. Both guide and rubric promoted integration. When students also received explicit instruction, the learning rate of integration strategies was accelerated, and the impact of guide and rubric was greater.
Overview
CoChat Summary
TL;DR
This experimental study with 120 undergraduates tested whether a written guide versus an instructional rubric improves argumentative synthesis writing from conflicting sources, and whether explicit instruction enhances those aids. Both aids improved integration, but only the guide improved argument selection; explicit instruction produced greater and more durable integration gains after the aids were removed.
Key findings
- Both the guide and the rubric promoted integration: students moved from a mean score of 3 (refutation) to 5 or 6 (weighing), though most did not reach high integration levels (8–10).
- The guide facilitated selection of arguments, whereas the rubric did not: coverage of arguments was 0.907 points higher on average with the guide than with the rubric (95% CI [.289–1.525]); after adjusting for pre-test scores, corrected differences were 0.781 and 0.929 in favor of the guide at second practice and post-test.
- Explicit instruction increased integration: EI yielded a mean 0.686 points higher than WEI (95% CI [0.115–1.256]); post-test versus pre-test gains were 2.42 points in EI (d = 0.839) and 1.554 points in WEI (d = 0.492).
- After aids were withdrawn, EI students maintained integration (second practice vs. post-test p = .870), while WEI students showed a marginally significant decline (p = .092); at post-test, EI still outperformed WEI by 0.915 points (p = .048, d = 0.360).
Methods
- Participants were 120 second-year psychology undergraduates (17.5% male, 82.5% female; age 19–21, mean 19.3, SD = 0.49), all native Spanish speakers, volunteering for academic credit.
- An experimental inter/intra-subject factorial design 2 (Instruction: explicit vs. without explicit) × 2 (Aid: guide vs. rubric) × 4 (Time) was used; sessions ran online via Teams over 4 consecutive weeks and included pre-test, instruction plus practice 1, practice 2, and post-test, with counterbalanced text order.
- Mixed linear models assessed effects on two dependent variables: integration level (10-point scale) and coverage of arguments; inter-rater reliability was ICC = .918 for integration and ICC = .885 for coverage, and models were also adjusted for pre-test scores.
Limitations
- The paper states that it did not analyze how students used the aids and suggests in-depth interviews or discussion groups would help understand their usefulness; it also notes that the aids did not lead most students to high integration (levels 8–10) and that integrative conclusions remain particularly complex.
- Evident from the text: the sample was limited to second-year psychology undergraduates, mostly female, native Spanish speakers, and volunteers, which may restrict generalizability.
- Evident from the text: all sessions were implemented online because of COVID-19, and the study did not examine longer-term retention beyond the post-test.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
6 Valid · 1 Not valid · 6 Not checked · 76 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Our results are coherent with those obtained in other studies investigating the direct or indirect effect -through collaborative practice sessions with the support of a guide-of explicit instruction on two argumentative synthesis quality indicators (selection and integration) (Granado-Peinado et al., 2023;Mateos et al., 2020).”
§ How to improve argumentative syntheses written by undergraduates using guides and instructional rubrics > Discussion
What the sources say
- Supportedcites Granado-Peinado et al., 2023 source
The Contributions of the Study section states that the paper analysed the direct effect of explicit instruction on level of integration and its indirect effect via the synthesis written during practice in pairs on argument identification (i.e., argument selection/coverage), and explicitly says 'These findings are similar to those obtained in the previous study by Mateos et al., (2020)'. The Results section 'Level of Integration' confirms a direct programme effect on intermediate and final integration quality with no significant indirect effect, while the mediation analysis for argument identification showed the indirect path. Thus the cited work is indeed one of the studies investigating the direct or indirect (guide-supported collaborative practice) effect of explicit instruction on the two argumentative-synthesis quality indicators the citing sentence names. (LLM, conf=0.88)
- Supportedcites Mateos et al., 2020 source
The Discussion (p. n/a) states the paper's aim exactly as the citing sentence attributes it: 'to test assumed causal relations between explicit instruction on the two indicators of good argumentative synthesis texts-argument coverage and integration-directly and/or via the subsequent collaborative practice', with practice modelled as 'two collaborative practice sessions, supported by a written strategy guide'. The Abstract and Results confirm the design (explicit instruction via explaining and modelling vs self-study of the guide, then two guided collaborative practice sessions) and the finding that instruction was effective for both indicators while the indirect path via practice held only for coverage — consistent with the citing sentence's disjunctive 'direct or indirect effect'. The only wording shift is the citing authors' label 'selection' for what this source calls argument (ation) coverage / 'identification and coverage of the arguments from both sources' (Discussion), which the source itself treats as the same construct, so nothing attributed here is unsupported. (LLM, conf=0.88)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“In the case of argument selection, previous studies show that intermediate practice sessions with the support of a guide mediate the full impact of explicit instruction on the number of arguments and counterarguments selected by students from the sources and included in their final syntheses (Granado-Peinado et al., 2023).”
§ How to improve argumentative syntheses written by undergraduates using guides and instructional rubrics > Discussion
What the sources say
- Supportedcites Granado-Peinado et al., 2023 source
The Results section 'Argument Identification' reports that 'the mediation analysis indicated that for argument identification, the effect of the programme on the final collaborative synthesis is completely mediated by the quality of synthesis written on practice in pairs sessions' (direct effect of programme on the final synthesis non-significant, p = .674; total indirect effect p = .014). The 'Contributions of the Study' section repeats that synthesis written during practice in pairs 'fully mediated the effect of the explicit instruction only on the dimension of argument identification', and argument identification is defined in Methods as the proportion/number of arguments and counterarguments identified from the source texts in the collaborative syntheses. The only loose edge is the phrase 'with the support of a guide': in our copy the written guide is part of the intervention programmes (CPG/EICS/EIS) rather than the mediator itself, whose mediator is the quality of the intermediate practice syntheses. (LLM, conf=0.82)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“These findings align with previous research indicating that the effectiveness of guides is enhanced when synthesis writing strategies are explicitly taught.”
§ How to improve argumentative syntheses written by undergraduates using guides and instructional rubrics > Discussion
What the sources say
- Supportedcites Granado-Peinado et al., 2023 source
The paper's design directly contrasts a guide-only condition with guide-plus-explicit-instruction conditions: 'the students in the CPG programme only received the guide in this session, without any instruction about its purpose or how to use it,' whereas CPG+EIS/CPG+EICS paired the guide with explicit instruction and video modelling (Methods, 'Explicit Instruction with Video-Modelling'). The Results section reports that 'the pairs of students in the explicit instruction programmes (CPG+EICS and CPG+EIS) significantly outperformed the quality of the syntheses than the non-explicit programmes (CPG and CP),' i.e. the guide's support was more effective when synthesis strategies were explicitly taught. Its review of earlier work (Mateos et al., 2018, CPG vs CPG+EI: 'Only those students who received additional explicit instruction improved their ability to integrate conflicting information') states the same point, so the background claim is supported, though in this study the population is low-proficiency undergraduates writing collaborative argumentative syntheses. (LLM, conf=0.88)
- Not checked yetcites Mateos et al., 2018 source
- Supportedcites Luna et al., 2023 source
The Discussion states that the results 'concur with previous research (Ferreti & Lewis, 2013; González-Lamas et al., 2016; Mateos et al., 2018), because explicit instruction on processes with the help of a written guide as a scaffold is more useful than other types of instructional aids', and that integration rose more in the process intervention (which paired the guide with explicit instruction). The 'Teaching argumentative writing' section backs this by summarising Mateos et al. (2018) — only the group receiving the guide plus explicit instruction on selecting, contrasting and integrating arguments improved — and González-Lamas et al. (2016), where only the process-focused intervention with explicit instruction yielded improved integration. The claim's framing as 'previous research' matches how the source itself presents it, so the paraphrase is supported. (LLM, conf=0.85)
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Integration means vary across the study and EI yields a significantly higher mean than WEI (0.686 points higher on average 95 % IC [0.115 -1.256]).”
§ How to improve argumentative syntheses written by undergraduates using guides and instructional rubrics > Integration
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In general, there was an increase in the means for integration and coverage of arguments in the two practical sessions with respect to the pre-test moment.”
§ How to improve argumentative syntheses written by undergraduates using guides and instructional rubrics > Descriptive analysis
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
55 more claims
Create a free account to see all 89 claims and to verify the 12 references still open.
- Probably not a substantial claim. 16
- It may state two separate claims as one. 19
Statistics that don’t add up
Each reported p-value recomputed from its test statistic and degrees of freedom.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“F(1,240) = 2.391, p = .094”
§ How to improve argumentative syntheses written by undergraduates using guides and instructional rubrics > Coverage of arguments
Reported p = .094 · recomputed p = .123
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Doesn’t add upF(1, 240) = 2.391reported p = .094 · recomputed p = .123
“F(1,240) = 2.391, p = .094”
- ConsistentF(3, 211) = 46.8reported p < .001 · recomputed p < .001
“F(3,211)=46.800,p < .001”
- ConsistentF(1, 121) = 5.993reported p = .016 · recomputed p = .016
“F(1,121)=5.993,p = .016”
- ConsistentF(3, 211) = 2.34reported p = .074 · recomputed p = .074
“F(3,211)=2.340,p = .074”
- ConsistentF(3, 360) = 4.179reported p = .006 · recomputed p = .006
“F(3,360) = 4.179, p = .006”
4 more recomputed tests
Create a free account to see every test the Stats check recomputed.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ How to improve argumentative syntheses written by undergraduates using guides and instructional rubrics > Conclusions and Educational implications
“| [7] T + Ins + Aid + TxIns + TxAid + InsxAid + | 2118.5 | 2185.3 | 2.810 (1) | .0936 |”
§ How to improve argumentative syntheses written by undergraduates using guides and instructional rubrics > Conclusions and Educational implications
“| [7] T + Ins + Aid + TxIns + TxAid + InsxAid + | 2177.4 | 2265.0 | 2.849 (3) | .4155 |”
Citations
Cites
A guideline of selecting and reporting intraclass correlation coefficients for reliability research
Koo, T. K., & Li, M. Y. (2016) · Journal of Chiropractic Medicine · doi:10.1016/j.jcm.2016.02.012 · Cited by 31,709 · cited 1×
“Following the typology of Koo and Li (2016), a Two-Way Mixed-Effect Model of absolute agreement for the mean of two raters was calculated.”
Spillane, J. P. (2003) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737025004343 · Cited by 4,286
MacArthur, C. A., Graham, S. W., & Fitzgerald, J. (2005) · Cited by 2,327
What is assessment for learning
Wiliam, D. (2011) · Studies in Educational Evaluation · doi:10.1016/j.stueduc.2011.03.001 · Cited by 1,282 · cited 1×
“In recent decades, much research has been conducted to analyze the potential of rubrics to promote students’ learning processes (Wiliam, 2011).”
Blanshard, B., & The Hegeler Institute (1968) · Monist · doi:10.5840/monist196852112 · Cited by 775
Reliability and validity of rubrics for assessment through writing
Rezaei, A. R., & Lovorn, M. (2010) · Assessing Writing · doi:10.1016/j.asw.2010.01.003 · Cited by 299 · cited 1×
“Within the specific field of writing, rubrics may be employed to evaluate the quality of texts (e.g. Rezaei & Lovorn, 2010), but also to provide detailed feedback contributing to learning this process and to developing self-regulation skills related to writing (Saddler & Andrade, 2004).”
Promoting Argument-Counterargument Integration in Students' Writing
Nussbaum, E. M., & Schraw, G. (2007) · The Journal of Experimental Education · doi:10.3200/jexe.76.1.59-92 · Cited by 279 · cited 2×
“…the “my-side bias” (Wolfe et al., 2009) and apply in their final text synthesis writing strategies involving balanced reasoning, such as “weighing” and “synthesizing” (Nussbaum & Schraw, 2007), both in the body of the text and their conclusions after weighing the pros and cons of both perspectives.”
Readers as Writers Composing From Sources
Spivey, N. N., & King, J. R. (1989) · Reading Research Quarterly · doi:10.1598/rrq.24.1.1 · Cited by 276 · cited 1×
“It requires students to understand the ideas contained in the texts; to select information; to organize and compare it and integrate it to establish a new structure that gives rise to a new text (Nelson & King, 2023; Segev-Miller, 2007; Spivey & King, 1989).”
Andrade, H. L., Du, Y., & Wang, X. (2008) · Educational Measurement: Issues and Practice · doi:10.1111/j.1745-3992.2008.00118.x · Cited by 256 · cited 1×
“Rubrics promote the objectivity, validity and dependability of the assigned ratings (e.g. Dempsey et al., 2009), but also make visible the learning objectives and provide useful feedback to identify strengths and weaknesses in a written product (Andrade et al., 2008).”
Nussbaum, E. M., & Edwards, O. V. (2011) · Journal of the Learning Sciences · doi:10.1080/10508406.2011.564567 · Cited by 198 · cited 3×
“…selecting, comparing and prioritizing the arguments put forward by different sources and questions to prompt the use of integration strategies, by adapting the critical questions used by Nussbaum and Edwards (2011), as well as the steps in the writing process (planning, textualizing and revising).”
Research works printed but not linked: 4
- European Commission, Directorate-General for Education, Youth, Sport and Culture. (2019). Key competences for lifelong learning, Publications Office. https://doi.org/10.2766/569540 · doi:10.2766/569540
- Klein, P. D., Arcon, N., & Baker, S. (2016). Writing to learn. In C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (pp. 243–256). Guilford Press.
- Sundeen, T. H. (2014). Essay development for secondary students with learning disabilities: Graphic organizers for visualizing organizational patterns. Rural Special Education Quarterly, 33(3), 29–36. https://doi.org/10.1177/8756870514033003 · doi:10.1177/8756870514033003
- Szigriszt Pazos, F. (2001). Sistemas predictivos de legilibilidad del mensaje escrito: fórmula de perspicuidad [Predictive systems of readability of the written message: perspicuity formula]. Universidad Complutense de Madrid, Servicio de Publicaciones.
Cited by
Aids for argumentative synthesis processes and texts for undergraduate students
Mateos, M., Olmos, R., Granado-Peinado, M., Vandermeulen, N., & Cuevas, I. M. A. (2026) · Reading and Writing · doi:10.1007/s11145-026-10774-9
Self-citation
Metadata
- Authors
- Cuevas, I., Mateos, M., Casado-Ledesma, L., Olmos, R., Granado-Peinado, M., Luna, M., Núñez, J. A., & Martín, E.
- Affiliations
- Universidad Autónoma de Madrid (ES)
- University of Florence (IT)
- Universidad Francisco de Vitoria (ES)
- Universidad a Distancia de Madrid (ES)
- Corresponding author
- Cuevas, Isabel
- Mateos, Mar
- Casado-Ledesma, Lidia
- Olmos, Ricardo
- Granado-Peinado, Miriam
- Luna, María
- ORCID
- Venue
- European Journal of Psychology of Education
- Published
- 2024-07-29
- Volume
- 39
- Issue
- 4
- Pages
- 4573-4596
- ISSN
- 0256-2928 (ISSN-L)
- 1878-5174
- OpenAlex
- W4401075492
- Cited by
- 1
- References
- 42
- Citations by year
- 2026: 1
- FWCI
- 1.26
- Type
- journal article
- Language
- en
- Open access
- hybrid
- dx.doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- European Journal of Psychology of Education (publishedVersion, cc-by)
- Biblos-e Archivo (Universidad Autónoma de Madrid) (submittedVersion, cc-by)
- Hispana (submittedVersion, cc-by)
- Funders
- Ministerio de Ciencia e Innovación (PID2019-105250RB-I00)
- Università degli Studi di Firenze
- Topics
- Social Sciences › Education › Student Assessment and Feedback
- Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
- Social Sciences › Psychology › Developmental and Educational Psychology › Educational Strategies and Epistemologies
- Keywords
- writing guidelines, explicit instruction, argument selection, undergraduate writing, writing strategies, argumentative writing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: How to improve argumentative syntheses written by undergraduates using guides and instructional rubrics. https://app.cochat.ai/papers/10.1007/s10212-024-00890-x
- The paper
- Cuevas, I., Mateos, M., Casado-Ledesma, L., Olmos, R., Granado-Peinado, M., Luna, M., Núñez, J. A., & Martín, E. (2024). How to improve argumentative syntheses written by undergraduates using guides and instructional rubrics. European Journal of Psychology of Education. https://doi.org/10.1007/s10212-024-00890-x
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.