CoChat
Sign Up
Cited by 46Evaluation of Teaching PracticesEducational Assessment and PedagogyHigher Education Research Studies

Building the foundations for measuring learning gain in higher education: a conceptual framework and measurement instrument

Open access
Vermunt, Jan D., Ilie, Sonia, Vignoles, Anna·2018·Higher Education Pedagogies
DOI 10.1080/23752696.2018.1484672
Open this paper in CoChat — free accountOpen PDF

Abstract

In this paper, we set out the first step towards the measurement of learning gain in higher education by putting forward a conceptual framework for understanding learning gain that is relevant across disciplines. We then introduce the operationalisation of this conceptual framework into a new set of measurement tools. With the use of data from a large-scale survey of 11 English universities and over 4,500 students, we test the reliability and validity of the measurement instrument empirically. We find support in the data for the reliability of most of the measurement scales we put forward, as well as for the validity of the conceptual framework. Based on these results, we reflect on the conceptual framework and associated measurement tools in the context of at-scale deployment and the potential implications for policy and practice in higher education.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1080/23752696.2018.1484672

Cites

118 of 118 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • It remains uncertain to what extent the digital affordances of AI chatbots contribute to measurable learning gains, and there is a need to analyze and distinguish these learning gains and their assessment methods.

    “Lastly, it remains uncertain to what extent the digital affordances of AI chatbot contributes to measurable learning gains, and there is a need to analyze and distinguish these learning gains and their assessment methods (21, 22).”

    From Digital affordances of AI chatbots in nursing education: a systematic review of learning gains and gaps in the evidence (2026) · cites this paper as [22]

    The paper distinguishes learning gain from learning outcomes and discusses the difficulty of choosing measures for learning gain (Operationalising definitions and approaches to measurement, p. 6). It also proposes a framework and measurement instrument (Discussion and conclusions, p. 27). These passages support the need to distinguish and assess learning gains, but they do not address whether AI-chatbot affordances produce measurable gains. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • There is a lack of in-depth analysis of the similarities and differences in learning gains across different contexts, which raises new questions regarding how to meaningfully and practically assess cognitive, affective, and behavioral learning gains.

    “However, there is a lack of in-depth analysis of the similarities and differences in learning gains across different contexts, which raises new questions regarding how to meaningfully and practically assess cognitive, affective, and behavioral learning gains (22–24).”

    From Digital affordances of AI chatbots in nursing education: a systematic review of learning gains and gaps in the evidence (2026) · cites this paper as [22]

    Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 11

  1. Allen, M.D., & Allen, M. (1988). The goals of Universities . Milton Keynes: Open University Press.
  2. Bauer, K.W., & Liang, Q. (2003). The e ff ect of personality and precollege characteristics on fi rstyear activities and academic performance. Journal of College Student Development , 44 (3), 277 -290.
  3. Buchannan, A. (2015). Education and social moral epistemology. In H. Brighouse & M. McPherson (Eds.), The aims of higher education: Problems of morality and justice . Chicago: University of Chicago Press.
  4. Fryer, L.K., & Ainley, M. (2017). Supporting interest in a study domain: A longitudinal test of the interplay between interest, utility-value, and competence beliefs. Learning and Instruction . Corrected proof. Retrieved from https://doi.org/10.1016/j.learninstruc.2017.11.002 · DOI
  5. Kelley, K., & Lai, K. (2018). Con fi rmatory factor models: Power and accuracy for e ff ects of interest. In P. Irwing, T. Booth, & D. Hughes (Eds.), The Wiley handbook of psychometric testing: A multidisciplinary reference on survey, scale and test development (pp. 113 -139). Chichester: Wiley.
  6. Liu, O.L. (2011). Measuring value added in education: Conditions and caveats. Assessment & Evaluation in Higher Education , 36 (1), 81 -94.
  7. Marton, F., & Säljö, R. (1984). Approaches to learning. In F. Marton, D. Hounsell, & N. Entwistle (Eds.), The experience of learning (pp. 36 -55). Edinburgh: Scottish Academic Press.
  8. Mason-Apps, E. (2017). Portsmouth HEFCE learning gain project. Presentation at the Annual Teaching and Learning Conference, Portsmouth. Retrieved http://www.port.ac.uk/research/ learning-gain/about-the-project/
  9. OECD (2012c). AHELO feasibility study report. Volume 3: Value-added Measurement and the Conference proceedings. Retrieved from http://www.oecd.org/edu/skills-beyond-school/ AHELOFSReportVolume3.pdf
  10. Pascarella, E.T., & Terenzini, P.T. (2005). How college affects students: A third decade of research (Vol. 2.). San Francisco: Jossey-Bass.
  11. Verhine, R.E., Dantas, L.M.V., & Soares, J.F. (2006). Do Provão ao ENADE: uma análise comparativa dos exames nacionais utilizados no Ensino Superior Brasileiro. Retrieved from http://www.scielo.br/pdf/ensaio/v14n52/a02v1452.pdf

Cited by

Sorted by citations

Related

Authors
Vermunt, J. D., Ilie, S., & Vignoles, A.
Affiliations
University of Cambridge (GB)
Corresponding author
Vermunt, Jan D.
ORCID
Venue
Higher Education Pedagogies
Publisher
Informa UK Limited
Published
2018-01
Volume
3
Issue
1
Pages
266-301
ISSN
2375-2696 (ISSN-L)
DOI
10.1080/23752696.2018.1484672
OpenAlex
W2890112699
Cited by
46
References
118
Citations by year
2026: 6 · 2025: 6 · 2024: 5 · 2023: 5 · 2022: 5 · 2021: 7 · 2020: 8 · 2019: 2 · 2018: 1 · 2017: 1
FWCI
14.59
Type
journal article
Language
en
Open access
Licence
Locations
  • Higher Education Pedagogies (publishedVersion, cc-by)
  • TU/e Research Portal (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • Apollo (University of Cambridge) (submittedVersion, cc-by)
  • TU/e Research Portal (submittedVersion, other-oa)
  • Apollo (cc-by)
Funders
Higher Education Funding Council for England
Topics
  • Social Sciences › Education › Evaluation of Teaching Practices
  • Social Sciences › Education › Educational Assessment and Pedagogy
  • Social Sciences › Education › Higher Education Research Studies
Keywords
learning gains, higher education, conceptual framework, measurement instruments, learning outcomes, scale reliability, scale validity, student survey, higher education policy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review