Construct and consequential validity for learning analytics based on trace data
The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
This article analyzes the concept of validity to set out key factors bearing on claims about validity in general and particularly regarding learning analytics. Because uses of trace data in learning analytics are increasing rapidly, specific consideration is given to reliability of trace data and their role in claiming validity for interpretations grounded on trace data. This analysis reveals the essential and inescapable role of theory in deciding what trace data should be gathered and how trace data can contribute to recommendations for improving learning, one main goal for generating and using learning analytics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
THEORY OF GENERALIZABILITY: A LIBERALIZATION OF RELIABILITY THEORY†
Cronbach, L. J., Rajaratnam, N., & Gleser, G. C. (1963) · British Journal of Statistical Psychology · doi:10.1111/j.2044-8317.1963.tb00206.x · Cited by 737
Standards of validity and the validity of standards in performance asessment
Messick, S. (1995) · Educational Measurement Issues and Practice · doi:10.1111/j.1745-3992.1995.tb00881.x · Cited by 697
Cognition and Metacognition within Self-Regulated Learning
Winne, P. H. (2017) · Handbook of Self-Regulation of Learning and Performance · doi:10.4324/9781315697048-3 · Cited by 274
Self-citationMinimizing the black box problem to enhance the validity of theories about instructional effects.
Winne, P. H. (1982) · Instructional Science · doi:10.1007/bf00120978 · Cited by 123
Self-citation
Cited by
Explainable Artificial Intelligence in education
Khosravi, H., Shum, S. B., Chen, G., Conati, C., Tsai, Y.-S., Kay, J., Knight, S., Martinez-Maldonado, R., Sadiq, S., & Gašević, D. (2022) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2022.100074 · Cited by 863
Towards investigating the validity of measurement of self-regulated learning based on trace data
Fan, Y., van der Graaf, J., Lim, L., Raković, M., Singh, S., Kilgour, J., Moore, J., Molenaar, I., Bannert, M., & Gašević, D. (2022) · Metacognition and Learning · doi:10.1007/s11409-022-09291-1 · Cited by 101
“The validity challenge in trace-based SRL measurement identified in this and previous studies (Saint et al., 2020a; Winne, 2020, 2014) may encourage more researchers to consider validity in the development of trace-based SRL measurement protocols.”
BackgroundMethodInfluentialGašević, D., Greiff, S., & Shaffer, D. W. (2022) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107304 · Cited by 82
Temporal Assessment of Self-Regulated Learning by Mining Students’ Think-Aloud Protocols
Lim, L., Bannert, M., van der Graaf, J., Molenaar, I., Fan, Y., Kilgour, J., Moore, J., & Gašević, D. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.749749 · Cited by 44
“However, the issue of validity and reliability of trace data remain a challenge (Winne, 2020).”
BackgroundDo Instrumentation Tools Capture Self-Regulated Learning?
van der Graaf, J., Lim, L., Fan, Y., Kilgour, J., Moore, J., Bannert, M., Gasevic, D., & Molenaar, I. (2021) · LAK21: 11th International Learning Analytics and Knowledge Conference · doi:10.1145/3448139.3448181 · Cited by 44
Metadata
- Authors
- Winne, P. H.
- Venue
- Computers in Human Behavior
- Publisher
- Elsevier BV
- Published
- 2020-11
- Volume
- 112
- Pages
- 106457
- ISSN
- 0747-5632 (ISSN-L)
- OpenAlex
- W3038090822
- Cited by
- 111
- References
- 4
- FWCI
- 13.43
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.