Evaluating Individual Students' Perceptions of Instructional Quality: An Investigation of their Factor Structure, Measurement Invariance, and Relations to Educational Outcomes
Open accessThe DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
Students' perceptions of instructional quality are among the most important criteria for evaluating teaching effectiveness. The present study evaluates different latent variable modeling approaches (confirmatory factor analysis, exploratory structural equation modeling, and bifactor modeling), which are used to describe these individual perceptions with respect to their factor structure, measurement invariance, and the relations to selected educational outcomes (achievement, self-concept, and motivation in mathematics). On the basis of the Programme for International Student Assessment (PISA) 2012 large-scale data sets of Australia, Canada, and the USA (N = 26,746 students), we find support for the distinction between three factors of individual students' perceptions and full measurement invariance across countries for all modeling approaches. In this regard, bifactor exploratory structural equation modeling outperformed alternative approaches with respect to model fit. Our findings reveal significant relations to the educational outcomes. This study synthesizes different modeling approaches of individual students' perceptions of instructional quality and provides insights into the nature of these perceptions from an individual differences perspective. Implications for the measurement and modeling of individually perceived instructional quality are discussed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance
Cheung, G. W., & Rensvold, R. B. (2002) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem0902_5 · Cited by 16,280
Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance
Chen, F. F. (2007) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705510701301834 · Cited by 12,217
Motivational beliefs, values, and goals
Eccles, J. S., & Wigfield, A. (2002) · Annual Review of Psychology · doi:10.1146/annurev.psych.53.100901.135153 · Cited by 7,587
Rhemtulla, M., Brosseau-Liard, P. É., & Savalei, V. (2012) · Psychological Methods · doi:10.1037/a0029315 · Cited by 2,816 · cited 2×
“Robust maximum likelihood estimation (MLR) with standard errors and tests of fit that were robust against non-normality of item responses and the use of categorical variables in the presence of at least four response categories was used (Rhemtulla et al., 2012).”
MethodThe Rediscovery of Bifactor Measurement Models
Reise, S. P. (2012) · Multivariate Behavioral Research · doi:10.1080/00273171.2012.715555 · Cited by 2,112 · cited 5×
“…(ESEM) is a new modeling framework that specifically deals with cross-loadings (Marsh et al., 2009, 2014), bifactor modeling which is available both within the traditional CFA and the ESEM framework differentiates between general and specific factors of constructs (Chen et al., 2012; Reise, 2012).”
BackgroundMethodInfluentialTeachers’ Mathematical Knowledge, Cognitive Activation in the Classroom, and Student Progress
Baumert, J., Kunter, M., Blum, W., Brunner, M., Voss, T., Jordan, A., Klusmann, U., Krauss, S., Neubrand, M., & Tsai, Y.-M. (2010) · American Educational Research Journal · doi:10.3102/0002831209345157 · Cited by 2,100 · cited 4×
“This measurement issue also limits meaningful comparisons that could shed light on how teaching effectiveness relates to educational systems and teacher education in multilevel settings (Baumert et al., 2010; Scherer and Gustafsson, 2015).”
BackgroundInfluentialStatistical Approaches to Measurement Invariance
Millsap, R. E. (2012) · doi:10.4324/9780203821961 · Cited by 1,554
Academic self‐concept and academic achievement: Relations and causal ordering
Marsh, H. W., & Martin, A. J. (2011) · British Journal of Educational Psychology · doi:10.1348/000709910x503501 · Cited by 975 · cited 2×
“Students’ self-concept in mathematics refers to their beliefs in their abilities in mathematics and can be regarded as an important predictor of student achievement (Marsh, 1986; Marsh and Martin, 2011).”
BackgroundRutkowski, L., & Svetina, D. (2014) · Educational and Psychological Measurement · doi:10.1177/0013164413498257 · Cited by 767 · cited 3×
“Instead, we followed the recommendations given by Cheung and Rensvold (2002) and Rutkowski and Svetina (2014), and considered the changes of the incremental fit indices as practically insignificant if the CFI and TLI decreased less than 0.010, and the RMSEA and SRMR decreased less than 0.015,…”
BackgroundMethodInfluentialKlusmann, U., Kunter, M., Trautwein, U., Lüdtke, O., & Baumert, J. (2008) · Journal of Educational Psychology · doi:10.1037/0022-0663.100.3.702 · Cited by 508
Research works printed but not linked: 68
- Asparouhov, 2005, Sampling weights in latent variable modeling, Struct. Eq. Model., 12, 411, 10.1207/s15328007sem1203_4 · DOI · Paper page
- Asparouhov, 2009, Exploratory structural equation modeling, Struct. Eq. Model., 16, 397, 10.1080/10705510903008204 · DOI · Paper page
- Asparouhov, 2010, Weighted Least Squares Estimation with Missing Data
- Beauducel, 2006, On the performance of maximum likelihood versus means and variance adjusted weighted least square estimation in confirmatory factor analysis, Struct. Eq. Model., 13, 186, 10.1207/s15328007sem1302_2 · DOI · Paper page
- Blazar, 2015, Effective teaching in elementary mathematics: identifying classroom practices that support student achievement, Econ. Educ. Rev., 48, 16, 10.1016/j.econedurev.2015.05.005 · DOI
- Bulle, 2011, Comparing OECD educational models through the prism of PISA, Comp. Educ., 47, 503, 10.1080/03050068.2011.555117 · DOI
- Charalambous, 2014, Using generic and content-specific teaching practices in teacher evaluation: an exploratory study of teachers' perceptions, Teach. Teacher Educ., 41, 22, 10.1016/j.tate.2014.03.001 · DOI
- Chen, 2012, Modeling general and specific variance in multifaceted constructs: a comparison of the bifactor model to other approaches, J. Pers., 80, 219, 10.1111/j.1467-6494.2011.00739.x · DOI · Paper page
- Chung, 2015, Bifactor approach to modeling multi-dimensionality of physical self-perception profile, Measurem. Phys. Educat. Exercise Sci., 20, 1, 10.1080/1091367X.2015.1081594 · DOI
- Creemers, 2008, The Dynamics of Educational Effectiveness
- Desa, 2014, Evaluating Measurement Invariance of TALIS 2013 Complex Scales, 10.1787/5jz2kbbvlb7k-en · DOI
- Dietrich, 2015, Teacher support and its influence on students' intrinsic value and effort: dimensional comparison effects across subjects, Learn. Instruct., 39, 45, 10.1016/j.learninstruc.2015.05.007 · DOI · Paper page
- Dolan, 2009, Testing measurement invariance in the target rotated multigroup exploratory factor model, Struct. Eq. Model., 16, 295, 10.1080/10705510902751416 · DOI · Paper page
- Enders, 2010, Applied Missing Data Analysis
- Fauth, 2014, Student ratings of teaching quality in primary school: dimensions and prediction of student outcomes, Learn. Instruct., 29, 1, 10.1016/j.learninstruc.2013.07.001 · DOI · Paper page
- Greenwald, 1997, Validity concerns and usefulness of student ratings of instruction, Am. Psychol., 52, 1182, 10.1037/0003-066X.52.11.1182 · DOI · Paper page
- Grossman, 2013, The relationship between measures of instructional practice in middle school english language arts and teachers' value-added scores, Am. J. Educat., 119, 445, 10.1086/669901 · DOI · Paper page
- Guenole, 2014, The consequence of ignoring measurement invariance for path coefficients in structural equation models, Front. Psychol., 5, 10.3389/fpsyg.2014.00980 · DOI
- Gustafsson, 2010, Unidimensionality and interpretability of psychological instruments, 97
- Hattie, 2009, Visible Learning—A Synthesis of Meta-Analyses in Education
- He, 2013, Methodological issues in cross-cultural studies in educational psychology, 39
- Jak, 2014, Measurement bias in multilevel data, Struct. Equat. Model., 21, 31, 10.1080/10705511.2014.856694 · DOI · Paper page
- Jonassen, 2011, Supporting problem solving in PBL, Interdiscipl. J. Problem Based Learn., 5, 95, 10.7771/1541-5015.1256 · DOI · Paper page
- Klieme, 2013, The role of large-scale assessments in research on educational effectiveness and school development, 115, 10.1007/978-94-007-4629-9_7 · DOI
- Klieme, 2009, The Pythagoras study: Investigating effects of teaching and learning in Swiss and German mathematics classrooms, 137
- Kristen, 2005, Longitudinal Studies for Education Reports: European and North American Examples (Education Reform Volume 10)
- Kuenzi, 2009, Assembling fragments into a lens: a review, critique, and proposed research agenda for the organizational work climate literature, J. Manage., 35, 634, 10.1177/0149206308330559 · DOI · Paper page
- Kunter, 2007, Effective classroom management and the development of subject-related interest, Learn. Instruct., 17, 494, 10.1016/j.learninstruc.2007.09.002 · DOI · Paper page
- Kunter, 2008, Students' and mathematics teachers' perceptions of teacher enthusiasm and instruction, Learn. Instruct., 18, 468, 10.1016/j.learninstruc.2008.06.008 · DOI · Paper page
- Lazarides, 2012, Instructional quality and attitudes toward mathematics: do self-concept and interest differ across students' patterns of perceived instructional quality in mathematics classrooms?, Child Dev. Res., 2012, 813920, 10.1155/2012/813920 · DOI
- Little, 2013, Longitudinal Structural Equation Modeling
- Lipowsky, 2009, Quality of geometry instruction and its short-term impact on students' understanding of the Pythagorean Theorem, Learn. Instruct., 19, 527, 10.1016/j.learninstruc.2008.11.001 · DOI · Paper page
- Lüdtke, 2009, Assessing the impact of learning environments: how to use student ratings of classroom or school characteristics in multilevel modeling, Contemp. Educ. Psychol., 34, 77, 10.1016/j.cedpsych.2008.12.001 · DOI · Paper page
- Marsh, 1986, Verbal and math self-concepts: an internal/external frame of reference model, Am. Educ. Res. J., 23, 129, 10.3102/00028312023001129 · doi:10.3102/00028312023001129 · Paper page
- Marsh, 2013a, Factorial, convergent, and discriminant validity of TIMSS math and science motivation measures: a comparison of Arab and Anglo-Saxon countries, J. Educ. Psychol., 105, 108, 10.1037/a0029907 · DOI · Paper page
- Marsh, 1997, Making students' evaluations of teaching effectiveness effective, Am. Psychol., 52, 1187, 10.1037/0003-066X.52.11.1187 · doi:10.1037/0003-066x.52.11.1187 · Paper page
- Marsh, 2005, Goodness of fit evaluation in structural equation modeling, 275
- Marsh, 2012, Classroom climate and contextual effects: conceptual and methodological issues in the evaluation of group-level effects, Educ. Psychol., 47, 106, 10.1080/00461520.2012.670488 · DOI · Paper page
- Marsh, 2014, Exploratory structural equation modeling: an integration of the best features of exploratory and confirmatory factor analysis, Annu. Rev. Clin. Psychol., 10, 85, 10.1146/annurev-clinpsy-032813-153700 · DOI · Paper page
- Marsh, 2009, Exploratory structural equation modeling, integrating CFA and EFA: application to students' evaluations of university teaching, Struct. Equat. Model., 16, 439, 10.1080/10705510903008220 · DOI · Paper page
- Marsh, 2013b, Measurement invariance of big-five factors over the life span: ESEM tests of gender, age, plasticity, maturity, and la dolce vita effects, Dev. Psychol., 49, 1194, 10.1037/a0026913 · DOI · Paper page
- Martin, 2013, Effective schools in reading, mathematics, and science at fourth grade, 109
- McCoach, 2013, Instrument Development in the Affective Domain, 10.1007/978-1-4614-7135-6 · DOI · Paper page
- MET Project, 2013, Ensuring Fair and Reliable Measures of Effective Teaching (Policy and Practice Brief)
- Morin, 2016, A bifactor exploratory structural equation modeling framework for the identification of distinct sources of construct-relevant psychometric multidimensionality, Struct. Equat. Model., 23, 116, 10.1080/10705511.2014.961800 · DOI · Paper page
- Morin, 2014, Doubly latent multilevel analyses of classroom climate: an illustration, J. Experim. Educat., 82, 143, 10.1080/00220973.2013.769412 · DOI · Paper page
- Murray, 1938, Explorations in Personality
- Muthén, 1998–2014, Mplus 7.3 [Computer Software]
- OECD, 2013a, PISA 2012 Results: Ready to Learn: Students' Engagement, Drive and Self-Beliefs (Volume III), 10.1787/9789264201170-en · DOI · Paper page
- OECD, 2013b, PISA 2012 Assessment and Analytical Framework, 10.1787/9789264190511-en · DOI · Paper page
- OECD, 2014a, PISA 2012 Results: What Students Know and Can Do, 10.1787/6341a959-en · DOI · Paper page
- OECD, 2014b, PISA 2012 Technical Report, 10.1787/6341a959-en · DOI · Paper page
- Parker, 2003, Relationships between psychological climate perceptions and work outcomes: a meta-analytic review, J. Organ. Behav., 24, 389, 10.1002/job.198 · DOI · Paper page
- Parker, 2014, Predicting career aspirations and university majors from academic ability and self-concept, 224, 10.1017/CBO9781139128933.015 · DOI
- Perera, 2015, Construct validity of the social provisions scale: a bifactor exploratory structural equation modeling approach, Assessment, 10.1177/1073191115589344 · DOI
- Roche, 2002, Teaching self-concept in higher education, 179, 10.1007/978-94-010-0593-7_9 · DOI
- Satorra, 2010, Ensuring positiveness of the scaled difference chi-square test statistic, Psychometrika, 75, 243, 10.1007/s11336-009-9135-y · DOI · Paper page
- Scherer, 2015, Student assessment of teaching as a source of information about aspects of teaching quality in multiple subject domains: an application of multilevel bifactor structural equation modeling, Front. Psychol., 6, 10.3389/fpsyg.2015.01550 · DOI
- Seidel, 2007, Teaching effectiveness research in the past decade: the role of theory and research design in disentangling meta-analysis results, Rev. Educ. Res., 77, 454, 10.3102/0034654307310317 · DOI · Paper page
- Stenling, 2015, Using bifactor exploratory structural equation modeling to examine global and specific factors in measures of sports coaches' interpersonal styles, Front. Psychol., 6, 10.3389/fpsyg.2015.01303 · DOI · Paper page
- Stern, 1970, People in Context. Measuring Person–Environment Congruence in Education and Industry
- van de Schoot, 2013, Facing off with Scylla and Charybdis: a comparison of scalar, partial, and the novel possibility of approximate measurement invariance, Front. Psychol., 4, 10.3389/fpsyg.2013.00700 · DOI · Paper page
- Van Tartwijk, 2011, The neglected role of classroom management in teacher education, Teach. Educat., 22, 109, 10.1080/10476210.2001.567836 · doi:10.1080/10476210.2001.567836
- von Davier, 2009, What are plausible values and why are they useful?, 9
- Wagner, 2013, Construct validity of student perceptions of instructional quality is high, but not perfect: dimensionality and generalizability of domain-independent assessments, Learn. Instruct., 28, 1, 10.1016/j.learninstruc.2013.03.003 · DOI · Paper page
- Wiesner, 2013, Exploratory structural equation modeling, bifactor models, and standard confirmatory factor analysis models: application to the BASC-2 Behavioral and Emotional Screening System Teacher Form, J. Sch. Psychol., 51, 751, 10.1016/j.jsp.2013.09.001 · DOI
- Wu, 2005, The role of plausible values in large-scale surveys, Studies Educat. Eval., 31, 114, 10.1016/j.stueduc.2005.05.005 · DOI · Paper page
- Yang, 2011, Coefficient alpha: a reliability coefficient for the 21st century?, J. Psychoeduc. Assess., 29, 377, 10.1177/0734282911406668 · DOI · Paper page
Cited by
Utami, S. P. T., Andayani, A., Winarni, R., & Sumarwati, S. (2023) · Contemporary Educational Technology · doi:10.30935/cedtech/13419 · Cited by 124
“Students ’ perceptions are practical tools which determine the students ’ behavior (Scherer et al., 2016).”
Aldrup, K., Klusmann, U., Lüdtke, O., Göllner, R., & Trautwein, U. (2018) · Journal of Educational Psychology · doi:10.1037/edu0000256 · Cited by 109
“Some studies found a positive association, while others revealed a negative relationship or no link at all (den Brok et al., 2004; Scherer et al., 2016; Wagner et al., 2016).”
BackgroundInfluentialBelay, S., & Melesse, T. (2024) · SAGE Open · doi:10.1177/21582440241281855 · Cited by 28
“Scherer et al. (2016) complemented that the teaching quality of teachers is one important contributor to enhancing students’ learning results.”
Metadata
- Authors
- Scherer, R., Nilsen, T., & Jansen, M.
- Venue
- Frontiers in Psychology
- Publisher
- Frontiers Media SA
- Published
- 2016-02-08
- Volume
- 7
- Pages
- 110-110
- ISSN
- 1664-1078 (ISSN-L)
- OpenAlex
- W2280568119
- Cited by
- 126
- References
- 17
- FWCI
- 8.39
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Evaluating Individual Students' Perceptions of Instructional Quality: An Investigation of their Factor Structure, Measurement Invariance, and Relations to Educational Outcomes. https://app.cochat.ai/papers/10.3389/fpsyg.2016.00110
- The paper
- Scherer, R., Nilsen, T., & Jansen, M. (2016). Evaluating Individual Students' Perceptions of Instructional Quality: An Investigation of their Factor Structure, Measurement Invariance, and Relations to Educational Outcomes. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2016.00110
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.