Construct Measurement and Validation Procedures in MIS and Behavioral Research: Integrating New and Existing Techniques1
Abstract
Despite the fact that validating the measures of constructs is critical to building cumulative knowledge in MIS and the behavioral sciences, the process of scale development and validation continues to be a challenging activity. Undoubtedly, part of the problem is that many of the scale development procedures advocated in the literature are limited by the fact that they (1) fail to adequately discuss how to develop appropriate conceptual definitions of the focal construct, (2) often fail to properly specify the measurement model that relates the latent construct to its indicators, and (3) underutilize techniques that provide evidence that the set of items used to represent the focal construct actually measures what it purports to measure. Therefore, the purpose of the present paper is to integrate new and existing techniques into a comprehensive set of recommendations that can be used to give researchers in MIS and the behavioral sciences a framework for developing valid measures. First, we briefly elaborate upon some of the limitations of current scale development practices. Following this, we discuss each of the steps in the scale development process while paying particular attention to the differences that are required when one is attempting to develop scales for constructs with formative indicators as opposed to constructs with reflective indicators. Finally, we discuss several things that should be done after the initial development of a scale to examine its generalizability and to enhance its usefulness.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The study covered the scale development phases conceptualization, development of measures, model specification, and scale evaluation and refinement suggested by MacKenzie et al. (2011).
“With these steps, we cover the scale development phases conceptualization, development of measures, model specification, as well as scale evaluation and refinement suggested by MacKenzie et al. (2011).”
From Trust in the Sharing Economy (2016) · cites this paper as MacKenzie et al. 2011
The largest correlation was 0.49, less than recommended 0.71.
“Second, we observed the correlation matrix in Table 3, finding that the largest correlation was 0.49, less than recommended 0.71 (Mackenzie et al 2011).”
From Trust, Satisfaction, and Online Repurchase Intention: The Moderating Role of Perceived Effectiveness of E-Commerce Institutional Mechanisms1 (2014) · cites this paper as Mackenzie et al 2011
Internal consistency reliabilities (ICR) for each latent variable exceeded 0.70, suggesting good reliability.
“Table 3 showed that internal consistency reliabilities (ICR) for each of the latent variables exceeded 0.70, suggesting good reliability (MacKenzie, Podsakoff, and Podsakoff 2011).”
From Trust, Satisfaction, and Online Repurchase Intention: The Moderating Role of Perceived Effectiveness of E-Commerce Institutional Mechanisms1 (2014) · cites this paper as MacKenzie, Podsakoff, and Podsakoff 2011
Thresholds for multicollinearity are typically set below 10, but Petter et al. recommend a more restrictive cutoff of 3.3.
“The thresholds for multicollinearity are typically set at below values of 10 [18], however, Petter et al. [123] recommend a more restrictive cutoff value of 3.3.”
The survey instrument was developed following the guidelines suggested by MacKenzie et al.
“To ensure that the developed survey instrument was valid and robust, we followed the guidelines suggested by MacKenzie et al. [18].”
For formative constructs, it is important to test for multicollinearity and examine weights, loadings, and significance levels.
“Specifically, they highlight that it is important to test for multicollinearity, and examine weights, loadings, and significance levels.”
Cites
Evaluating Structural Equation Models with Unobservable Variables and Measurement Error
Fornell, C., & Larcker, D. F. (1981) · Journal of Marketing Research · doi:10.2307/3151312 · Cited by 109,314 · see citing works
Evaluating structural equation models with unobservable variables and measurement error
Fornell, C., & Larcker, D. F. (1981) · Journal of Marketing Research · doi:10.1177/002224378101800104 · Cited by 109,190 · see citing works
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788 · see citing works
Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.5.879 · Cited by 82,194 · see citing works
Self-citationPerceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483 · see citing works
Structural equation modeling in practice: A review and recommended two-step approach
Anderson, J. C., & Gerbing, D. W. (1988) · Psychological Bulletin · doi:10.1037/0033-2909.103.3.411 · Cited by 41,941 · see citing works
Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116 · see citing works
A Paradigm for Developing Better Measures of Marketing Constructs
Churchill, G. A. (1979) · Journal of Marketing Research · doi:10.1177/002224377901600110 · Cited by 16,549 · see citing works
Asymptotic Confidence Intervals for Indirect Effects in Structural Equation Models
Sobel, M. E. (1982) · Sociological Methodology · doi:10.2307/270723 · Cited by 12,810 · see citing works
Construct validity in psychological tests.
Cronbach, L. J., & Meehl, P. E. (1955) · Psychological Bulletin · doi:10.1037/h0040957 · Cited by 11,195 · see citing works
What is coefficient alpha? An examination of theory and applications
Cortina, J. M. (1993) · Journal of Applied Psychology · doi:10.1037/0021-9010.78.1.98 · Cited by 8,810 · cited 1× · see citing works
“However, it is widely known that alpha increases as the number of measures increases, so a higher value may be appropriate in cases where the number of measures is large (Cortina 1993).”
BackgroundA QUANTITATIVE APPROACH TO CONTENT VALIDITY<sup>1</sup>
LAWSHE, C. H. (1975) · Personnel Psychology · doi:10.1111/j.1744-6570.1975.tb01393.x · Cited by 7,313 · see citing works
Constructing validity: Basic issues in objective scale development.
Clark, L. A., & Watson, D. (1995) · Psychological Assessment · doi:10.1037/1040-3590.7.3.309 · Cited by 6,567 · see citing works
Introduction to Statistical Mediation Analysis
MacKinnon, D. P. (2012) · doi:10.4324/9780203809556 · Cited by 5,618 · see citing works
Sample size in factor analysis.
MacCallum, R. C., Widaman, K. F., Zhang, S., & Hong, S. (1999) · Psychological Methods · doi:10.1037/1082-989x.4.1.84 · Cited by 5,497 · see citing works
Bharadwaj, A. S. (2000) · MIS Quarterly · doi:10.2307/3250983 · Cited by 5,042 · see citing works
Diamantopoulos, A., & Siguaw, J. A. (2006) · British Journal of Management · doi:10.1111/j.1467-8551.2006.00500.x · Cited by 3,410 · see citing works
A Review of Scale Development Practices in the Study of Organizations
Hinkin, T. R. (1995) · Journal of Management · doi:10.1177/014920639502100509 · Cited by 3,193 · see citing works
Content validity in psychological assessment: A functional approach to concepts and methods.
Haynes, S. N., Richard, D. C. S., & Kubany, E. S. (1995) · Psychological Assessment · doi:10.1037//1040-3590.7.3.238 · Cited by 2,387 · cited 1× · see citing works
“These items may come from a variety of sources (see Churchill 1979; Haynes et al. 1995; Nunnally and Bernstein 1994), including reviews of the literature, deduction from the theoretical definition of the construct, previous theoretical and empirical research on the focal construct, suggestions from…”
BackgroundBarrick, M. R., Mount, M. K., & Judge, T. A. (2001) · International Journal of Selection and Assessment · doi:10.1111/1468-2389.00160 · Cited by 2,356 · see citing works
Methods of coping with social desirability bias: A review
Nederhof, A. J. (1985) · European Journal of Social Psychology · doi:10.1002/ejsp.2420150303 · Cited by 2,340 · cited 1× · see citing works
“Finally, efforts should also be made to refine or remove items that contain obvious social desirability (see Nederhof 1985).”
BackgroundConsumers' Need for Uniqueness: Scale Development and Validation
Tian, K. T., Bearden, W. O., & Hunter, G. L. (2001) · Journal of Consumer Research · doi:10.1086/321947 · Cited by 1,791 · see citing works
Using the implicit association test to measure self-esteem and self-concept
Greenwald, A. G., & Farnham, S. D. (2000) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.79.6.1022 · Cited by 1,393 · see citing works
Multidimensional constructs in organizational behavior research: an integrative analytical framework
Edwards, J. R. (2001) · Organizational Research Methods · doi:10.1177/109442810142004 · Cited by 1,128 · see citing works
Personality measurement and employment decisions: Questions and answers.
Hogan, R., Hogan, J., & Roberts, B. W. (1996) · American Psychologist · doi:10.1037/0003-066x.51.5.469 · Cited by 572 · see citing works
On the Meaning of within-Factor Correlated Measurement Errors
Gerbing, D. W., & Anderson, J. C. (1984) · Journal of Consumer Research · doi:10.1086/208993 · Cited by 526 · cited 2× · see citing works
“(2003), MacKenzie et al. (2005), and Petter et al. (2007) suggest that structural parameter estimates can be biased when indicators that should be modeled as having formative relationships with a construct are modeled as having reflective relationships.”
Background
Flagged first, then by citations
Cited by
Sorted by citations
Sources of method bias in social science research and recommendations on how to control it
Podsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. (2012) · Annual Review of Psychology · doi:10.1146/annurev-psych-120710-100452 · Cited by 15,744 · see citing works
Self-citationAn assessment of the use of partial least squares structural equation modeling in marketing research
Hair, J. F., Sarstedt, M., Ringle, C. M., & Mena, J. A. (2012) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-011-0261-6 · Cited by 8,088 · see citing works
Boateng, G. O., Neilands, T. B., Frongillo, E. A., Melgar-Quiñonez, H. R., & Young, S. L. (2018) · Frontiers in Public Health · doi:10.3389/fpubh.2018.00149 · Cited by 5,925 · see citing works
Hair, J. F., Ringle, C. M., & Sarstedt, M. (2013) · Long Range Planning · doi:10.1016/j.lrp.2013.01.001 · Cited by 5,310 · see citing works
Common Method Bias in Marketing: Causes, Mechanisms, and Procedural Remedies
MacKenzie, S. B., & Podsakoff, P. M. (2012) · Journal of Retailing · doi:10.1016/j.jretai.2012.08.001 · Cited by 3,051 · see citing works
Self-citationBecker, J.-M., Klein, K., & Wetzels, M. (2012) · Long Range Planning · doi:10.1016/j.lrp.2012.10.001 · Cited by 2,279 · see citing works
Mikalef, P., & Gupta, M. (2021) · Information & Management · doi:10.1016/j.im.2021.103434 · Cited by 1,700 · see citing works
Toward the development of a big data analytics capability
Gupta, M., & George, J. F. (2016) · Information & Management · doi:10.1016/j.im.2016.07.004 · Cited by 1,445 · see citing works
Mikalef, P., Krogstie, J., Pappas, I. O., & Pavlou, P. (2020) · Information & Management · doi:10.1016/j.im.2019.05.004 · Cited by 1,072 · see citing works
“Next, to evaluate the validity of the items of formative constructs, we followed MacKenzie et al. [137] and Schmiedel et al. [138] guidelines using Edwards [139] adequacy coefficient (R 2 a).”
MethodHulland, J., Baumgartner, H., & Smith, K. M. (2018) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-017-0532-y · Cited by 1,067 · see citing works
Chao, C.-M. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.01652 · Cited by 1,041 · see citing works
Mikalef, P., Boura, M., Lekakos, G., & Krogstie, J. (2019) · British Journal of Management · doi:10.1111/1467-8551.12343 · Cited by 835 · see citing works
Fang, Y., Qureshi, I., Sun, H., McCole, P., Ramsey, E., & Lim, K. H. (2014) · MIS Quarterly · doi:10.25300/misq/2014/38.2.04 · Cited by 770 · see citing works
Gabriel, A. S., Podsakoff, N. P., Beal, D. J., Scott, B. A., Sonnentag, S., Trougakos, J. P., & Butts, M. M. (2019) · Organizational Research Methods · doi:10.1177/1094428118802626 · Cited by 755 · see citing works
Self-citationChowdhury, S., Budhwar, P., Dey, P. K., Joel-Edgar, S., & Abadie, A. (2022) · Journal of Business Research · doi:10.1016/j.jbusres.2022.01.069 · Cited by 531 · see citing works
Fast, N. J., Burris, E. R., & Bartel, C. A. (2014) · Academy of Management Journal · doi:10.5465/amj.2012.0393 · Cited by 440 · see citing works
Kehr, F., Kowatsch, T., Wentzel, D., & Fleisch, E. (2015) · Information Systems Journal · doi:10.1111/isj.12062 · Cited by 410 · see citing works
Ng, D. T. K., Wu, W., Leung, J. K. L., Chiu, T. K. F., & Chu, S. K. W. (2024) · British Journal of Educational Technology · doi:10.1111/bjet.13411 · Cited by 390 · see citing works
Mobile health technology adoption across generations: Narrowing the digital divide
Fox, G., & Connolly, R. (2018) · Information Systems Journal · doi:10.1111/isj.12179 · Cited by 344 · see citing works
Mikalef, P., & Krogstie, J. (2020) · European Journal of Information Systems · doi:10.1080/0960085x.2020.1740618 · Cited by 331 · see citing works
Brill, T. M., Munoz, L., & Miller, R. J. (2019) · Journal of Marketing Management · doi:10.1080/0267257x.2019.1687571 · Cited by 291 · see citing works
The role of information governance in big data analytics driven innovation
Mikalef, P., Boura, M., Lekakos, G., & Krogstie, J. (2020) · Information & Management · doi:10.1016/j.im.2020.103361 · Cited by 260 · see citing works
Algorithmic control and gig workers: a legitimacy perspective of Uber drivers
Wiener, M., Cram, W. A., & Benlian, A. (2023) · European Journal of Information Systems · doi:10.1080/0960085x.2021.1977729 · Cited by 251 · see citing works
Examining the Impact of Algorithmic Control on Uber Drivers’ Technostress
Cram, W. A., Wiener, M., Tarafdar, M., & Benlian, A. (2022) · Journal of Management Information Systems · doi:10.1080/07421222.2022.2063556 · Cited by 246 · see citing works
Daassi, M., & Debbabi, S. (2021) · Information & Management · doi:10.1016/j.im.2021.103453 · Cited by 239 · see citing works
Business model innovativeness: designing a formative measure for business model innovation
Spieth, P., & Schneider, S. (2016) · Journal of Business Economics · doi:10.1007/s11573-015-0794-0 · Cited by 232 · see citing works
Benlian, A., Hilkert, D., & Hess, T. (2015) · Journal of Information Technology · doi:10.1057/jit.2015.6 · Cited by 212 · see citing works
Hawlitschek, F., Teubner, T., & Weinhardt, C. (2016) · Die Unternehmung · doi:10.5771/0042-059x-2016-1-26 · Cited by 217 · see citing works
Fisher, G., Stevenson, R., Neubert, E., Burnell, D., & Kuratko, D. F. (2020) · Journal of Management Studies · doi:10.1111/joms.12584 · Cited by 200 · see citing works
O’Leary, M. B., Wilson, J. M., & Metiu, A. (2014) · MIS Quarterly · doi:10.25300/misq/2014/38.4.13 · Cited by 191 · see citing works
An international systematic review of cyberbullying measurements
Chun, J., Lee, J., Kim, J., & Lee, S. (2020) · Computers in Human Behavior · doi:10.1016/j.chb.2020.106485 · Cited by 171 · see citing works
Chen, X., Huang, Q., Davison, R. M., & Hua, Z. (2015) · International Journal of Electronic Commerce · doi:10.1080/10864415.2016.1087828 · Cited by 156 · see citing works
Jussupow, E., Spohrer, K., & Heinzl, A. (2022) · JMIR Formative Research · doi:10.2196/28750 · Cited by 138 · see citing works
Informal workplace learning: development and validation of a measure
Decius, J., Schaper, N., & Seifert, A. (2019) · Human Resource Development Quarterly · doi:10.1002/hrdq.21368 · Cited by 127 · see citing works
Duty orientation: Theoretical development and preliminary construct testing
Hannah, S. T., Jennings, P. L., Bluhm, D., Peng, A. C., & Schaubroeck, J. M. (2014) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2013.10.007 · Cited by 117 · see citing works
Do the Hustle! Empowerment from Side-Hustles and Its Effects on Full-Time Work Performance
Sessions, H., Nahrgang, J. D., Vaulont, M. J., Williams, R., & Bartels, A. L. (2021) · Academy of Management Journal · doi:10.5465/amj.2018.0164 · Cited by 116 · see citing works
Liu, Y., & Tang, X. (2018) · Information Technology & People · doi:10.1108/itp-10-2016-0242 · Cited by 111 · see citing works
Costa, S., Ingoglia, S., Inguglia, C., Liga, F., Lo Coco, A., & Larcan, R. (2018) · Measurement and Evaluation in Counseling and Development · doi:10.1080/07481756.2017.1347021 · Cited by 101 · see citing works
Hazzam, J., & Lahrech, A. (2018) · Journal of Medical Internet Research · doi:10.2196/12035 · Cited by 97 · see citing works
AI Literacy - Towards Measuring Human Competency in Artificial Intelligence
Pinski, M., & Benlian, A. (2023) · Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.24251/hicss.2023.021 · Cited by 94 · see citing works
The impact of eudaimonic well-being on experience and loyalty: a tourism context
Al-okaily, N. S., Alzboun, N., Alrawadieh, Z., & Slehat, M. (2023) · Journal of Services Marketing · doi:10.1108/jsm-08-2021-0282 · Cited by 59 · see citing works
Curbing shopping cart abandonment in C2C markets — an uncertainty reduction approach
Tang, H., & Lin, X. (2019) · Electronic Markets · doi:10.1007/s12525-018-0313-6 · Cited by 56 · see citing works
Growing with smart products: Why customization capabilities matter for manufacturing firms
Schulz, C., Kortmann, S., Piller, F. T., & Pollok, P. (2023) · Journal of Product Innovation Management · doi:10.1111/jpim.12680 · Cited by 49 · see citing works
Single-case experimental research designs.
Kazdin, A. E. (2015) · Methodological issues and strategies in clinical research (4th ed.). · doi:10.1037/14805-029 · Cited by 39 · see citing works
Yankouskaya, A., Babiker, A., Rizvi, S., Alshakhsi, S., Liebherr, M., & Ali, R. (2025) · ACM Transactions on the Web · doi:10.1145/3765895 · Cited by 32 · see citing works
Sebastian, G. (2026) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2026.124697 · Cited by 10 · see citing works
- Authors
- MacKenzie, S. B., Podsakoff, P. M., & Podsakoff, N. P.
- Affiliations
- Indiana University Bloomington (US)
- University of Arizona (US)
- Venue
- MIS Quarterly
- Publisher
- MIS Quarterly
- Published
- 2011-06-01
- Volume
- 35
- Issue
- 2
- Pages
- 293-A5
- ISSN
- 0276-7783 (ISSN-L)
- 2162-9730
- OpenAlex
- W3126055879
- Cited by
- 3,165
- References
- 26
- Citations by year
- 2026: 366 · 2025: 270 · 2024: 289 · 2023: 274 · 2022: 267 · 2021: 308 · 2020: 232 · 2019: 237 · 2018: 185 · 2017: 168 · 2016: 138 · 2015: 186 · 2014: 111 · 2013: 82 · 2012: 40
- FWCI
- 105.25
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- MIS Quarterly (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Management Science and Operations Research › Business Strategies and Management Research
- Social Sciences › Education › Health Education and Validation
- Keywords
- scale development, construct measurement, scale validation, formative indicators, reflective indicators, measurement model, conceptual definitions, construct validity, generalizability, behavioral research, information systems research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
