The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
An Experimental Comparison of Different Approaches to Determining Weights in Additive Utility Models
Abstract
Several studies this past decade have examined differences between holistic and decomposed approaches to determining weights in additive utility models. Some have argued that it matters little which procedure is used, whereas others strongly favored particular methods. In this paper we address this controversy experimentally by comparing five conceptually different approaches in terms of their weights and predictive ability. The five methods are (1) multiple linear and non-linear regression analyses of ten and fifteen holistic assessments, (2) direct decomposed tradeoffs as proposed by Keeney and Raiffa (Keeney, R. L., H. Raiffa. 1977. Decisions with Multiple Objectives. Wiley, New York.), (3) a recent eigen-vector technique of Saaty (Saaty, T. L. 1977. A scaling method for priorities in hierarchical structures. J. Math. Psych. 15 (3) 234–281.) involving redundant pairwise comparisons of attributes, (4) a straightforward allocation of hundred importance points, and (5) unit weighting (i.e., equal weighting after standardizing the attributes). The decision task involved college admissions. Subjects were asked to evaluate hypothetical college applicants on the basis of verbal SAT, quantitative SAT, high-school grade point average, and a measure of extra-curricular activity. Linear as well as nonlinear attribute utility functions were used in constructing the additive models. The nonlinear functions were specified graphically by the subjects through selection from five different shapes (i.e., one per attribute). To test the predictive ability of the various models, each subject made twenty separate pairwise comparisons of alternatives (including direction and strength of preference). The prediction criteria were percentage correct predictions as well as correlations (using these twenty pairs). Seventy subjects were tested, using an (order-controlled) within-subject design, in comparing the different methods of weight determination. Monetary incentives were used to enhance motivation. In terms of findings, the methods generally differed systematically concerning the weights given to the various attributes, as well as the variances of the resulting predictions. On average, however, the methods predicted about equally well, except for unit weighting which was clearly inferior. The findings differ in this regard from the general literature. Furthermore, nonlinear models were found to be inferior to linear ones. Finally, subjects judged the methods to differ significantly in difficulty and trustworthiness, which were found to correlate inversely. The overall results raise various applied and theoretical issues, which are discussed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The magical number seven, plus or minus two: Some limits on our capacity for processing information.
Miller, G. A. (1994) · Psychological Review · doi:10.1037/0033-295x.101.2.343 · Cited by 17,161
A scaling method for priorities in hierarchical structures
Saaty, T. L. (1977) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(77)90033-5 · Cited by 10,360
Decisions with Multiple Objectives: Preferences and Value Trade-Offs
Keeney, R. L., & Raiffa, H. (1993) · Cambridge University Press eBooks · doi:10.1017/cbo9781139174084 · Cited by 4,426
Intransitivity of preferences.
Tversky, A. (1969) · Psychological Review · doi:10.1037/h0026750 · Cited by 2,188
The robust beauty of improper linear models in decision making.
Dawes, R. M. (1979) · American Psychologist · doi:10.1037/0003-066x.34.7.571 · Cited by 1,966
(1990) · Elsevier eBooks · doi:10.1016/c2009-0-21635-7 · Cited by 1,953
Linear models in decision making.
Dawes, R. M., & Corrigan, B. (1974) · Psychological Bulletin · doi:10.1037/h0037613 · Cited by 1,875
Comparison of Bayesian and regression approaches to the study of information processing in judgment
Slovic, P., & Lichtenstein, S. (1971) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(71)90033-x · Cited by 1,819
Multiple regression in psychological research and practice.
Darlington, R. B. (1968) · Psychological Bulletin · doi:10.1037/h0025471 · Cited by 1,132
How to Use Multiattribute Utility Measurement for Social Decisionmaking
Edwards, W. (1977) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1977.4309720 · Cited by 1,033
The paramorphic representation of clinical judgment.
Hoffman, P. J. (1960) · Psychological Bulletin · doi:10.1037/h0047807 · Cited by 698
Measurable Multiattribute Value Functions
Dyer, J. S., & Sarin, R. K. (1979) · Operations Research · doi:10.1287/opre.27.4.810 · Cited by 675
Unit weighting schemes for decision making
Einhorn, H. J., & Hogarth, R. M. (1975) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(75)90044-6 · Cited by 673
Estimating coefficients in linear models: It don't make no nevermind.
Wainer, H. (1976) · Psychological Bulletin · doi:10.1037//0033-2909.83.2.213 · Cited by 656
Goldberg, L. R. (1970) · Psychological Bulletin · doi:10.1037/h0029230 · Cited by 617
Analyzing the components of clinical inference.
Hammond, K. R., Hursch, C. J., & Todd, F. J. (1964) · Psychological Review · doi:10.1037/h0040736 · Cited by 406
Linear regression and process-tracing models of judgment.
Einhorn, H. J., Kleinmuntz, D. N., & Kleinmuntz, B. (1979) · Psychological Review · doi:10.1037//0033-295x.86.5.465 · Cited by 397
Mosteller, F. (1951) · Psychometrika · doi:10.1007/bf02313422 · Cited by 343
Eckenrode, R. T. (1965) · Management Science · doi:10.1287/mnsc.12.3.180 · Cited by 289
Weighting Systems for Linear Functions of Correlated Variables When There is no Dependent Variable
Wilks, S. S. (1938) · Psychometrika · doi:10.1007/bf02287917 · Cited by 272
Schmidt, F. L. (1971) · Educational and Psychological Measurement · doi:10.1177/001316447103100310 · Cited by 228
Multi-Attribute Utility Models: A Review of Field and Field-Like Studies
Huber, G. P. (1974) · Management Science · doi:10.1287/mnsc.20.10.1393 · Cited by 199
A COMPARISON OF WEIGHTING METHODS IN POWER PLANT SITING*
Hobbs, B. F. (1980) · Decision Sciences · doi:10.1111/j.1540-5915.1980.tb01173.x · Cited by 162
A comparison of seven methods for obtaining subjective descriptions of judgmental policy
Cook, R. L., & Stewart, T. R. (1975) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(75)90004-5 · Cited by 105
Fischer, G. W. (1977) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(77)90032-0 · Cited by 89
Multidimensional utility models for risky and riskless choice
Fischer, G. W. (1976) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(76)90057-x · Cited by 68
Newman, J. R. (1977) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(77)90010-1 · Cited by 42
Statistical and subjective weights. Some problems and proposals
Schmitt, N., & Levine, R. L. (1977) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(77)90041-1 · Cited by 41
Integration of Information in a Clinical Judgment Task, an Empirical Comparison of Six Models
Nystedt, L., & Magnusson, D. (1975) · Perceptual and Motor Skills · doi:10.2466/pms.1975.40.2.343 · Cited by 16
Behavioral Issues in Multiattribute Utility Modeling and Decision Analysis
Schoemaker, P. J. H. (1981) · Lecture notes in economics and mathematical systems · doi:10.1007/978-3-642-45527-8_29 · Cited by 10
Self-citation
Cited by
User Acceptance of Computer Technology: A Comparison of Two Theoretical Models
Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989) · Management Science · doi:10.1287/mnsc.35.8.982 · Cited by 25,709
The Analytic Hierarchy Process—A Survey of the Method and its Applications
Zahedi, F. (1986) · Interfaces · doi:10.1287/inte.16.4.96 · Cited by 1,329
Inter-company comparison using modified TOPSIS with objective weights
Deng, H., Yeh, C.-H., & Willis, R. J. (2000) · Computers & Operations Research · doi:10.1016/s0305-0548(99)00069-6 · Cited by 1,022
The Theory of Ratio Scale Estimation: Saaty's Analytic Hierarchy Process
Harker, P. T., & Vargas, L. G. (1987) · Management Science · doi:10.1287/mnsc.33.11.1383 · Cited by 866
Decision Quality Using Ranked Attribute Weights
Barron, F. H., & Barrett, B. E. (1996) · Management Science · doi:10.1287/mnsc.42.11.1515 · Cited by 673
Metadata
- Authors
- Schoemaker, P. J. H., & Waid, C. C.
- Venue
- Management Science
- Published
- 1982-02
- Volume
- 28
- Issue
- 2
- Pages
- 182-196
- ISSN
- 0025-1909 (ISSN-L)
- 1526-5501
- OpenAlex
- W2047021735
- Cited by
- 382
- References
- 30
- FWCI
- 20.58
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.