The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Constructing Experimental Designs for Discrete-Choice Experiments: Report of the ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force
Open accessAbstract
Stated-preference methods are a class of evaluation techniques for studying the preferences of patients and other stakeholders. While these methods span a variety of techniques, conjoint-analysis methods-and particularly discrete-choice experiments (DCEs)-have become the most frequently applied approach in health care in recent years. Experimental design is an important stage in the development of such methods, but establishing a consensus on standards is hampered by lack of understanding of available techniques and software. This report builds on the previous ISPOR Conjoint Analysis Task Force Report: Conjoint Analysis Applications in Health-A Checklist: A Report of the ISPOR Good Research Practices for Conjoint Analysis Task Force. This report aims to assist researchers specifically in evaluating alternative approaches to experimental design, a difficult and important element of successful DCEs. While this report does not endorse any specific approach, it does provide a guide for choosing an approach that is appropriate for a particular study. In particular, it provides an overview of the role of experimental designs for the successful implementation of the DCE approach in health care studies, and it provides researchers with an introduction to constructing experimental designs on the basis of study objectives and the statistical model researchers have selected for the study. The report outlines the theoretical requirements for designs that identify choice-model preference parameters and summarizes and compares a number of available approaches for constructing experimental designs. The task-force leadership group met via bimonthly teleconferences and in person at ISPOR meetings in the United States and Europe. An international group of experimental-design experts was consulted during this process to discuss existing approaches for experimental design and to review the task force's draft reports. In addition, ISPOR members contributed to developing a consensus report by submitting written comments during the review process and oral comments during two forum presentations at the ISPOR 16th and 17th Annual International Meetings held in Baltimore (2011) and Washington, DC (2012).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We followed recommended guidelines to design and implement the DCE. [7, 8, 11–16]”
From Assessing the American public's preferences for reforms to teen driving licensure systems: a discrete choice experiment (2026) · cites this paper as [12]
In the citing paper
“To improve experimental efficiency and avoid redundancy, this study adopted a D-efficient design, which reduces the number of choice cards while maintaining the precision of parameter estimation, thereby enabling participants to provide more efficient data contributions within a limited number of tasks (59, 60).”
From Designing age-friendly interfaces and wearable–environment integrations to enhance medication adherence: a discrete choice experiment among older adults (2026) · cites this paper as [59]
In the citing paper
“we used an efficient fractional factorial design to generate 7 choice scenarios, maximizing D-efficiency with zero priors on the coefficients. [12, 15]”
From Assessing the American public's preferences for reforms to teen driving licensure systems: a discrete choice experiment (2026) · cites this paper as [12]
In the citing paper
“The most widely used type of stated preference method is the discrete choice experiment (DCE) [7].”
From Understanding Uptake of Digital Health Products: Methodology Tutorial for a Discrete Choice Experiment Using the Bayesian Efficient Design (2021) · cites this paper as [7]
Citations
Cites
Louviere, J. J., Hensher, D. A., Swait́, J., & Adamowicz, W. (2000) · Cambridge University Press eBooks · doi:10.1017/cbo9780511753831 · Cited by 7,008
Applied choice analysis a primer
Hensher, D. A., Rose, J. M., & Greene, W. H. (2005) · Cited by 2,619
Fedorov, V. V., Studden, W. J., & Klimko, E. M. (1972) · Cited by 2,057
Bridges, J. F. P., Hauber, A. B., Marshall, D., Lloyd, A., Prosser, L. A., Regier, D. A., Johnson, F. R., & Mauskopf, J. (2011) · Value in Health · doi:10.1016/j.jval.2010.11.013 · Cited by 2,029
Self-citationTheory of Optimal Experiments.
Herzberg, A. M., Wynn, H. P., Fedorov, V. V., Studden, W. J., & Klimko, E. M. (1972) · Biometrika · doi:10.2307/2334826 · Cited by 1,896
Conducting Discrete Choice Experiments to Inform Healthcare Decision Making: A User??s Guide
Lancsar, E., & Louviere, J. (2008) · PharmacoEconomics · doi:10.2165/00019053-200826080-00004 · Cited by 1,707
Self-citationStatistical Models in Engineering
Brownlee, K. A., Hahn, G. J., & Shapiro, S. S. (1968) · Technometrics · doi:10.2307/1266243 · Cited by 1,193
Getting Started with Conjoint Analysis: Strategies for Product Design and Pricing Research
Orme, B. K. (2005) · Cited by 1,082
The Importance of Utility Balance in Efficient Choice Designs
Huber, J. C., & Zwerina, K. (1996) · Journal of Marketing Research · doi:10.1177/002224379603300305 · Cited by 1,023
Determining the Appropriate Response to Evidence of Public Concern: The Case of Food Safety
Finn, A. H. R., & Louviere, J. J. (1992) · Journal of Public Policy & Marketing · doi:10.1177/074391569201100202 · Cited by 802
Best–worst scaling: What it can do for health care research and how to do it
Flynn, T. N., Louviere, J. J., Peters, T. J., & Coast, J. (2006) · Journal of Health Economics · doi:10.1016/j.jhealeco.2006.04.002 · Cited by 787
Using Discrete Choice Experiments to Value Health and Health Care
(2008) · The Economics of Non-Market Goods and Resources · doi:10.1007/978-1-4020-5753-3 · Cited by 765
Discrete Choice Experiments Are Not Conjoint Analysis
Louviere, J. J., Flynn, T. N., & Carson, R. T. (2010) · Journal of Choice Modelling · doi:10.1016/s1755-5345(13)70014-9 · Cited by 738
Scarpa, R., & Rose, J. M. (2008) · Australian Journal of Agricultural and Resource Economics · doi:10.1111/j.1467-8489.2007.00436.x · Cited by 643
Ryan, M., & Gerard, K. (2003) · PubMed · Cited by 630
Designing efficient stated choice experiments in the presence of reference alternatives
Rose, J. M., Bliemer, M. C. J., Hensher, D. A., & Collins, A. T. (2008) · Transportation Research Part B: Methodological · doi:10.1016/j.trb.2007.09.002 · Cited by 528
Designing Conjoint Choice Experiments Using Managers' Prior Beliefs
Sándor, Z., & Wedel, M. (2001) · Journal of Marketing Research · doi:10.1509/jmkr.38.4.430.18904 · Cited by 493
Efficient Experimental Design with Marketing Research Applications
Kuhfeld, W. F., Tobias, R. D., & Garratt, M. J. (1994) · Journal of Marketing Research · doi:10.2307/3151882 · Cited by 491
Lancsar, E., & Louviere, J. J. (2006) · Health Economics · doi:10.1002/hec.1104 · Cited by 415
Self-citationA Comparison of Algorithms for Constructing Exact D-Optimal Designs
Cook, R. D., & Nachtrheim, C. J. (1980) · Technometrics · doi:10.1080/00401706.1980.10486162 · Cited by 398
Design techniques for stated preference methods in health economics
Carlsson, F., & Martinsson, P. (2002) · Health Economics · doi:10.1002/hec.729 · Cited by 393
Bliemer, M. C. J., & Rose, J. M. (2010) · Transportation Research Part B Methodological · doi:10.1016/j.trb.2009.12.004 · Cited by 366
Conjoint Analysis Applications in Health – How are Studies being Designed and Reported?
Marshall, D. A., Bridges, J. F. P., Hauber, B., Cameron, R., Donnalley, L., Fyie, K., & Johnson, F. R. (2010) · Patient · doi:10.2165/11539650-000000000-00000 · Cited by 364
Self-citationThe Construction of Optimal Stated Choice Experiments: Theory and Methods
Street, D. J., & Burgess, L. (2007) · Cited by 314
Optimal Design for Multinomial Choice Experiments
Kanninen, B. J. (2002) · Journal of Marketing Research · doi:10.1509/jmkr.39.2.214.19080 · Cited by 307
Self-citationSeiden, E., & Zemach, R. (1966) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177699280 · Cited by 285
Flynn, T. N. (2010) · Expert Review of Pharmacoeconomics & Outcomes Research · doi:10.1586/erp.10.29 · Cited by 273
Profile Construction in Experimental Choice Designs for Mixed Logit Models
Sándor, Z., & Wedel, M. (2002) · Marketing Science · doi:10.1287/mksc.21.4.455.131 · Cited by 223
Choice experiments in health: the good, the bad, the ugly and toward a brighter future
Louviere, J. J., & Lancsar, E. (2009) · Health Economics Policy and Law · doi:10.1017/s1744133109990193 · Cited by 209
Self-citationHeterogeneous Conjoint Choice Designs
Sándor, Z., & Wedel, M. (2005) · Journal of Marketing Research · doi:10.1509/jmkr.42.2.210.62285 · Cited by 205
Bridges, J. F. P. (2003) · PubMed · Cited by 194
Self-citationStated choice experimentation, contextual influences and food choice: A case study
Jaeger, S. R., & Rose, J. M. (2008) · Food Quality and Preference · doi:10.1016/j.foodqual.2008.02.005 · Cited by 186
Hall, J. P., Kenny, P. M., King, M. T., Louviere, J. J., Viney, R. C., & Yeoh, A. (2002) · Health Economics · doi:10.1002/hec.694 · Cited by 181
Efficient stated choice experiments for estimating nested logit models
Bliemer, M. C. J., Rose, J. M., & Hensher, D. A. (2008) · Transportation Research Part B Methodological · doi:10.1016/j.trb.2008.05.008 · Cited by 160
Experimental Design For Stated-Choice Studies
Johnson, F. R., Kanninen, B. J., Bingham, M. F., & Özdemir, S. (2007) · Springer eBooks · doi:10.1007/1-4020-5313-4_7 · Cited by 160
Self-citationLouviere, J. J., & Flynn, T. N. (2010) · Patient · doi:10.2165/11539660-000000000-00000 · Cited by 145
Kanninen, B. J. (2010) · The Economics of non-market goods and resources · doi:10.1007/1-4020-5313-4 · Cited by 136
Self-citationChoice-Based Conjoint Analysis: Models and Designs
Raghavarao, D., Wiley, J. B., & Chitturi, P. (2010) · Thammasat University Digital Collections · Cited by 133
Viney, R. C., Savage, E., & Louviere, J. J. (2005) · Health Economics · doi:10.1002/hec.981 · Cited by 109
The Construction of Optimal Stated Choice Experiments: Theory and Methods
Dean, A. (2008) · Journal of the American Statistical Association · doi:10.1198/jasa.2008.s248 · Cited by 105
An experiment on simplifying conjoint analysis designs for measuring preferences
Maddala, T., Phillips, K. A., & Johnson, F. R. (2003) · Health Economics · doi:10.1002/hec.798 · Cited by 97
An Efficient Algorithm for Constructing Bayesian Optimal Choice Designs
Kessels, R., Jones, B., Goos, P., & Vandebroek, M. L. (2009) · Journal of Business and Economic Statistics · doi:10.1198/jbes.2009.0026 · Cited by 96
Fowkes, T., & Wardman, M. R. (1988) · Journal of Transport Economics and Policy · doi:10.3828/jtep.1988.22.1.27 · Cited by 95
Patient preference methods: A patient-centered evaluation paradigm
Bridges, J., Onukwugha, E., Johnson, F. R., & Hauber, B. (2007) · Cited by 79
Self-citationChoice-Based Conjoint Analysis
Raghavarao, D., Wiley, J. B., & Chitturi, P. (2010) · Chapman & Hall/CRC eBooks · doi:10.1201/9781420099973 · Cited by 69
Stated choice experimental design theory: the who, the what and the why
Rose, J. M., & Bliemer, M. C. J. (2014) · Edward Elgar Publishing eBooks · doi:10.4337/9781781003152.00013 · Cited by 67
Efficiency and sample size requirements for stated choice studies
Bliemer, M. C. J., & Rose, J. M. (2005) · The Sydney eScholarship Repository (The University of Sydney) · Cited by 61
Designing Discrete Choice Experiments for Health Care
Street, D. J., Burgess, L., Viney, R. C., & Louviere, J. J. (2007) · The Economics of non-market goods and resources · doi:10.1007/978-1-4020-5753-3_2 · Cited by 48
Survey-Design and Analytical Strategies for Better Healthcare Stated-Choice Studies
Johnson, F. R., & Mansfield, C. A. (2008) · Patient · doi:10.2165/1312067-200801040-00011 · Cited by 14
Non-orthogonal stated preference design
Fowkes, T., Wardman, T., & Holden, D. (1993) · Cited by 13
Marshall, D., Johnson, F. R., Kulin, N., Özdemir, S., Marshall, J., Walsh, J. M. E., Bebber, S. L. V., & Phillips, K. A. (2007) · Value in Health · doi:10.1016/s1098-3015(10)65234-5 · Cited by 3
Self-citation
Research works printed but not linked: 5
- Szeinbach, 2011, Understanding conjoint analysis applications in health, ISPOR CONNECTIONS, 17, 8
- Johnson FR, Yang J-C, Mohamed AF. In defense of imperfect experimental designs: statistical efficiency and measurement error in choice-format conjoint analysis. In: Proceedings of 2012 Sawtooth Software Conference. Orlando, FL, March 2012.
- Lancsar E. Discrete choice experimental design for alternative specific choice models: an application exploring preferences for drinking water [Invited paper]. Presented at: the Design of Experiments in Healthcare Conference, Isaac Newton Institute of Mathematical Science, Cambridge, United Kingdom, 2011.
- Fowkes, 1988, The design of stated preference travel choice experiments, J Transport Econ Policy, 13, 27
- Rose JM, Bliemer MCJ, Designing efficient data for stated choice experiments [poster]. Presented at: the 11th International Conference on Travel Behaviour Research. Kyoto, Japan, August 16–20, 2006.
Cited by
Sorted by citations
Contemporary Guidance for Stated Preference Studies
Johnston, R. J., Boyle, K., Adamowicz, W., Bennett, J., Brouwer, R., Cameron, T. A., Hanemann, W. M., Hanley, N. D., Ryan, M., Scarpa, R., et al. (2017) · Journal of the Association of Environmental and Resource Economists · doi:10.1086/691697 · Cited by 1,566
“Experimental design in these contexts, and particularly for CEs, is complex and evolving (Sándor and Wedel 2002; Johnson et al. 2006, 2013; Ferrini and Scarpa 2007; Rose et al. 2008; Scarpa and Rose 2008; Rose and Bliemer 2009, 2014; Bliemer and Rose 2010; Vermeulen et al. 2011).”
BackgroundHauber, A. B., González, J. M., Groothuis-Oudshoorn, C. G. M., Prior, T., Marshall, D. A., Cunningham, C., IJzerman, M. J., & Bridges, J. F. P. (2016) · Value in Health · doi:10.1016/j.jval.2016.04.004 · Cited by 1,330
“Although these eight profiles can be combined into a large number of distinct pairs, we could use amaineffects orthogonal design [2] to generate an experimental design consisting of four paired-profile choice tasks (Appendix Table 1).”
Self-citationSample Size Requirements for Discrete-Choice Experiments in Healthcare: a Practical Guide
de Bekker-Grob, E. W., Donkers, B., Jonker, M. F., & Stolk, E. A. (2015) · Patient · doi:10.1007/s40271-015-0118-z · Cited by 987
“The DCE approach combines consumer theory [3], random utility theory [4], experimental design theory [5], and econometric analysis [1].”
MethodValuing Alternative Work Arrangements
Mas, A., & Pallais, A. (2017) · American Economic Review · doi:10.1257/aer.20161500 · Cited by 925
“It has long been recognized in the literature on contingent valuation and discrete choice experiments that inattention is a cause for concern (Johnson et al., 2013).”
BackgroundDiscrete Choice Experiments in Health Economics: A Review of the Literature
Clark, M. D., Determann, D., Petrou, S., Moro, D., & de Bekker-Grob, E. W. (2014) · PharmacoEconomics · doi:10.1007/s40273-014-0170-x · Cited by 898
“[3], so in the interests of brevity, we refer the reader to that paper and to another key citation [11].”
Background
Metadata
- Authors
- Reed Johnson, F., Lancsar, E., Marshall, D., Kilambi, V., Mühlbacher, A., Regier, D. A., Bresnahan, B. W., Kanninen, B., & Bridges, J. F. P.
- Affiliations
- Monash University (AU)
- University of Calgary (CA)
- RTI Health Solutions (US)
- Duke University (US)
- BC Cancer Agency (CA)
- University of Washington (US)
- Johns Hopkins University (US)
- ORCID
- Venue
- Value in Health
- Publisher
- Elsevier BV
- Published
- 2013-01
- Volume
- 16
- Issue
- 1
- Pages
- 3-13
- ISSN
- 1098-3015 (ISSN-L)
- OpenAlex
- W2159936838
- Cited by
- 1,799
- References
- 51
- Citations by year
- 2026: 158 · 2025: 181 · 2024: 213 · 2023: 162 · 2022: 190 · 2021: 186 · 2020: 150 · 2019: 117 · 2018: 109 · 2017: 109 · 2016: 99 · 2015: 56 · 2014: 50 · 2013: 15
- FWCI
- 171.60
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Value in Health (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Economic and Environmental Valuation
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
- Social Sciences › Decision Sciences › Management Science and Operations Research › Optimal Experimental Design Methods
- Keywords
- discrete choice experiment, experimental design, stated preference methods, conjoint analysis, health care preferences, choice models, good scientific practice
- MeSH terms
- Choice Behavior, Delivery of Health Care, Humans, International Cooperation, Research Design, Models, Statistical, Patient Preference
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
