Understanding Uptake of Digital Health Products: Methodology Tutorial for a Discrete Choice Experiment Using the Bayesian Efficient Design
Open accessPublished version of a preprint
Abstract
Understanding the preferences of potential users of digital health products is beneficial for digital health policy and planning. Stated preference methods could help elicit individuals’ preferences in the absence of observational data. A discrete choice experiment (DCE) is a commonly used stated preference method—a quantitative methodology that argues that individuals make trade-offs when engaging in a decision by choosing an alternative of a product or a service that offers the greatest utility, or benefit. This methodology is widely used in health economics in situations in which revealed preferences are difficult to collect but is much less used in the field of digital health. This paper outlines the stages involved in developing a DCE. As a case study, it uses the application of a DCE to reveal preferences in targeting the uptake of smoking cessation apps. It describes the establishment of attributes, the construction of choice tasks of 2 or more alternatives, and the development of the experimental design. This tutorial offers a guide for researchers with no prior knowledge of this research technique.
AI snapshot
Written from the paper’s full text.
TL;DR
This methodology tutorial describes how to develop a discrete choice experiment (DCE) for digital health uptake, using smoking cessation app uptake as a case study. It details attribute selection, choice task construction, experimental design, piloting, and generation of a Bayesian efficient design.
Key findings
- The case study retained 5 of 14 initially identified factors for app uptake: monthly price, who developed the app, star ratings, description, and images; attributes had either 3 or 4 levels.
- The DCE used full-profile, unlabeled choice tasks with an opt-out option (“Neither of these 2”); participants completed 12 choice tasks, with 48 choice tasks blocked into 4 survey versions of 12 each.
- A full-factorial design would have 432 alternatives (4² × 3³), reduced to a fractional sample of 96 alternatives arranged in 48 choice pairs.
- A pilot survey with 34 participants refined price and rating levels; a pilot DCE with 49 individuals informed priors, and the longest survey completion time was under 12 minutes. The final design used a Bayesian efficient (Db-efficient) design after a Dp-efficient pilot.
Methods
- Methodology tutorial outlining DCE development stages, using a smoking cessation app uptake case study based on a systematic review and a think-aloud/interview study.
- Attribute and level selection, choice task construction, experimental design generation in Ngene, pilot analysis in Apollo/R, and final Bayesian efficient design generation.
- Pilot data sources: a 34-participant survey to refine attribute levels and a 49-participant pilot DCE to estimate priors; the paper reports design and pilot details, not final DCE preference estimates.
Limitations
- The paper states that DCEs’ hypothetical nature raises external validity concerns and an intention-behavior gap; it notes that only 2% of health care DCE studies (k=7) report external validity details, an earlier meta-analysis (k=6) found only moderate accuracy in predicting health choices, and no published study has investigated external validity of a digital health DCE.
- The paper states that any single choice set is unlikely to present all relevant attributes, and selecting the most relevant attributes requires comprehensive preparatory research that can lengthen development.
- The paper states that opt-out choices provide no information on trade-offs and that the multinomial logit model has assumptions/limitations (eg, independence of irrelevant alternatives, homogeneity of preferences, independence of observed choices); evident from the text, this is a methodology tutorial and does not report final DCE results or uptake predictions.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 75 cited sources checked · 0 supported · 0 flagged
- Findingreasoning
Of the 14 factors initially identified as relevant for uptake of health and well-being apps, 5 were retained and included in the DCE: monthly price, developer, star ratings, description, and images.
“Of the 14 factors initially identified as being relevant for the uptake of health and well-being apps, 5 were retained and included in the DCE: the monthly price of the app, who developed the app, the star ratings of the app, the description of the app, and images shown.”
Quote in the paperSubstantial claim - Findingown data
The longest completion time for the survey was under 12 min.
“The longest completion time for the survey was under 12 min.”
Quote in the paperSubstantial claim - Findingown data
In the pilot study with 49 individuals, feedback suggested a tendency to ignore the last two attributes (app description and images) when they were in the initial order.
“In our pilot study conducted with 49 individuals, feedback from the participants suggested that with the initial order of the attributes, there was a tendency to ignore the last two attributes, app description and images of the app, the most text-heavy attributes.”
Quote in the paperSubstantial claim - Premisecitation
The literature suggests that a feasible limit is 18 choice sets per participant.
“The literature suggests that a feasible limit is 18 choice sets per participant [45,46].”
Quote in the paperSubstantial claim0 of 2 sources checked - Premisecitation
Outputs from statistical models developed using DCE data can be beneficial for estimating uptake of new products or services, including digital health tools, where observational data are not available or are difficult to obtain otherwise.
“The outputs from statistical models developed using DCE data can be beneficial for estimating uptake of new products or services, including digital health tools, where observational data are not available or are difficult to obtain otherwise [15,16].”
Quote in the paperSubstantial claim0 of 2 sources checked
- Premisecitation
DCEs using blocking require a larger sample size.
“Therefore, DCEs using blocking require a larger sample size [47].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [47] source
Not checked in the free run.
- Premisecitation
The higher the number of attributes, alternatives, and choice tasks, the higher the task complexity.
“The higher the number of attributes, alternatives, and choice tasks, the higher the task complexity [20].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [20] source
Not checked in the free run.
- Premisecitation
In the review by Marshall et al, most studies included between 7 and 16 choice sets.
“In the review by Marshall et al [27], most studies included between 7 and 16 choice sets.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [27] source
Not checked in the free run.
- Premisecitation
If a range is not suitable, participants might consider the differences between levels unimportant.
“If a range is not suitable, participants might consider the differences between levels unimportant [25].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [25] source
Not checked in the free run.
- Premisecitation
When a range is not wide enough, there is a risk that participants could ignore the attributes because they judge the difference between levels to be insignificant.
“When a range is not wide enough, there is a risk that participants could ignore the attributes because they judge the difference between levels to be insignificant [20].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [20] source
Not checked in the free run.
- Premisecitation
Lack of observational data often implies a requirement to seek scientific views and comments from experts in order to generate predictions of a target behavior.
“Lack of observational data often implies a requirement to seek scientific views and comments from experts in order to generate predictions of a target behavior [17].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [17] source
Not checked in the free run.
- Premisecitation
An opt-out alternative has the potential to make choices more realistic by simulating a real-world context where individuals can exercise their right not to take up an app.
“The opt-out option has the potential to make the choices more realistic [34] by simulating a real-world context where individuals can exercise their right not to take up an app, given the apps on offer [20].”
Quote in the paperNot a substantial claim0 of 2 sources checked - Premisecitation
When a participant selects the opt-out option, no information is provided on how they trade-off attribute levels or alternatives.
“However, when a participant selects the opt-out option, no information is provided on how they trade-off attribute levels or alternatives [13].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [13] source
Not checked in the free run.
- Premisecitation
A full-factorial design is often considered impractical due to the high number of choice tasks required.
“However, this type of design is often considered impractical due to the high number of choice tasks required [20].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [20] source
Not checked in the free run.
- Interpretationown data
The tendency to ignore the last two attributes may have compromised the examination of their relative importance.
“This may have compromised the examination of the relative importance of those two attributes (app description and images of the app).”
Quote in the paperSubstantial claim - Premisecitation
Answering a repeat choice inconsistently is not a violation of random utility theory.
“Although internal validity checks provide some measure of data quality, it should be noted that answering a repeat choice inconsistently is not a violation of random utility theory [53].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [53] source
Not checked in the free run.
- Interpretationown data
The number of choice tasks did not need to be reduced.
“Thus, we concluded that the number of choice tasks did not need to be reduced.”
Quote in the paperSubstantial claim - Premisecitation
Assessing the internal validity of a DCE can help with understanding the consistency and trade-off assumptions made by participants.
“Assessing the internal validity of a DCE can help with understanding the consistency and trade-off assumptions made by participants [52].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [52] source
Not checked in the free run.
- Interpretationnone given
When reasonable assumptions about the distributions are made, a Db-efficient design will outperform a Dp-efficient design.
“Further, when reasonable assumptions about the distributions are made, a Db-efficient design will outperform a Dp-efficient design.”
Quote in the paperNot a substantial claim - Premisecitation
As few as 30 responses may be sufficient to generate useable data.
“Accuracy of the priors will improve with increasing sample size, but as few as 30 responses may be sufficient to generate useable data [44].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [44] 2012, NGene.1.1.1 User Manual & Reference Guide A
Not checked in the free run.
- Premisecitation
To date, there has been limited progress in testing for external validity due to the difficulty in investigating preferences in the real world.
“Nevertheless, to date, there has been limited progress in testing for external validity due to the difficulty in investigating preferences in the real world [38].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [38] source
Not checked in the free run.
- Premisecitation
Accuracy of the priors will improve with increasing sample size.
“Accuracy of the priors will improve with increasing sample size, but as few as 30 responses may be sufficient to generate useable data [44].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [44] 2012, NGene.1.1.1 User Manual & Reference Guide A
Not checked in the free run.
- Premisereasoning
Insights from attempts to measure engagement with digital tools that can be translated to uptake are limited.
“Although several attempts have been made to measure engagement with digital tools using a wide range of methodologies [56-58], the insights we have from them that can be translated to uptake are limited.”
Quote in the paperSubstantial claim0 of 3 sources checked - Premisecitation
When parameter priors are different from zero, the efficient design generated produces smaller prediction errors than orthogonal designs.
“It is noteworthy that when the parameter priors are different from zero, the efficient design generated produces smaller prediction errors than orthogonal designs [21,49,50].”
Quote in the paperSubstantial claim0 of 3 sources checked - Premisecitation
The hypothetical nature of the DCE choice set raises concerns about external validity and the degree to which real-world decisions might equate to those made by study participants under experimental conditions, a phenomenon known as the intention-behavior gap.
“As with all expressed preference methodologies, the hypothetical nature of the DCE choice set raises concerns about external validity and the degree to which real-world decisions might equate to those made by study participants under experimental conditions, a phenomenon known as the intention-behavior gap [60].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [60] source
Not checked in the free run.
- Premisecitation
A recent systematic review of the literature on DCEs in health care reported that only 2% of the included studies (k=7) report details of the investigation of external validity.
“Indeed, a recent systematic review of the literature on DCEs in health care reported that only 2% of the included studies (k=7) report details of the investigation of external validity [47], while an earlier systematic review and meta-analysis (k=6) found that DCEs have only a moderate level of accuracy in predicting behaviors of health choices [61].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [47] source
Not checked in the free run.
- Premisecitation
Any single choice set is unlikely to be able to present the user with all relevant attributes, regardless of how well it has been developed.
“A final significant concern associated with the use of a DCE is that any single choice set is unlikely to be able to present the user with all relevant attributes, regardless of how well it has been developed [61].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [61] source
Not checked in the free run.
- Premisecitation
DCEs can be used in hypothetical circumstances, enabling the measurement of preferences for a potential policy change or digital health system change before it is implemented.
“Therefore, DCEs can be used in hypothetical circumstances, enabling the measurement of preferences for a potential policy change or digital health system change before it is implemented [13], such as the recent investigation of the uptake of a COVID-19 test-and-trace health app [3,4].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [13] source
Not checked in the free run.
- Classified as probably not a substantial claim. 32
- May combine two claims. 14
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationData analysis in DCEs involves regression modeling in a random utility framework.
“Data analysis involves regression modeling in a random utility framework [8].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [8] source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationExtensions of MNL, such as nested logit, mixed logit, and latent class models, may be used to account for MNL's limitations.
“Extensions of MNL (eg, nested logit, mixed logit, and latent class models) may be used to account for these limitations [39,40].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [39] Hensher, DA, 2005, Applied Choice Analysis: A Primer
Not checked in the free run.
- cites [40] source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe identification of attributes in a DCE is typically based on primary and secondary data collection to ensure that the DCE is tailored to the study setting.
“The identification of attributes is typically based on primary and secondary data collection to ensure that the DCE is tailored to the study setting [13].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [13] source
Not checked in the free run.
Classified as probably not a substantial claim. May combine two claims.
Study design and conductreasoningIn our DCE, 12 choice tasks per participant were deemed low enough to avoid excessive cognitive load but high enough to establish sufficient statistical precision.
“In our DCE, we administered 12 choice tasks per participant, which were deemed a number low enough to avoid excessive cognitive load but high enough to establish sufficient statistical precision.”
Quote in the paperSubstantial claimMay combine two claims.
PremisecitationTo undertake sample size calculation, it is crucial to know the number of alternatives per choice set, the largest number of levels of any attribute (for main effects) or the largest level of any two attributes (for interaction effects), and the number of blocks.
“It is noteworthy that to undertake the sample size calculation, it is crucial to know the number of alternatives per choice set, the largest number of levels of any attribute (for DCEs looking at main effects only) or the largest level of any two attributes (for a DCE looking at interaction effects), and the number of blocks [38].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [38] source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationIn recent years, DCEs have been increasingly used in health and health care settings, addiction research, and digital health.
“In recent years, DCEs have been increasingly used in health and health care settings [9,10], as well as in addiction research [11] and digital health [4-6].”
Quote in the paperNot a substantial claim0 of 6 sources checkedClassified as probably not a substantial claim. May combine two claims.
PremisecitationA balance exists between simpler designs with lower numbers of levels, which reduce respondent burden and are useful for identifying attribute rankings, and more complex designs with higher levels, which have higher statistical precision and are more sensitive to identifying trade-offs.
“Yang et al [32] suggested that a balance exists between simpler designs with lower numbers of levels, which reduce the respondent burden (and consequently measurement error) and are useful for identifying attribute rankings, and more complex designs with higher levels (and higher statistical precision) and is more sensitive to identifying trade-offs between attributes.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [32] source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe importance of qualitative research lies in ensuring inclusion of attributes that are relevant to most participants.
“The importance of qualitative research lies in ensuring inclusion of attributes that are relevant to most participants [25].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [25] source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationImbalance in the numbers of levels across attributes raises the importance of the attributes with higher levels.
“and that imbalance in the numbers of levels across attributes raises the importance of the attributes with higher levels [32].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [32] source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe most widely used type of stated preference method is the discrete choice experiment (DCE).
“The most widely used type of stated preference method is the discrete choice experiment (DCE) [7].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [7] source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningThe increasing number of DCEs in digital health highlights their potential, although they are currently underused.
“The increasing number of DCEs in digital health highlights their potential, although they are currently underused.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing).
PremisecitationIncreasing the number of levels for an attribute increases the relative importance of that attribute.
“It has been suggested that increasing the number of levels for an attribute increases the relative importance of that attribute [31]”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [31] Ratcliffe, J, 2002, Int J Technol Assess Health Care
Not checked in the free run.
May combine two claims.
PremisecitationDCEs can provide an empirical alternative to expert opinions while accounting for possible interactions between attributes, which are otherwise often ignored.
“However, DCEs can provide an empirical alternative to expert opinions, while accounting for possible interactions between attributes (eg, design of a product and brand name), which are otherwise often ignored [18].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [18] source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationThe random utility model conventionally used is based on the Lancaster theory of consumer demand, and together they assume that individuals make trade-offs when making a decision and choose an option that offers the greatest utility, determined by how much importance they place on the attributes associated with the product.
“The random utility model conventionally used is also based on the Lancaster theory of consumer demand [35], which together assume that individuals make trade-offs when making a decision and would choose an option that offers the greatest utility [36], determined by how much importance they place on the attributes associated with the product [37].”
Quote in the paperNot a substantial claim0 of 3 sources checkedProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductcitationThe study protocol of the research this paper is based on is registered on the Open Science Framework.
“The study protocol of the research this paper is based on is registered on the Open Science Framework [19].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [19] SzinayDRoryCJonesAWhittyJChadbornTBrownJNaughtonFAdult Smokers' Preferences for the Uptake of Smoking Cessation Apps: A Discrete Choice Experiment202103122020-11-01https://osf.io/5439x/
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe multinomial logit (MNL) model is typically a starting point for basic model estimation in DCEs.
“The multinomial logit (MNL) model has been previously described as the “workhorse” of DCE estimation [38,39], and it typically serves as a starting point for basic model estimation (although alternative models, such as probit, may be used).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [38] source
Not checked in the free run.
- cites [39] Hensher, DA, 2005, Applied Choice Analysis: A Primer
Not checked in the free run.
May combine two claims.
PremisecitationFive attributes are generally considered low enough to complete a full-profile choice task, which consequently maximizes the information about trade-offs.
“Five attributes are generally considered low enough to complete a full-profile choice task, which consequently maximizes the information about trade-offs [33].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [33] source
Not checked in the free run.
May combine two claims.
Study design and conductcitationOur DCE's attributes were based on a comprehensive systematic review and a qualitative think-aloud and interview study.
“Our DCE was based on a comprehensive systematic review investigating factors influencing the uptake and engagement with health and well-being smartphone apps [28] and a qualitative research component that consisted of a think-aloud and interview study to examine further the previously identified factors or attributes [29].”
Quote in the paperSubstantial claim0 of 2 sources checkedProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationMNL requires assumptions and limitations, including independence of irrelevant alternatives, homogeneity of preferences, and independence of observed choices.
“It is important to note that MNL requires some important assumptions and limitations—for example, independence of irrelevant alternatives, homogeneity of preferences, and independence of observed choices [40,41].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [40] source
Not checked in the free run.
- cites [41] Train, K, 2009, Logit. Discrete Choice Methods with Simulation. 2 ed
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationBlocking is a technique widely used in DCEs to reduce cognitive burden by partitioning large experimental designs into subsets of equal size, thereby reducing the number of choice tasks that any one respondent is required to complete.
“Blocking is a technique widely used in DCEs to reduce cognitive burden by partitioning large experimental designs into subsets of equal size, thereby reducing the number of choice tasks that any one respondent is required to complete [47].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [47] source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThere has been a recent change in thinking toward a nonorthogonal and statistically more efficient design.
“There has therefore been a recent change in thinking toward a nonorthogonal and statistically more efficient design [38].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [38] source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationDCEs are widely used to understand patient and provider choices in health care.
“Although DCEs are widely used to understand patient and provider choices in health care [8,10,15,55], they have only recently started to gain popularity in digital health [4-6] and as such represent an underused approach in digital health.”
Quote in the paperNot a substantial claim0 of 4 sources checkedMay combine two claims.
PremisecitationDCEs have only recently started to gain popularity in digital health and represent an underused approach in digital health.
“Although DCEs are widely used to understand patient and provider choices in health care [8,10,15,55], they have only recently started to gain popularity in digital health [4-6] and as such represent an underused approach in digital health.”
Quote in the paperSubstantial claim0 of 3 sources checkedProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe DCE methodology is considered a convenient approach to investigate the uptake of new interventions, including digital health interventions.
“The DCE methodology is also considered a convenient approach to investigate the uptake of new interventions, including digital health interventions [38], for example, digital behavior change interventions using a health and well-being smartphone app.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [38] source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningChoosing the most relevant attributes to test in a DCE requires comprehensive preparatory research, which can lengthen the time required to undertake the development phase of any piece of work.
“Choosing the most relevant attributes to test in a DCE, therefore, requires comprehensive preparatory research, which can lengthen the time required to undertake the development phase of any piece of work.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe most widely used efficiency measure is the D-error, which refers to the efficiency of the experimental design in extracting information from respondents.
“The most widely used efficiency measure is the D-error, which may be easily estimated using various software packages, such as Ngene, and refers to the efficiency of the experimental design in extracting information from respondents [21].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [21] source
Not checked in the free run.
Classified as probably not a substantial claim.
InterpretationreasoningDCEs enable the researcher to gain measurable insights into situations in which quantitative measures are hard to otherwise obtain, such as the factors impacting the uptake of health apps on curated health app portals.
“For example, as illustrated by the case study here, they enable the researcher to gain measurable insights into situations in which quantitative measures are hard to otherwise obtain, such as the factors impacting the uptake of health apps on curated health app portals.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationPilot findings may suggest that the DCE needs to be amended (e.g., reducing choice sets or attributes) to better reflect participants' preferences and improve precision in parameter estimates.
“The findings of the pilot may suggest that the DCE needs to be amended, such as reducing the number of choice sets or the number of attributes, so that the responses are a better reflection of the participants’ preferences and improve the precision in the parameter estimates [13].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [13] source
Not checked in the free run.
Classified as probably not a substantial claim. May combine two claims.
Study design and conductcitationFollowing the advice of Lancsar and Louviere, we did not exclude participants who failed the internal validity check, as that might have caused statistical bias or affected statistical efficiency.
“Following the advice of Lancsar and Louviere [54], we did not exclude participants who failed the internal validity check, as that might have caused statistical bias or affected statistical efficiency.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [54] source
Not checked in the free run.
- Rests on an assumption: There is no consensus on what to do with the data from responses that fail validity tests.
Classified as probably not a substantial claim.
PremisecitationWhen perfect orthogonality and balance cannot be achieved or are not desirable, an efficient design can be applied.
“When perfect orthogonality and balance cannot be achieved or are not desirable, an efficient design can be applied [20].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [20] source
Not checked in the free run.
May combine two claims.
PremisecitationA D-efficient design will outperform an orthogonal design, and given reliable priors, a Dp-efficient design will outperform a Dz-efficient design.
“Hence, a D-efficient design will outperform an orthogonal design, and (given reliable priors), a Dp-efficient design will outperform a Dz-efficient design [21].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [21] source
Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable).
InterpretationreasoningDCEs bring several benefits to help overcome the issue of measuring uptake in digital health or in other areas where the measurement of the predictors of uptake in a good or service is required.
“DCEs bring several benefits to help overcome the issue of measuring uptake in digital health or in other areas where the measurement of the predictors of uptake in a good or service is required.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.
Interpretationnone givenDCEs have significant potential in digital health research and can serve as an important decision-making tool in a field where observational data are lacking.
“In summary, DCEs have significant potential in digital health research and can serve as an important decision-making tool in a field where observational data are lacking.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationcitationParticipants may believe they would choose a scenario presented and described in a choice task, but in real life, there might be other factors that would influence their behaviors, such as the aesthetics of the app.
“As such, participants may believe they would choose a scenario presented and described in a choice task, but in real life, there might be other factors that would influence their behaviors, such as the aesthetics of the app [28].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [28] source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataIn our research, we used the stability validity test to check internal validity by repeating a randomly generated choice task (the fourth).
“In our research, we used the stability validity test to check the internal validity by repeating a randomly generated choice task (in our case, it was the fourth).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenWe reported data on internal validity to enable the reader to make a judgement on likely biases.
“However, we reported data on internal validity to enable the reader to make a judgement on likely biases.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based).
InterpretationreasoningA plausible explanation for limited translatable insights is that uptake of digital tools is difficult to empirically measure.
“One plausible explanation is that uptake of digital tools is difficult to empirically measure.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationAn earlier systematic review and meta-analysis (k=6) found that DCEs have only a moderate level of accuracy in predicting behaviors of health choices.
“Indeed, a recent systematic review of the literature on DCEs in health care reported that only 2% of the included studies (k=7) report details of the investigation of external validity [47], while an earlier systematic review and meta-analysis (k=6) found that DCEs have only a moderate level of accuracy in predicting behaviors of health choices [61].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [61] source
Not checked in the free run.
Classified as probably not a substantial claim.
PremisecitationA DCE helps to quantify preferences to support more complex decisions.
“A DCE also helps to quantify preferences to support more complex decisions [59].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [59] source
Not checked in the free run.
The paper relies on this without support.
Uptake of digital tools is generally low.
“This is important because uptake of digital tools is generally low.”
The paper relies on this without support.
Stated preference methods could help elicit individuals’ preferences in the absence of observational data.
“Stated preference methods could help elicit individuals’ preferences in the absence of observational data.”
The paper relies on this without support.
The allocation of attribute levels within the experimental design can affect statistical power; if a certain level is underrepresented in the choice sets generated, then the coefficient for that level cannot be easily estimated.
“The allocation of the attribute levels within the experimental design can affect statistical power; if a certain level is underrepresented in the choice sets generated, then the coefficient for that level cannot be easily estimated.”
The paper relies on this without support.
Dominant alternatives do not provide information about how trade-offs are made, as individuals usually would select the dominant alternatives.
“Dominant alternatives, where all attribute levels of one alternative are more desirable than all attribute levels in the others, do not provide information about how trade-offs are made, as individuals usually would select the dominant alternatives.”
The paper relies on this without support.
In our DCE, there was a lack of empirical evidence to suggest the presence of potential interactions between attributes.
“In our DCE, given the novel nature of the research on the uptake of health apps and the lack of empirical evidence to suggest the presence of potential interactions between attributes, we decided to only look at main effects.”
The paper relies on this without support.
For studies with five or more attributes with two or more levels, an orthogonal design may not be practical.
“For studies with five or more attributes with two or more levels, an orthogonal design may not be practical.”
The paper relies on this without support.
There is no formal guidance on how large the pilot sample should be.
“There is no formal guidance on how large the pilot sample should be, and this is largely guided by the budget and complexity of the experimental design.”
The paper relies on this without support.
The size of the pilot sample is largely guided by the budget and complexity of the experimental design.
“There is no formal guidance on how large the pilot sample should be, and this is largely guided by the budget and complexity of the experimental design.”
The paper relies on this without support.
Piloting offers an opportunity to ensure that the information is presented clearly and that the choices are realistic and meaningful.
“In addition to providing estimations for the choice matrix design described above, piloting offers an opportunity to ensure that the information is presented clearly and that the choices are realistic and meaningful.”
The paper relies on this without support.
Piloting provides insight into how cognitively demanding it is for respondents to complete.
“It also provides insight into how cognitively demanding it is for respondents to complete.”
The paper relies on this without support.
There is no consensus on what to do with the data from responses that fail validity tests.
“Furthermore, there is no consensus on what to do with the data from responses that fail validity tests.”
The paper relies on this without support.
To our knowledge, no study has been published that investigates the external validity of a DCE developed in digital health.
“To our knowledge, no study has been published that investigates the external validity of a DCE developed in digital health.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The study used a D-efficient design to reduce the number of choice cards while maintaining the precision of parameter estimation.
“To improve experimental efficiency and avoid redundancy, this study adopted a D-efficient design, which reduces the number of choice cards while maintaining the precision of parameter estimation, thereby enabling participants to provide more efficient data contributions within a limited number of tasks (59, 60).”
From Designing age-friendly interfaces and wearable–environment integrations to enhance medication adherence: a discrete choice experiment among older adults (2026) · cites this paper as [60]
The Experimental Design section says D-efficient designs increase the precision of parameter estimates for a given sample size. It attributes the reduction in choice tasks to the fractional-factorial design and blocking, rather than to D-efficiency alone. The cited paper therefore supports the precision rationale, but not the claim that D-efficiency itself reduced the number of choice cards. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Cites
The theory of planned behavior
Ajzen, I. (1991) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(91)90020-t · Cited by 90,587 · see citing works
Cragg, J. G., & Zarembka, P. (1975) · Canadian Journal of Economics/Revue canadienne d économique · doi:10.2307/134075 · Cited by 8,385 · see citing works
Applied choice analysis a primer
Hensher, D. A., Rose, J. M., & Greene, W. H. (2005) · Cited by 2,619 · see citing works
The Mixed Logit model: The state of practice
Hensher, D. A., & Greene, W. H. (2003) · Transportation · doi:10.1023/a:1022558715350 · Cited by 2,047 · see citing works
Bridges, J. F. P., Hauber, A. B., Marshall, D., Lloyd, A., Prosser, L. A., Regier, D. A., Johnson, F. R., & Mauskopf, J. (2011) · Value in Health · doi:10.1016/j.jval.2010.11.013 · Cited by 2,029 · see citing works
Applied Choice Analysis: A Primer
Hensher, D. A., Rose, J. M., & Greene, W. H. (2005) · doi:10.1017/cbo9780511610356 · Cited by 1,823 · see citing works
Reed Johnson, F., Lancsar, E., Marshall, D., Kilambi, V., Mühlbacher, A., Regier, D. A., Bresnahan, B. W., Kanninen, B., & Bridges, J. F. P. (2013) · Value in Health · doi:10.1016/j.jval.2012.08.2223 · Cited by 1,799 · see citing works
Conducting Discrete Choice Experiments to Inform Healthcare Decision Making: A User??s Guide
Lancsar, E., & Louviere, J. (2008) · PharmacoEconomics · doi:10.2165/00019053-200826080-00004 · Cited by 1,707 · see citing works
Hauber, A. B., González, J. M., Groothuis-Oudshoorn, C. G. M., Prior, T., Marshall, D. A., Cunningham, C., IJzerman, M. J., & Bridges, J. F. P. (2016) · Value in Health · doi:10.1016/j.jval.2016.04.004 · Cited by 1,330 · see citing works
Discrete choice experiments in health economics: a review of the literature
de Bekker‐Grob, E. W., Ryan, M., & Gerard, K. (2012) · Health Economics · doi:10.1002/hec.1697 · Cited by 1,329 · see citing works
Hess, S., & Palma, D. (2019) · Journal of Choice Modelling · doi:10.1016/j.jocm.2019.100170 · Cited by 966 · see citing works
Discrete Choice Experiments in Health Economics: A Review of the Literature
Clark, M. D., Determann, D., Petrou, S., Moro, D., & de Bekker-Grob, E. W. (2014) · PharmacoEconomics · doi:10.1007/s40273-014-0170-x · Cited by 867 · see citing works
Discrete Choice Experiments in Health Economics: Past, Present and Future
Soekhai, V., de Bekker-Grob, E. W., Ellis, A. R., & Vass, C. (2018) · PharmacoEconomics · doi:10.1007/s40273-018-0734-2 · Cited by 815 · see citing works
Constructing Efficient Stated Choice Experimental Designs
Rose, J. M., & Bliemer, M. C. J. (2009) · Transport Reviews · doi:10.1080/01441640902827623 · Cited by 797 · see citing works
The development and evaluation of a survey to measure user engagement
O'Brien, H. L., & Toms, E. G. (2009) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21229 · Cited by 794 · see citing works
Mangham‐Jefferies, L., Hanson, K., & McPake, B. (2008) · Health Policy and Planning · doi:10.1093/heapol/czn047 · Cited by 635 · see citing works
Ferrini, S., & Scarpa, R. (2007) · Journal of Environmental Economics and Management · doi:10.1016/j.jeem.2006.10.007 · Cited by 516 · see citing works
Lancsar, E., & Louviere, J. J. (2006) · Health Economics · doi:10.1002/hec.1104 · Cited by 402 · see citing works
Quaife, M., Terris‐Prestholt, F., Tanna, G. L. D., & Vickerman, P. T. (2018) · The European Journal of Health Economics · doi:10.1007/s10198-018-0954-6 · Cited by 366 · see citing works
Conjoint Analysis Applications in Health – How are Studies being Designed and Reported?
Marshall, D. A., Bridges, J. F. P., Hauber, B., Cameron, R., Donnalley, L., Fyie, K., & Johnson, F. R. (2010) · Patient · doi:10.2165/11539650-000000000-00000 · Cited by 364 · see citing works
Self-citationSzinay, D., Jones, A., Chadborn, T., Brown, J., & Naughton, F. (2020) · Journal of Medical Internet Research · doi:10.2196/17572 · Cited by 289 · cited 2× · see citing works
“…based on a comprehensive systematic review investigating factors influencing the uptake and engagement with health and well-being smartphone apps [28] and a qualitative research component that consisted of a think-aloud and interview study to examine further the previously identified factors or…”
BackgroundSelf-citationJohnson, F. R., Yang, J., & Reed, S. D. (2018) · Value in Health · doi:10.1016/j.jval.2018.07.876 · Cited by 237 · see citing works
de Bekker-Grob, E. W., Hol, L., Donkers, B., van Dam, L., Habbema, J. D., van Leerdam, M. E., Kuipers, E. J., Essink‐Bot, M., & Steyerberg, E. W. (2009) · Value in Health · doi:10.1111/j.1524-4733.2009.00670.x · Cited by 227 · see citing works
Approximation of bayesian efficiency in experimental choice designs
Bliemer, M. C. J., Rose, J. M., & Hess, S. (2008) · Journal of Choice Modelling · doi:10.1016/s1755-5345(13)70024-1 · Cited by 213 · see citing works
Rakotonarivo, O. S., Schaafsma, M., & Hockley, N. J. (2016) · Journal of Environmental Management · doi:10.1016/j.jenvman.2016.08.032 · Cited by 199 · see citing works
Designing a Stated Choice Experiment: The Value of a Qualitative Process
Kløjgaard, M. E., Bech, M., & Søgaard, R. (2012) · Journal of Choice Modelling · doi:10.1016/s1755-5345(13)70050-2 · Cited by 184 · see citing works
Hall, J. P., Kenny, P. M., King, M. T., Louviere, J. J., Viney, R. C., & Yeoh, A. (2002) · Health Economics · doi:10.1002/hec.694 · Cited by 181 · see citing works
Choice Experiments to Quantify Preferences for Health and Healthcare: State of the Practice
Mühlbacher, A. C., & Johnson, F. R. (2016) · Applied Health Economics and Health Policy · doi:10.1007/s40258-016-0232-7 · Cited by 174 · see citing works
Discrete Choice Experiments in a Nutshell
Ryan, M., Gerard, K., & Amaya-Amaya, M. (2007) · The Economics of non-market goods and resources · doi:10.1007/978-1-4020-5753-3_1 · Cited by 170 · see citing works
Best–worst scaling vs. discrete choice experiments: An empirical comparison using social care data
Potoglou, D., Burge, P., Flynn, T. N., Netten, A., Malley, J., Forder, J., & Brazier, J. E. (2011) · Social Science & Medicine · doi:10.1016/j.socscimed.2011.03.027 · Cited by 164 · see citing works
Danner, M., Hummel, J. M., Volz, F., van Manen, J. G., Wiegard, B., Dintsios, C., Bastian, H., Gerber, A., & IJzerman, M. J. (2011) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462311000523 · Cited by 164 · see citing works
Using stated preference discrete choice modeling to evaluate health care programs
Hall, J. P., Viney, R. C., Haas, M. R., & Louviere, J. J. (2002) · Journal of Business Research · doi:10.1016/s0148-2963(02)00352-1 · Cited by 130 · see citing works
Hauber, B., Fairchild, A. O., & Johnson, F. R. (2013) · Applied Health Economics and Health Policy · doi:10.1007/s40258-013-0028-y · Cited by 127 · see citing works
Preferences for hospital quality in Zambia: results from a discrete choice experiment
Hanson, K., McPake, B., Nakamba, P., & Archard, L. (2005) · Health Economics · doi:10.1002/hec.959 · Cited by 122 · see citing works
Jonker, M. F., de Bekker-Grob, E. W., Veldwijk, J., Goossens, L. M. A., Bour, S. S., & Mölken, M. R. (2020) · JMIR mhealth and uhealth · doi:10.2196/20741 · Cited by 110 · see citing works
Revisiting consistency with random utility maximisation: theory and implications for practical work
Hess, S., Daly, A., & Batley, R. P. (2018) · Theory and Decision · doi:10.1007/s11238-017-9651-7 · Cited by 106 · see citing works
Lambooij, M., Harmsen, I. A., Veldwijk, J., de Melker, H. E., Mollema, L., van Weert, Y. W., & de Wit, G. A. (2015) · BMC Medical Research Methodology · doi:10.1186/s12874-015-0010-5 · Cited by 105 · see citing works
An Efficient Algorithm for Constructing Bayesian Optimal Choice Designs
Kessels, R., Jones, B., Goos, P., & Vandebroek, M. L. (2009) · Journal of Business and Economic Statistics · doi:10.1198/jbes.2009.0026 · Cited by 94 · see citing works
Discrete Choice Experiment Response Rates: A Meta‐analysis
Watson, V., Becker, F., & de Bekker-Grob, E. W. (2016) · Health Economics · doi:10.1002/hec.3354 · Cited by 89 · see citing works
Leigh, S., Ashall-Payne, L., & Andrews, T. (2020) · JMIR mhealth and uhealth · doi:10.2196/17704 · Cited by 77 · see citing works
Janssen, E. M., Hauber, B., & Bridges, J. F. P. (2017) · Value in Health · doi:10.1016/j.jval.2017.07.001 · Cited by 74 · see citing works
Women's experiences of and preferences for services after rape in South Africa: interview study
Christofides, N. J., Muirhead, D., Jewkes, R. K., Penn‐Kekana, L., & Conco, D. N. (2005) · BMJ · doi:10.1136/bmj.38664.482060.55 · Cited by 73 · see citing works
Trapero‐Bertran, M., Rodríguez‐Martin, B., & López‐Bastida, J. (2019) · PLoS ONE · doi:10.1371/journal.pone.0219905 · Cited by 72 · see citing works
Perski, O., BLANDFORD, A. E., Garnett, C. V., Crane, D., West, R., & Michie, S. (2019) · Translational Behavioral Medicine · doi:10.1093/tbm/ibz039 · Cited by 70 · see citing works
Terris‐Prestholt, F., Hanson, K., MacPhail, C. L., Vickerman, P. T., Rees, H., & Watts, C. H. (2013) · PLoS ONE · doi:10.1371/journal.pone.0083193 · Cited by 66 · see citing works
The Assessment of User Engagement with eHealth Content: The eHealth Engagement Scale1
Lefebvre, R. C., Tada, Y., Hilfiker, S. W., & Baur, C. (2010) · Journal of Computer-Mediated Communication · doi:10.1111/j.1083-6101.2009.01514.x · Cited by 62 · see citing works
Kornas, K., Fong, G. T., Gallopel‐Morvan, K., Islam, T., & Hammond, D. G. (2015) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntv114 · Cited by 48 · see citing works
Preferences for new and existing contraceptive products
Fiebig, D. G., Knox, S. A., Viney, R. C., Haas, M. R., & Street, D. J. (2010) · Health Economics · doi:10.1002/hec.1686 · Cited by 46 · see citing works
Kruijshaar, M. E., Essink‐Bot, M., Donkers, B., Looman, C. W., Siersema, P. D., & Steyerberg, E. W. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-31 · Cited by 46 · see citing works
Ratcliffe, J., & Longworth, L. (2002) · PubMed · Cited by 43 · see citing works
Parameterising User Uptake in Economic Evaluations: The role of discrete choice experiments
Terris‐Prestholt, F., Quaife, M., & Vickerman, P. T. (2016) · Health Economics · doi:10.1002/hec.3297 · Cited by 38 · see citing works
Wiertz, C., Banerjee, A., Acar, O. A., & Ghosh, A. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3589199 · Cited by 37 · see citing works
The cost‐effectiveness of multi‐purpose HIV and pregnancy prevention technologies in South Africa
Quaife, M., Terris‐Prestholt, F., Eakle, R., Escobar, M. A. C., Kilbourne‐Brook, M., Mvundura, M., Meyer‐Rath, G., Delany‐Moretlwe, S., & Vickerman, P. T. (2018) · Journal of the International AIDS Society · doi:10.1002/jia2.25064 · Cited by 35 · see citing works
Szinay, D., Perski, O., Jones, A., Chadborn, T., Brown, J., & Naughton, F. (2021) · JMIR mhealth and uhealth · doi:10.2196/27173 · Cited by 33 · see citing works
Self-citationSpinks, J., Chaboyer, W. P., Bucknall, T. K., Tobiano, G., & Whitty, J. A. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-008941 · Cited by 27 · see citing works
Self-citationBuchanan, J. P., Blair, E. M., Thomson, K. L., Ormondroyd, E., Watkins, H. C., Taylor, J. C., & Wordsworth, S. (2019) · European Journal of Human Genetics · doi:10.1038/s41431-019-0452-z · Cited by 17 · see citing works
An Efficient Algorithm for Constructing Bayesian Optimal Choice DesignsPreprint
Kessels, R., Jones, B., Goos, P., et al. (2007) · SSRN Electronic Journal · doi:10.2139/ssrn.968620 · Cited by 16 · see citing works
Jin, W., Jiang, H., Liu, Y., & Klampfl, E. (2017) · PLoS ONE · doi:10.1371/journal.pone.0178826 · Cited by 16 · see citing works
Self-Monitoring App Preferences for Sun Protection: Discrete Choice Experiment Survey Analysis
Nittas, V., Mütsch, M., Braun, J., & Puhan, M. A. (2020) · Journal of Medical Internet Research · doi:10.2196/18889 · Cited by 12 · see citing works
Nittas, V., Mütsch, M., & Puhan, M. A. (2020) · JMIR Research Protocols · doi:10.2196/16087 · Cited by 10 · see citing works
Yang, J., Reed, S. D., Hass, S., Skeen, M. B., & Johnson, F. R. (2021) · Medical Decision Making · doi:10.1177/0272989x20979833 · Cited by 8 · see citing works
Eliciting Preferences for the Uptake of Smoking Cessation Apps: Discrete Choice Experiment
Szinay, D., Cameron, R. A., Jones, A., Whitty, J. A., Chadborn, T., Brown, J., & Naughton, F. (2024) · Journal of Medical Internet Research · doi:10.2196/37083 · Cited by 3 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 3
- Ratcliffe, J, 2002, Int J Technol Assess Health Care, 18, 139
- Train, K, 2009, Logit. Discrete Choice Methods with Simulation. 2 ed, 34
- 2012, NGene.1.1.1 User Manual & Reference Guide A, 1
Cited by
Sorted by citations
Li, X., Min, H., Wu, Y., Zhang, J., Ning, Y., Long, L., Jia, K., Jing, W., & Sun, X. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1160629 · Cited by 37 · see citing works
Understanding demand for broken rice and its potential food security implications in Colombia
Phillips, J., Durand‐Morat, A., Nalley, L. L., Graterol, E., Bonatti, M., Pava, K. L. D. L., Urioste, S., & Yang, W. (2023) · Journal of Agriculture and Food Research · doi:10.1016/j.jafr.2023.100884 · Cited by 25 · see citing works
Antonini, M., Genie, M. G., Attwell, K., Attema, A. E., Ward, J. K., Melegaro, A., Torbica, A., Kelly, B. J., Berardi, C., Sequeira, A. R., et al. (2025) · Social Science & Medicine · doi:10.1016/j.socscimed.2025.117687 · Cited by 19 · see citing works
Key Information Influencing Patient Decision-Making About AI in Health Care: Survey Experiment Study
Zhu, X., Stroud, A. M., Minteer, S. A., Yoo, D. W., Ridgeway, J. L., Mooghali, M., Miller, J. E., & Barry, B. A. (2025) · Journal of Medical Internet Research · doi:10.2196/75615 · Cited by 18 · see citing works
Genie, M. G., Ryan, M., & Krucien, N. (2023) · Health Economics · doi:10.1002/hec.4658 · Cited by 16 · see citing works
Houwen, T., Vugts, M. A. P., Lansink, K. W. W., Theeuwes, H. P., Neequaye, N., Beerekamp, M. S. H., Joosen, M. C. W., & de Jongh, M. A. C. (2022) · JMIR Human Factors · doi:10.2196/35342 · Cited by 15 · see citing works
Fischer, A.-K., & Mühlbacher, A. C. (2023) · JMIR Research Protocols · doi:10.2196/46056 · Cited by 14 · see citing works
Arije, O., Madan, J. J., & Hlungwani, T. M. (2024) · Health Economics Review · doi:10.1186/s13561-024-00497-4 · Cited by 13 · see citing works
Citta, S. P., Uehara, T., Cordier, M., Tsuge, T., & Asari, M. (2024) · Frontiers in Sustainability · doi:10.3389/frsus.2024.1391491 · Cited by 13 · see citing works
Valuing mangrove ecosystem services along the Brazilian Amazon coast
Otieno, P., Owuor, M. A., Pacheco, C. F. O., Coppo, G., Santos, T. M., Ferreira, T. O., & Bernardino, A. F. (2026) · Ecosystem Services · doi:10.1016/j.ecoser.2026.101831 · Cited by 11 · see citing works
Xie, Z., Liu, H., & Or, C. K. (2023) · mHealth · doi:10.21037/mhealth-22-39 · Cited by 11 · see citing works
Selim, S. M., Senanayake, S. J., McPhail, S., Carter, H. E., Naicker, S., & Kularatna, S. (2024) · Patient · doi:10.1007/s40271-024-00692-9 · Cited by 11 · see citing works
Patient preferences for Remote cochlear implant management: A discrete choice experiment
Sucher, C. M., Norman, R., Chaffey, E. L., Bennett, R. J., & Ferguson, M. (2025) · PLoS ONE · doi:10.1371/journal.pone.0320421 · Cited by 10 · see citing works
Factors influencing consumer preferences for sustainably produced rice: A systematic review
Zhou, X., Checco, J., Akber, M. A., & Aziz, A. A. (2025) · Cleaner and Responsible Consumption · doi:10.1016/j.clrc.2025.100309 · Cited by 9 · see citing works
Tan, S., Wang, Y., Tang, Y., Jiang, R., Chen, M., Chen, H., & Yang, F. (2022) · Frontiers in Public Health · doi:10.3389/fpubh.2022.1005453 · Cited by 8 · see citing works
Soe, N. N., Latt, P. M., King, A. J., Lee, D. M., Phillips, T. R., Fairley, C. K., Zhang, L., & Ong, J. J. (2024) · Patient · doi:10.1007/s40271-024-00720-8 · Cited by 8 · see citing works
Chen, N., Bai, D., & Ning, J. (2025) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1508369 · Cited by 7 · see citing works
Son, Y.-G., Lee, Y., & Jo, J. (2024) · Forests · doi:10.3390/f15030551 · Cited by 7 · see citing works
Gisbert, J. P., Schreiber, S., Siegel, C. A., Magro, F., Jus, A., Whichello, C. L., Michaels-Igbokwe, C., Heidenreich, S., Oortwijn, A. F., & Vermeire, S. (2024) · Inflammatory Bowel Diseases · doi:10.1093/ibd/izae162 · Cited by 7 · see citing works
Latt, P. M., Soe, N. N., King, A. J., Lee, D. M., Phillips, T. R., Xu, X., Chow, E. P. F., Fairley, C. K., Zhang, L., & Ong, J. J. (2024) · BMC Public Health · doi:10.1186/s12889-024-20688-2 · Cited by 7 · see citing works
Dzobo, M., Dzinamarira, T., Strauss, M., & Mashamba-Thompson, T. P. (2024) · AJOG Global Reports · doi:10.1016/j.xagr.2024.100414 · Cited by 6 · see citing works
Moglia, M., Tapsuwan, S., Ghaderi, ., Nygaard, C. A., Dia, H., & Smith, D. A. (2025) · Transportation Research Part A Policy and Practice · doi:10.1016/j.tra.2025.104396 · Cited by 6 · see citing works
Ho, T. Q. A., Engel, L., Ride, J., Le, L. K., Melvin, G. A., Le, H., & Mihalopoulos, C. (2025) · Applied Health Economics and Health Policy · doi:10.1007/s40258-025-00958-9 · Cited by 6 · see citing works
Preferences in atopic dermatitis: A systematic review
Pinborg, E. H., Laursen, S. B., Thein, D., Vittrup, I., Schlapbach, C., Maul, J., Dégboé, B., Egeberg, A., Thomsen, S. F., Thyssen, J. P., et al. (2025) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.20753 · Cited by 6 · see citing works
Understanding Family Forest Landowners’ Preferences for Carbon Offset Programs in Central Appalachia
Gazal, K., Eastman, B. A., Cheye, S., Arano, K., & Dahle, G. (2025) · Forests · doi:10.3390/f16091415 · Cited by 5 · see citing works
Women’s preferences for testing to predict breast cancer risk – a discrete choice experiment
Wang, Y., Ho, P. J., Mou, L., & Li, J. (2025) · Journal of Translational Medicine · doi:10.1186/s12967-025-06119-9 · Cited by 5 · see citing works
Nguyen, H. T. T., Vo, T. Q., Tran, H. T. B., Nguyen, B. T., Nguyen, H. T., Nguyen, T. D., & Anuratpanich, L. (2022) · The International Journal of Health Planning and Management · doi:10.1002/hpm.3597 · Cited by 5 · see citing works
Economic Dimensions of Air Pollution: Cost Analysis, Valuation, and Policy Impacts
Obahiagbon, E. G., & Kosoe, E. A. (2024) · The handbook of environmental chemistry · doi:10.1007/698_2024_1116 · Cited by 5 · see citing works
Mishkin, K. E., Li, Q., Earla, J. R., Mejia, J. A., Hirshfield, K. M., Krupsky, K. L., Lankin, J., Beusterien, K. M., Mulvihill, E., Honomichl, R. D., et al. (2025) · Patient · doi:10.1007/s40271-025-00783-1 · Cited by 4 · see citing works
Fischer, A.-K., Sadler, A., Kohlmann, T., & Mühlbacher, A. C. (2026) · The European Journal of Health Economics · doi:10.1007/s10198-026-01914-7 · Cited by 4 · see citing works
El‐Toukhy, S., Pike, J. R., Zuckerman, G., & Hegeman, P. (2023) · JMIR Research Protocols · doi:10.2196/47567 · Cited by 4 · see citing works
Willing to Wait? Acceptance for Load Management at e-Vehicle Charging Stations in Germany
Fabianek, P., & Madlener, R. (2024) · International Conference on the European Energy Market (EEM) · doi:10.1109/eem60825.2024.10608842 · Cited by 4 · see citing works
Gribben, J. G., Bachy, E., Ray, M. D., Krupsky, K. L., Beusterien, K. M., Kopenhafer, L., Beygi, S., Best, T., Ball, G., Will, O., et al. (2025) · Frontiers in Oncology · doi:10.3389/fonc.2025.1589722 · Cited by 3 · see citing works
The use of discrete choice experiments in evaluating telehealth: a systematic review
Savira, F., Frith, M., Nguyen, D., Norman, R., Senyel, D., Boyd, J., & Robinson, S. (2025) · Health Policy · doi:10.1016/j.healthpol.2025.105421 · Cited by 3 · see citing works
Maina, C. N., Kaggiah, A., Barthold, D., Tran, J., Saldarriaga, E. M., Simoni, J. M., Hauber, B., Kinuthia, J., & Graham, S. M. (2025) · AIDS Care · doi:10.1080/09540121.2025.2587906 · Cited by 3 · see citing works
Women’s preferences for HPV self-sampling in cervical cancer screening: a discrete choice experiment
Ding, N., Chen, Y., Zhou, X., Wang, G., Li, J., & Wang, X. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1779443 · Cited by 3 · see citing works
Eliciting Preferences for the Uptake of Smoking Cessation Apps: Discrete Choice Experiment
Szinay, D., Cameron, R. A., Jones, A., Whitty, J. A., Chadborn, T., Brown, J., & Naughton, F. (2024) · Journal of Medical Internet Research · doi:10.2196/37083 · Cited by 3 · see citing works
Self-citationTradeoffs in Triage of Motor Vehicle Trauma by Rural 911 Emergency Medical Services Practitioners
Benson, J. S., Wolfson, D. L., & van den Broek‐Altenburg, E. M. (2023) · Medical Decision Making · doi:10.1177/0272989x221145677 · Cited by 3 · see citing works
Xu, Y., Xiao, D., Li, Y., & Hu, L. (2025) · Nursing and Health Sciences · doi:10.1111/nhs.70026 · Cited by 2 · see citing works
Ren, Y., Li, Y., Wang, W., Lv, R., Fang, X., Wang, Y., & Zhong, J. (2025) · Digital Health · doi:10.1177/20552076251319662 · Cited by 2 · see citing works
Sun, X., Zhang, J., Wu, Y., Duan, T., & Min, H. (2025) · Journal of Applied Gerontology · doi:10.1177/07334648251398602 · Cited by 2 · see citing works
Checco, J., Demont, M., Coote, L. V., Schulze, M., Permani, R., Muflikh, Y. N., Mitchell, J. H., & Aziz, A. A. (2025) · Food Quality and Preference · doi:10.1016/j.foodqual.2025.105812 · Cited by 2 · see citing works
Willing to wait? Acceptance of load management at e-vehicle charging stations
Fabianek, P., & Madlener, R. (2025) · Transportation Research Interdisciplinary Perspectives · doi:10.1016/j.trip.2025.101677 · Cited by 2 · see citing works
Wang, W., Khalajzadeh, H., Grundy, J., Madugalla, A. K., & Obie, H. O. (2026) · Information and Software Technology · doi:10.1016/j.infsof.2026.108014 · Cited by 2 · see citing works
Personalizing Font Size Normalization: Linguistic Background Shapes Which Font Attributes Matter
Suhail, Y., Khadka, K., Llana, B., Hosseinpour, M., & Wallace, S. (2026) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3774935.3806166 · Cited by 2 · see citing works
Tao, W., Bao, T., Gu, T., Pan, J., Li, W., & Li, R. (2024) · International Journal of Health Policy and Management · doi:10.34172/ijhpm.8259 · Cited by 2 · see citing works
Kong, Q., de Vries, H., Poyraz, D. D., & Kayyal, A. (2024) · Social Science & Medicine · doi:10.1016/j.socscimed.2024.117637 · Cited by 2 · see citing works
Choi, S. H., Templin, T., & Glenn, D. (2025) · Health Education Research · doi:10.1093/her/cyaf010 · Cited by 2 · see citing works
Lee, J., Kim, J. H., Choi, M., & Shin, J. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01610-5 · Cited by 2 · see citing works
Manson, A., Golley, R. K., Brown, V., McCann, D., & Johnson, B. J. (2025) · Appetite · doi:10.1016/j.appet.2025.108335 · Cited by 1 · see citing works
Tobin, D., Pfaff, A. S. P., Monroy, D., Salgado-Almeida, B., Kiefer, A. M., & Garcés, D. (2025) · Environment and Development Economics · doi:10.1017/s1355770x2510017x · Cited by 1 · see citing works
Sarker, J. R., Rolfe, J. C., & Ananda, J. (2026) · PLoS ONE · doi:10.1371/journal.pone.0347541 · Cited by 1 · see citing works
Dhir, A. M., Hanrahan, C. F., Kaptchuk, R. P., Singh, A., Farley, J. E., & Clipman, S. J. (2026) · Journal of the Association of Nurses in AIDS Care · doi:10.1097/jnc.0000000000000653 · Cited by 1 · see citing works
Neuendorf, N. L., Angermayr, K., & Scherr, S. (2026) · Social Science & Medicine · doi:10.1016/j.socscimed.2026.119431 · Cited by 1 · see citing works
Dhir, A. M., Clipman, S. J., Farley, J. E., & Singh, A. (2026) · Journal of the Association of Nurses in AIDS Care · doi:10.1097/jnc.0000000000000655 · Cited by 1 · see citing works
Coulibaly, A., Badolo, F., Yila, J. O., Koné, B., Jumbo, M. B., Ogunbayo, A., & Savadogo, K. (2026) · Journal of Agriculture and Food Research · doi:10.1016/j.jafr.2026.102701 · Cited by 1 · see citing works
Audibert, C., Borghini-Fuhrer, I., Gaaloul, M. E., Ramachandruni, H., Tchouatieu, A.-M., & Lugand, M. M. (2026) · PLOS Global Public Health · doi:10.1371/journal.pgph.0005607 · Cited by 1 · see citing works
Gómez-Valenzuela, V., Holl, A., Oviedo, J. L., & Martínez, C. (2026) · Science and Public Policy · doi:10.1093/scipol/scag020 · Cited by 1 · see citing works
Belay, Y. A., Yitayal, M., Atnafu, A., & Taye, F. A. (2024) · MDM Policy & Practice · doi:10.1177/23814683241273635 · Cited by 1 · see citing works
Alemu, M. B., Norman, R., Dantas, J. A. R., Belay, D. G., Haile, T. G., Pereira, G. F., & Tessema, G. A. (2025) · Sexual and Reproductive Health Matters · doi:10.1080/26410397.2025.2520682 · Cited by 1 · see citing works
Qin, J., Wu, Z., Zhang, H., & Qi, X. (2026) · Frontiers in Medicine · doi:10.3389/fmed.2026.1749507
“…efficiency and avoid redundancy, this study adopted a D-efficient design, which reduces the number of choice cards while maintaining the precision of parameter estimation, thereby enabling participants to provide more efficient data contributions within a limited number of tasks (59, 60).”
Stum, M. L., Clavé, A., Agbemanyole, K. A., Stindel, É., & Goff-Pronost, M. L. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e30041
Frith, M., Boyd, J., Savira, F., Senyel, D., Norman, R., Jones, J. M., & Robinson, S. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-082577
Bowker-Lonnecker, L., Dixon, P., Petrou, S., & Buckell, J. (2024) · medRxiv · doi:10.1101/2024.10.18.24315590
Pham, G. H., Tran, Q. V., Bui, H. T., Nguyen, T. D., Vo, T. Q., Jedsadayanmata, A., Huynh, B. H., & Sholihat, N. K. (2025) · Narra J · doi:10.52225/narra.v5i1.1107
Senanayake, S. J., Barnett, A. G., Brain, D., Allen, M. J., Powell, E. E., O’Beirne, J. P., Valery, P. C., Hickman, I. J., & Kularatna, S. (2025) · F1000Research · doi:10.12688/f1000research.157017.1
Smallholder farmers and aggregation success: a stakeholders’ perspective
Marisetty, V. B., & Nema, S. (2025) · Applied Economics · doi:10.1080/00036846.2025.2490858
Key information influencing patient decision-making about AI in healthcare – A survey experimentPreprint
Zhu, X., Stroud, A. M., Minteer, S. A., Yoo, D. W., Ridgeway, J. L., Mooghali, M., Miller, J. E., & Barry, B. A. (2025) · doi:10.31219/osf.io/4hx7r_v2
Bugnon, B., Nicolet, A., Marti, J., Bonnabry, P., & von Plessen, C. (2025) · Journal of Participatory Medicine · doi:10.2196/preprints.78056
Violette, L. R., Delacruz, J., Niemann, L. A., Cannon, C. A., Stekler, J. D., & Katz, D. A. (2025) · JAIDS Journal of Acquired Immune Deficiency Syndromes · doi:10.1097/qai.0000000000003718
Zhang, J., Li, L., Xi, H., Wang, Q., Jiang, F., Sun, X., Qiao, X., & Xu, A. (2025) · BMC Complementary Medicine and Therapies · doi:10.1186/s12906-025-04937-4
Hamamoto, Y., Inagawa, A., Yamamoto, T., Ito, H., & Matsumoto, H. (2025) · Esophagus · doi:10.1007/s10388-025-01143-6
Caivano, V., Lucarelli, C., Mazzocchini, F. J., & Soccorso, P. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5382972
Shah, M. V., Solem, C. T., Liao, A., Bell, K. F., Wang, Y., Yang, H., Meng, Y., Ye, M., & Tapan, Ü. (2025) · Advances in Therapy · doi:10.1007/s12325-025-03347-9
Feasibility study of a digital NHS Health Check
Gidlow, C. J., Sams, L., Riley, V., Ellis, N., Povey, R. C., Nolan, A., Walsh, G., Green, H., & Thompson, K. (2025) · BMC Digital Health · doi:10.1186/s44247-025-00161-9
Factors Influencing Foodborne Illness Self-Identification From User-Generated Data – Minnesota, 2024
Krampf, L., Hedberg, C. W., Hauber, B., Walmsley, A., & Firestone, M. J. (2025) · Journal of Food Protection · doi:10.1016/j.jfp.2025.100612
Ashaye, A. O., Ramachandran, N., Quaife, M., Wu, Y., Gutiérrez-Vargas, Á. A., Jiongco, A., Kota, V. K., Savani, B. N., & Thomas, C. (2025) · Future Oncology · doi:10.1080/14796694.2025.2565497
Najafi, M., Hallajian, H., Aghasi, M. E., Golshekan, M., Taramsari, M. R., Davoudi-Kiakalayeh, A., & Rad, E. H. (2025) · Traffic Injury Prevention · doi:10.1080/15389588.2025.2554855
Le, H., Coulter, J., Culver, K., Cappelleri, J. C., Rifi, N., Lu, H., Quaife, M., Meginnis, K., Fernández, G., Vaghela, S., et al. (2025) · Lung Cancer · doi:10.1016/j.lungcan.2025.108850
Mpirirwe, R., Mujugira, A., Agaba, B. B., Nangendo, J., Dennis, M. L., Semitala, F. C., Kyambadde, P., Kalyango, J. N., Karamagi, C., Kiragga, A. K., et al. (2025) · medRxiv · doi:10.1101/2025.11.20.25340720
Solano-Hermosilla, G., & Barreiro-Hurle, J. (2025) · Economía Agraria y Recursos Naturales · doi:10.7201/earn.2025.02.07
Formosa, L., Vargas, R., & Wilson, L. (2025) · Journal of the American Pharmacists Association · doi:10.1016/j.japh.2025.103010
Wang, J., Song, P., Li, Y., Zhang, Y., Wu, T., & Zhou, B. (2025) · IEEE Access · doi:10.1109/access.2025.3649758
WeQuit: Pilot Results for a Couple-Based Smoking Cessation Web App
Choi, S. H., Jenuwine, E., YARANDI, H. N., Zhang, X., & Ledgerwood, D. M. (2026) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntag001
Post-stroke individual preferences for lower limb rehabilitation exoskeletonsPreprint
Agbemanyole, K. A., Stum, M. L., Leplaideur, S., Brunet-Houdard, S., THIEBAUT-CLERIN, M., Plouzennec, J.-P., Coignard, P., Lenca, P., Rémy‐Néris, O., & Goff-Pronost, M. L. (2026) · Research Square · doi:10.21203/rs.3.rs-8620015/v1
Aghdaei, Z. A. T., Coote, L. V., McColl‐Kennedy, J. R., & Erdem, S. (2026) · Acta Psychologica · doi:10.1016/j.actpsy.2026.106394
Gong, Y. (2026) · Applied Health Economics and Health Policy · doi:10.1007/s40258-026-01033-7
Understanding Conservation Decision-makers’ Preferences for EvidencePreprint
Jeevachandran, A., & Christie, A. (2026) · doi:10.32942/x2wd4b
Wright, S. J., Thapa, S., Salisbury, A., Hindmarch, S., French, D., Howell, S. J., & Payne, K. (2026) · BJC Reports · doi:10.1038/s44276-026-00209-x
Preferences for Choosing Automated Buses: A Discrete Choice Experiment
Fonzone, A., & Downey, L. (2026) · Transportation research procedia · doi:10.1016/j.trpro.2026.02.133
Gardner, G. E., Sampson, G. S., & Zhao, S. (2026) · Australian Journal of Agricultural and Resource Economics · doi:10.1111/1467-8489.70081
Ngugi, S. W., Echoka, E., Were, V. O., Thuo, N. B., Ogello, V., Gikwa, M., Muteti, N., Mwangi, P. M., Kazungu, J., & Ngure, K. (2026) · Scientific Reports · doi:10.1038/s41598-026-45900-8
Preferences for Infection Prevention in Hospital-Acquired Infections
Elangovan, S., Senanayake, S. J., Venkatachalam, I., Ling, M. L., Graves, N., & Kularatna, S. (2026) · JAMA Network Open · doi:10.1001/jamanetworkopen.2026.7432
Proposal for m-health modular architecture tailored for public safety officers
Fettermann, D. C., Calegari, L. P., Borriello, A., & Bouzon, M. (2026) · International Journal of Occupational Safety and Ergonomics · doi:10.1080/10803548.2026.2652145
Sun, H., Shi, Z., Xia, Y., Bai, Y., Yu, M., Liu, R., & He, P. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-115755
Telehealth and Remote Monitoring Solutions: Advancing Accessible and Effective Healthcare Delivery
(2026) · Frontiers research topics · doi:10.3389/978-2-8325-8113-1
Angermayr, K., Neuendorf, N. L., & Scherr, S. (2026) · Health Communication · doi:10.1080/10410236.2026.2666885
Consumers’ preferences for energy efficiency in refrigerators: a choice experiment in South Korea
Kim, J.-Y., Choi, H., & Yoo, S. (2026) · Energy Efficiency · doi:10.1007/s12053-026-10451-4
Dong, Y. (2026) · Society & Natural Resources · doi:10.1080/08941920.2026.2693816
- Authors
- Szinay, D., Cameron, R., Naughton, F., Whitty, J. A., Brown, J., & Jones, A.
- Affiliations
- University of East Anglia (GB)
- University College London (GB)
- Corresponding author
- Szinay, Dorothy
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2021-10-11
- Volume
- 23
- Issue
- 10
- Pages
- e32365
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC8546533
- OpenAlex
- W3200143140
- Semantic Scholar
- 599563c0f55d31b0ab4dbdb1b3e4999ce36f24e7
- Cited by
- 106
- References
- 60
- Citations by year
- 2026: 32 · 2025: 44 · 2024: 20 · 2023: 7 · 2022: 3
- FWCI
- 5.72
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- other-oa
- Locations
- Journal of Medical Internet Research (publishedVersion, other-oa)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- UEA Digital Repository (University of East Anglia) (acceptedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Griffith Research Online (submittedVersion, cc-by)
- Funders
- Cancer Research UK (22962)
- National Institute for Health and Care Research
- Department of Health and Social Care
- University of Leeds
- University of Sydney
- Medical Research Council (MR/S037519/1)
- Topics
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Economic and Environmental Valuation
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
- Health Sciences › Health Professions › Health Information Management › Health, Technology, Consumer Behavior
- Keywords
- discrete choice experiment, digital health products, uptake, stated preference, smoking cessation apps, user preferences, experimental design, choice task, health technology assessment
- MeSH terms
- Bayes Theorem, Choice Behavior, Health Policy, Health Services, Humans, Patient Preference
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
