The DOI matches its Crossref record; 78 of the 78 references it cites were found in a registry or an index; 5 of the 92 sources its claims cite were checked for support.
Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research
Open accessAbstract
BACKGROUND: To improve the implementation of innovations in healthcare settings, it is important to understand factors influencing healthcare professionals' behaviors. We aimed to develop a generic questionnaire in English and in Dutch assessing the 14 domains of behavioral determinants from the revised TDF (Cane et al., 2012) that can be tailored to suit different targets, actions, contexts, and times of interest, and to investigate questionnaire items' discriminant content validity. METHODS: We identified existing questionnaires including items assessing constructs within TDF domains and developed new items where needed. Nineteen judges allocated 79 items to one or more TDF domains. One-sample t-tests were used to examine the discriminant content validity of each item, i.e., whether items measured intended domains or whether items measured a combination of domains. RESULTS: We identified items judged to discriminately measure 11 out of 14 domains. Items measuring the domains Reinforcement, Goals, and Behavioral regulation were judged to measure a combination of domains. CONCLUSIONS: We have developed a questionnaire in English and in Dutch able to discriminately assess the majority of TDF domains. The results partly support Cane et al.'s (2012) 14-domain validation of the TDF and suggest that Michie et al.'s (2005) 12-domain original version might be more applicable in developing a TDF-based questionnaire. The identified items provide a robust basis for developing a questionnaire to measure TDF-based determinants of healthcare professionals' implementation behaviors to suit different targets, actions, contexts, and times. Future research should investigate the concurrent and predictive validity and reliability of such a questionnaire in practice.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This study developed a generic English and Dutch Theoretical Domains Framework (TDF) questionnaire and tested the discriminant content validity of its items using expert judges. Of 79 items, 32 were judged to discriminately measure 11 of the 14 revised TDF domains, while items for Reinforcement, Goals, and Behavioral regulation were not.
Key findings
- Nineteen judges allocated 79 items to TDF domains; 32 items were classified as measuring their intended domain, covering 11 of 14 domains: Knowledge, Skills, Social/professional role and identity, Beliefs about capabilities, Optimism, Beliefs about consequences, Intentions, Memory/attention/decision processes, Environmental context and resources, Social influences, and Emotion.
- Items intended to measure Reinforcement, Goals, and Behavioral regulation were not classified to their intended domains and were excluded from the final questionnaire.
- Inter-rater agreement was moderate for all 79 items (Light’s κ = .56, 95% CI .50–.62) and almost perfect for the final 32 items (κ = .82, 95% CI .79–.85).
- The authors conclude the results partly support Cane et al.’s 14-domain TDF and suggest Michie et al.’s original 12-domain version might be more applicable for developing a TDF-based questionnaire.
Methods
- The authors developed a questionnaire with 79 items assessing the 14 revised TDF domains, using existing validated items where available and newly developed items where needed; items used a generic “[action] in [context, time] with [target]” TACT construction and were developed in English, translated to Dutch, and back-translated.
- Fifty-eight academics were approached and 19 participated (33% response rate); they rated their expertise on each domain and allocated each item to up to three TDF domains, rating confidence in each allocation from 0% to 100%.
- Discriminant content validity was tested with one-sample one-tailed t-tests on confidence-weighted judgments, with false discovery rate correction; Light’s kappa assessed inter-rater agreement.
Limitations
- The authors state the DCV task was challenging because judges allocated 79 items across 14 domains; a larger number of judges and a less complex task might have provided more information, and validity as perceived by an academic audience may differ from “real life.”
- The authors state the final questionnaire requires refinement: most domains need more items (at least three per domain), Environmental context and resources items were framed only in socio-political terms, Emotion items assessed general Stress rather than behavior-specific Affect, and Knowledge could be improved with additional items.
- Evident from the text: the final questionnaire contains no items for Reinforcement, Goals, or Behavioral regulation and only two items for several domains, and concurrent validity, predictive validity, and reliability were not tested in this study; the authors call for future research on these properties.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
82 Own result · 39 Not checked yet · 12 Own interpretation · 6 Own method · 5 No basis given · 1 Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We identified items judged to discriminately measure 11 out of 14 domains.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Abstract > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Items measuring the domains Reinforcement, Goals, and Behavioral regulation were judged to measure a combination of domains.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Abstract > Results
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Items measuring the domains Reinforcement, Goals, and Behavioral regulation were judged to measure a combination of domains.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Abstract > Results
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Items measuring the domains Reinforcement, Goals, and Behavioral regulation were judged to measure a combination of domains.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Abstract > Results
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Forty-seven items were allocated to more than one domain, of which 39 items were allocated to the intended domain as well as additional domains, while eight items were classified as measuring a domain other than the item aimed to measure.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > DCV results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With all items included, moderate agreement between judges was found for their allocation of items to the domain (κ = .55; 95% C.I. .41-.71; Table 4), while substantial agreement was found for the 32 final items (κ = .73; 95% C.I. .60-.81; Table 4).”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > DCV results > Beliefs about capabilities
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This was justified as the data indicated that judges used the first selected domain as the most preferable domain (i.e., domain with the highest confidence ratings) to allocate an item to.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Methods > Inter-rater agreement
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Forty-seven items were allocated to more than one domain, of which 39 items were allocated to the intended domain as well as additional domains, while eight items were classified as measuring a domain other than the item aimed to measure.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > DCV results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With all items included, moderate agreement between judges was found for their allocation of items to the domain (κ = .55; 95% C.I. .41-.71; Table 4), while substantial agreement was found for the 32 final items (κ = .73; 95% C.I. .60-.81; Table 4).”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > DCV results > Beliefs about capabilities
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The extent to which judges agreed on which items measured the domain was moderate with all items included (κ = .59; 95% C.I. .37-.75; Table 4) and almost perfect for the 32 final items (κ = .86; 95% C.I. .72-.93; Table 4).”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > DCV results > Social/professional role and identity
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 35
- It may state two separate claims as one. 37
Every source the paper cites for this claim was checked and supports it.
In the paper
“The framework has been used in a number of studies and was demonstrated to be useful for the development of qualitative [32,33] and quantitative [34-36] measurement tools to assess potential implementation behavior determinants.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Background
What the sources say
- Supportedcites [32] Patey AM, Islam R, Francis JJ, Bryson GL, Grimshaw JM: Anesthesiologists’ And surgeons’ perceptions about routine pre-operative testing in low-risk patients: application of the theoretical domains framework (TDF) to identify factors that influence physicians’ decisions to order pre-operative tests. Implement Sci. 2012, 7: 52-10.1186/1748-5908-7-52. source
The Background states that this study used the TDF to examine anesthesiologists’ and surgeons’ beliefs about pre-operative testing. It also identifies the article as one in a series documenting TDF use, so this paper supports its citation as an example of a study using the framework. (LLM, conf=0.92)
- Supportedcites [33] McSherry LA, Dombrowski SU, Francis JJ, Murphy J, Martin CM, O’Leary JJ, Sharp L, ATHENS Group: 'It’s a can of worms’: understanding primary care practitioners’ behaviours in relation to HPV using the theoretical domains framework. Implement Sci. 2012, 7: 73-10.1186/1748-5908-7-73. source
The Background identifies this as a study that used the TDF and describes it as one of a series on the framework’s use. The Discussion (“Reflections on using the TDF”; page unavailable) describes applying it to qualitative interview data. This paper therefore supports the citing sentence by providing an example of TDF use in a qualitative study. (LLM, conf=0.90)
- Supportedcites [34] Amemori M, Michie S, Korhonen T, Murtomaa H, Kinnunen TH: Assessing implementation difficulties in tobacco use prevention and cessation counselling among dental providers. Implement Sci. 2011, 6: 50-10.1186/1748-5908-6-50. source
The Methods section says the authors developed a questionnaire based on the theoretical-domains framework to assess factors affecting guideline implementation. The Discussion (Main findings) identifies the study as a quantitative application of that framework, supporting its citation as one of the studies using it to develop a measurement tool. (LLM, conf=0.92)
- Supportedcites [35] Beenstock J, Sniehotta FF, White M, Bell R, Milne EM, Araujo-Soares V: What helps and hinders midwives in engaging with pregnant women about stopping smoking? A cross-sectional survey of perceived implementation difficulties among midwives in the northeast of England. Implement Sci. 2012, 7: 36-10.1186/1748-5908-7-36. source
The Background describes several earlier studies using the TDF. This paper also uses it in a self-completion questionnaire to assess midwives’ perceived implementation difficulties, and the Conclusion calls it an appropriate way to understand healthcare professionals’ behaviour. It therefore supports the cited point about use in quantitative measurement, while noting that the questionnaire needs refinement. (LLM, conf=0.91)
- Supportedcites [36] Taylor N, Parveen S, Robins V, Slater B, Lawton R: Development and initial validation of the Influences on Patient Safety Behaviours Questionnaire. Implement Sci. 2013, 8: 81-10.1186/1748-5908-8-81. source
The Background says researchers have applied the TDF to healthcare practitioner behavior using qualitative interviews, indicating use in multiple studies. The Discussion also reports this paper’s development and testing of questionnaire scales based on the framework. (LLM, conf=0.91)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Of the six Knowledge items included in the DCV exercise, four items were classified as measuring the domain Knowledge (Table 4) and were included in the final questionnaire.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > DCV results > Knowledge
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All four Social/professional role and identity items included in the DCV were classified as measuring the intended domain (Table 3) and were included in the final questionnaire.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > DCV results > Social/professional role and identity
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Neither judges’ expertise with TDF domains nor their academic level (i.e., PhD student, PhD, Professor) was related to their performance on the classification of items to domains calculated as the number of 'matches’.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Results > Judges’ expertise in the use of domains
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Three out of four Skills items included in the DCV were classified as measuring the intended domain (Table 3) and were included in the final questionnaire.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > DCV results > Skills
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Only three judges indicated to be a layman on, respectively, one, two, and seven domains.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Results > Judges’ expertise in the use of domains
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The three items containing the word 'confident’ were classified as measuring the intended domain (Table 3) and were included in the final questionnaire.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > DCV results > Beliefs about capabilities
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On average, judges rated that they had most expertise on the domains Intentions and Goals, whereas lowest expertise ratings were given to the domains Social/professional role and identity, and Memory, attention, and decision processes.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Results > Judges’ expertise in the use of domains
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The item 'How much control do you have over…’ was allocated to the intended domain, but also to the domains Skills and Behavioral regulation.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > DCV results > Beliefs about capabilities
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Two out of six Optimism items included in the DCV were classified as measuring the domain Optimism (Table 3). These were included in the final questionnaire.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > DCV results > Optimism
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The items measuring the difficulty and possibility of [action] in [context, time] with [target] were allocated to more than one domain. In addition to the domain Beliefs about capabilities, they were often allocated to the domain Skills.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > DCV results > Beliefs about capabilities
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Eight Goals items were included in the DCV exercise. None of them were classified to the right domain (Table 3) and thus Goals items were not included in the final questionnaire.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > DCV results > Goals
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“items measuring the domain Emotion were adapted from previously published questionnaires”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Eight items were included in the DCV exercise to measure the domain Memory, attention, and decision processes. Four of these items were classified to measure the intended domain (Table 3) and were included in the final questionnaire.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > DCV results > Memory, attention, and decision processes
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The DCV exercise included three items intended to measure Reinforcement, but none of them was classified as measuring the domain (Table 3) and so none of them was included in the final questionnaire.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > DCV results > Reinforcement
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Items intended to measure these domains were allocated to multiple domains or classified to a domain other than the item intended to measure.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The four items measuring the construct Action planning were included in the DCV as measuring both the domain Goals and Behavioral regulation. They were not classified as measuring these two domains, because they were also often allocated to the domain Intentions.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > DCV results > Goals
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The item '…I get recognition from professionals who are important to me’ was classified as measuring the domain Social influences and the item '…I feel like I am making a difference’ was classified as measuring the domain Beliefs about consequences.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > DCV results > Reinforcement
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“five, three, and two judges did not allocate any of the items to, respectively, the domains Reinforcement, Goals, and Behavioral regulation”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Of the four Emotion items included in the DCV exercise, the two items measuring the construct Stress were classified as measuring the intended domain (Table 3). These items were included in the final questionnaire.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > DCV results > Emotion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“the items measuring these domains were allocated to all three domains, and thus are not able to discriminately measure them”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Eight items were included in the DCV to measure this domain, while only two items were classified as measuring the domain (Table 3) and therefore could be included in the final questionnaire.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > DCV results > Environmental context and resources
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Ten items, including Action planning items also aimed to measure the domain Goals, were included in the DCV to measure Behavioral regulation. None of them were classified to the right domain (Table 3) and therefore Behavioral regulation items were not included in the final questionnaire.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > DCV results > Behavioral regulation
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All four items included in the DCV to measure Intentions were classified as measuring the domain (Table 3) and included in the final questionnaire.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > DCV results > Intentions
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Items measuring the construct Stress demonstrated to be able to discriminately assess the domain Emotions, while Affect items did not.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Discussion
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Two items were allocated to more than one domain and two items measuring the construct Memory were classified as measuring a domain other than they were intended to measure (i.e., Knowledge and Beliefs about capabilities).”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > DCV results > Memory, attention, and decision processes
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“primary care-based interventions for increasing physical activity (PA) are effective [4-7]”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Background
What the sources say
- Not checked yetcites [4] Eakin EG, Glasgow RE, Riley KM: Review of primary care-based physical activity intervention studies. J Fam Practice. 2000, 49: 158-168.
- Not checked yetcites [5] Petrella RJ, Lattanzio CN: Does counseling help patients get practice? Systematic review of the literature. Can Fam Physician. 2002, 48: 72-80.
- Not checked yetcites [6] Sørensen JB, Skovgaard T, Puggaard L: Exercise on prescription in general practice: a systematic review. Scand J Prim Health. 2006, 24: 69-74. 10.1080/02813430600700027. source
- Not checked yetcites [7] Orrow G, Kinmonth A-L, Sanderson S, Sutton S: Effectiveness of physical activity promotion based in primary care: systematic review and meta-analysis of randomised controlled trials. BMJ. 2012, 344: e1389-10.1136/bmj.e1389. source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Improving the adoption and implementation of evidence-based interventions into routine practice involves changes in healthcare professionals’ behaviors that may be influenced by a range of individual, organizational, and social factors [16-20].”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Background
What the sources say
- Not checked yetcites [16] Fleuren M, Wiefferink K, Paulussen T: Determinants of innovation within health care organizations: literature review and Delphi study. Int J Qual Health C. 2004, 16: 107-123. 10.1093/intqhc/mzh030. source
- Not checked yetcites [17] Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O: Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004, 82: 581-629. 10.1111/j.0887-378X.2004.00325.x. source
- Not checked yetcites [18] Grol R, Wensing M, Eccles M: Improving Patient Care: The Implementation of Change in Clinical Practice. 2005, Oxford: Elsevier
- Not checked yetcites [19] Grol R, Grimshaw J: From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003, 362: 1225-1230. 10.1016/S0140-6736(03)14546-1. source
- Not checked yetcites [20] Rogers EM: Diffusion of Innovations. 1983, New York: The Free Press
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“Identifying the key factors associated with healthcare professional behavior can provide a basis for developing interventions to help healthcare professionals to use research findings more effectively [14].”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Background
What the sources say
- Not checked yetcites [14] Foy R, Eccles M, Grimshaw J: Why does primary care need more implementation research?. Fam Pract. 2001, 18: 353-355. 10.1093/fampra/18.4.353. source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
None of the sources cited for this claim have been checked yet.
In the paper
“This gap between research and practice reduces the impact that effective behavior change interventions can have on public health [12,13].”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Background
What the sources say
- Not checked yetcites [12] Glasgow RE, Bull SS, Gilette C, Klesges LM, Dzewaltowski DA: Behavior change intervention research in healthcare settings: a review of recent reports with emphasis on external validity. Am J Prev Med. 2002, 23: 62-69. 10.1016/S0749-3797(02)00437-3. source
- Not checked yetcites [13] Bartholomew KL, Parcel GS, Kok G, Gottlieb NH, Fernández ME: Planning Health Promotion Programs: An Intervention Mapping Approach. 2011, San Fransisco: Jossey-Bass
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“many advocate the use of theory to guide the selection of factors to investigate [15,21-23].”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Background
What the sources say
- Not checked yetcites [15] Eccles M, Grimshaw J, Walker A, Johnston M, Pitts N: Changing the behavior of healthcare professionals: the use of theory in promoting the uptake of research findings. J Clin Epidemiol. 2005, 58: 107-112. 10.1016/j.jclinepi.2004.09.002. source
- Not checked yetcites [21] Francis JJ, Tinmouth A, Stanworth SJ, Grimshaw JM, Johnston M, Hyde C, Stockton C, Brehaut JC, Fergusson D, Eccles MP: Using theories of behaviour to understand transfusion prescribing in three clinical contexts in two countries: development work for an implementation trial. Implement Sci. 2009, 4: 70-10.1186/1748-5908-4-70. source
- Not checked yetcites [22] Grol RPTM, Bosch MC, Hulscher MEJL, Eccles MP, Wensing M: Planning and studying improvement in patient care: the use of theoretical perspectives. Milbank Q. 2007, 85: 93-138. 10.1111/j.1468-0009.2007.00478.x. source
- Not checked yetcites [23] Michie S, Johnston M, Francis J, Hardeman W, Eccles M: From theory to intervention: mapping theoretically derived behavioural determinants to behaviour change techniques. Appl Psychol. 2008, 57: 660-680. 10.1111/j.1464-0597.2008.00341.x. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“the UK Medical Research Council guidance on developing and evaluating complex interventions recommends the use of theory in the intervention development phase [24].”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Background
What the sources say
- Not checked yetcites [24] Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M: Developing and evaluationg complex interventions: the new medical research council guidance. BMJ. 2008, 337: a1655-10.1136/bmj.a1655. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The questionnaire was developed in English, then translated to Dutch and back-translated to English by an independent translator.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Methods > Materials
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We have developed a TDF-based questionnaire in both English and Dutch able to discriminately assess the majority of domains.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“The sample size was based on estimates of between three and 20 participants as adequate for judgment tasks [39,40].”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Methods > Participants
What the sources say
- Not checked yetcites [39] Lynn MR: Determination and quantification of content validity. Nurs Res. 1986, 35: 382-385. source
- Not checked yetcites [40] Rubio DM, Berg-Weger M, Tebb SS, Lee ES, Rauch S: Objectifying content validity: conducting a content validity study in social work research. Soc Work Res. 2003, 27: 94-104. 10.1093/swr/27.2.94. source
plainn = 19- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The Theoretical Domains Framework (TDF) [31] was developed as an integrative framework of theories of behavior change to overcome these challenges.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Background
What the sources say
- Not checked yetcites [31] Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A: Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005, 14: 26-33. 10.1136/qshc.2004.011155. source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“A generalization of Cohen’s kappa (i.e., Light’s Kappa [59]) was calculated to assess agreement between judges across their allocation of all items to domains.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Methods > Inter-rater agreement
What the sources say
- Not checked yetcites [59] Conger A: Integration and generalization of kappas for multiple raters. Psychol Bull. 1980, 88: 322-328. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“Since its original development, the consensus study that produced the TDF [31] has been validated, leading to Cane et al.’s [37] refined TDF.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Background
What the sources say
- Not checked yetcites [31] Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A: Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005, 14: 26-33. 10.1136/qshc.2004.011155. source
- Not checked yetcites [37] Cane J, O’Connor D, Michie S: Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012, 7: 37-10.1186/1748-5908-7-37. source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“and internal consistency reliability was low [34] or could be improved [35,36].”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Background
What the sources say
- Not checked yetcites [34] Amemori M, Michie S, Korhonen T, Murtomaa H, Kinnunen TH: Assessing implementation difficulties in tobacco use prevention and cessation counselling among dental providers. Implement Sci. 2011, 6: 50-10.1186/1748-5908-6-50. source
- Not checked yetcites [35] Beenstock J, Sniehotta FF, White M, Bell R, Milne EM, Araujo-Soares V: What helps and hinders midwives in engaging with pregnant women about stopping smoking? A cross-sectional survey of perceived implementation difficulties among midwives in the northeast of England. Implement Sci. 2012, 7: 36-10.1186/1748-5908-7-36. source
- Not checked yetcites [36] Taylor N, Parveen S, Robins V, Slater B, Lawton R: Development and initial validation of the Influences on Patient Safety Behaviours Questionnaire. Implement Sci. 2013, 8: 81-10.1186/1748-5908-8-81. source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
In the paper
“Our findings provide an additional level of validation for the content of the TDF: not only do judges agree about the constructs within each domain and the domain structure as demonstrated by Cane et al.[37], but the majority of TDF domains have now been shown to be largely discriminately measurable.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Discussion
What the sources say
- Not checked yetcites Cane et al.[37] Cane J, O’Connor D, Michie S: Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012, 7: 37-10.1186/1748-5908-7-37. source
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The items of the questionnaire were listed below the definitions, in a random order.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Methods > Procedure
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The framework includes 12 theoretical domains of potential behavioral determinants and provides exemplar questions for the theoretical assessment of implementation problems.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Background
What the sources say
- Not checked yetcites [31] Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A: Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005, 14: 26-33. 10.1136/qshc.2004.011155. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The false discovery rate controlling procedure [57] was used to correct for multiple tests.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Methods > DCV analysis
What the sources say
- Not checked yetcites [57] Benjamini Y, Hochberg Y: Controlling the false discovery rate: a practical and powerful approach to multiple testing. J Roy Stat Soc B Met. 1995, 57: 289-300. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“It extends the original TDF to include the following 14 domains: Knowledge; Skills; Social/professional role and identity; Beliefs about capabilities; Optimism; Beliefs about consequences; Reinforcement; Intentions; Goals; Memory, attention and decision processes; Environmental context and resources; Social influences; Emotions; and Behavioral regulation.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Background
What the sources say
- Not checked yetcites [37] Cane J, O’Connor D, Michie S: Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012, 7: 37-10.1186/1748-5908-7-37. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In summary, the development of our questionnaire provides important evidence of content validity and is a first step towards the development of a valid and reliable questionnaire to measure TDF-based factors underlying healthcare professionals’ specific implementation behaviors.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“The framework has been used in a number of studies and was demonstrated to be useful for the development of qualitative [32,33] and quantitative [34-36] measurement tools to assess potential implementation behavior determinants.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Background
What the sources say
- Not checked yetcites [32] Patey AM, Islam R, Francis JJ, Bryson GL, Grimshaw JM: Anesthesiologists’ And surgeons’ perceptions about routine pre-operative testing in low-risk patients: application of the theoretical domains framework (TDF) to identify factors that influence physicians’ decisions to order pre-operative tests. Implement Sci. 2012, 7: 52-10.1186/1748-5908-7-52. source
- Not checked yetcites [33] McSherry LA, Dombrowski SU, Francis JJ, Murphy J, Martin CM, O’Leary JJ, Sharp L, ATHENS Group: 'It’s a can of worms’: understanding primary care practitioners’ behaviours in relation to HPV using the theoretical domains framework. Implement Sci. 2012, 7: 73-10.1186/1748-5908-7-73. source
- Not checked yetcites [34] Amemori M, Michie S, Korhonen T, Murtomaa H, Kinnunen TH: Assessing implementation difficulties in tobacco use prevention and cessation counselling among dental providers. Implement Sci. 2011, 6: 50-10.1186/1748-5908-6-50. source
- Not checked yetcites [35] Beenstock J, Sniehotta FF, White M, Bell R, Milne EM, Araujo-Soares V: What helps and hinders midwives in engaging with pregnant women about stopping smoking? A cross-sectional survey of perceived implementation difficulties among midwives in the northeast of England. Implement Sci. 2012, 7: 36-10.1186/1748-5908-7-36. source
- Not checked yetcites [36] Taylor N, Parveen S, Robins V, Slater B, Lawton R: Development and initial validation of the Influences on Patient Safety Behaviours Questionnaire. Implement Sci. 2013, 8: 81-10.1186/1748-5908-8-81. source
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“While Taylor et al.[36,63] used specific items (i.e., related to a specific application), our items are generic and allow for application within a range of different contexts in which implementation research takes place.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Discussion
What the sources say
- Not checked yetcites Taylor et al.[36,63]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“WAG and MRC supervised the development of the questionnaire and reviewed items’ face validity.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Methods > Materials
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“the domain Reinforcement is a refinement of the Beliefs about consequences domain and was originally included within the latter domain in the original TDF [31]”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Discussion
What the sources say
- Not checked yetcites [31] Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A: Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005, 14: 26-33. 10.1136/qshc.2004.011155. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“items measuring the construct Affect were based on the Positive and Negative Affect Schedule [49] and Stress items were based on the General Health Questionnaire [48]”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Discussion
What the sources say
- Not checked yetcites [49] Thompson ER: Development and Validation of an Internationally Reliable Short-Form of the Positive and Negative Affect Schedule (PANAS). J Cross Cult Psychol. 2007, 38: 227-242. 10.1177/0022022106297301. source
- Not checked yetcites [48] Goldberg DP, Blackwell B: Psychiatric illness in general practice: a detailed study using a new method of case identification. Brit Med J. 1970, 1: 439-443. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“the TDF domains are potential behavioral determinants, instead of factors proven to influence implementation behavior”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Discussion > Strengths and limitations of assessing TDF domains using questionnaires
What the sources say
- Not checked yetcites [30] Francis JJ, Stockton C, Eccles MP, Johnston M, Cuthbertson BH, Grimshaw JM, Hyde C, Tinmouth A, Stanworth SJ: Evidence-based selection of theories for designing behaviour change interventions: using methods based on theoretical construct domains to understand clinicians’ blood transfusion behaviour. Brit J Health Psychol. 2009, 14: 625-646. 10.1348/135910708X397025. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“at least three items with a loading above .80 will give a reliable component [64]”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Discussion
What the sources say
- Not checked yetcites [64] Stevens J: Applied Multivariate Statistics for the Social Sciences. 1992, Hillsdale: Lawrence Erlbaum
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“items assessing emotions in relation to performing a specific behavior should also be taken into account as these have been found to be linked to implementation behaviors in previous research [65-67]”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Discussion
What the sources say
- Not checked yetcites [65] Taylor N, Lawton R, Slater B, Foy R: The demonstration of a theory-based approach to the design of localized patient safety interventions. Implement Sci. 2013, 8: 123-10.1186/1748-5908-8-123. source
- Not checked yetcites [66] Dyson J, Lawton R, Jackson C, Cheater F: Development of a theory-based instrument to identify barriers and levers to best hand hygiene practice among healthcare practitioners. Implement Sci. 2013, 8: 111-10.1186/1748-5908-8-111. source
- Not checked yetcites [67] Dyson J, Lawton R, Jackson C, Cheater F: Does the use of a theoretical approach tell us more about hand hygiene behavior? The barriers and levers to hand hygiene. J Infect Prev. 2011, 12: 17-24. 10.1177/1757177410384300. source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“Major strengths of this study include the sample of academics with expertise on TDF domains and the formulation of items using the 'TACT principle’ [38]”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Discussion > Strengths and limitations of our methods
What the sources say
- Not checked yetcites [38] Fishbein M: Attitude and the Prediction of Behavior. Readings in Atttiude Theory and Measurement. Edited by: Fishbein M. 1967, New York: Wiley, 477-492.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“the operationalization and validation of the domains of the TDF are limited to these specific methods”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Discussion > Strengths and limitations of our methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“research on the development of a generic TDF questionnaire will improve our understanding of factors influencing healthcare professionals’ implementation and advance theory and methods in implementation research.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Conclusions
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
In the paper
“the framework does not specify relationships between domains [30]”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Discussion > Strengths and limitations of assessing TDF domains using questionnaires
What the sources say
- Not checked yetcites [30] Francis JJ, Stockton C, Eccles MP, Johnston M, Cuthbertson BH, Grimshaw JM, Hyde C, Tinmouth A, Stanworth SJ: Evidence-based selection of theories for designing behaviour change interventions: using methods based on theoretical construct domains to understand clinicians’ blood transfusion behaviour. Brit J Health Psychol. 2009, 14: 625-646. 10.1348/135910708X397025. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“this study is the first to develop a generic (i.e., formulation of items following the 'TACT principle’ [38]) TDF-based questionnaire in both English and Dutch including items which are able to discriminately measure a majority of the domains”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Conclusions
What the sources say
- Not checked yetcites [38] Fishbein M: Attitude and the Prediction of Behavior. Readings in Atttiude Theory and Measurement. Edited by: Fishbein M. 1967, New York: Wiley, 477-492.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“a valid TDF-based questionnaire will increase the use of theory in the assessment of barriers and facilitators for implementation problems [31,69,70], which can inform the selection of possible techniques that can be used to change healthcare professionals’ behaviors [15,23,26]”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Conclusions
What the sources say
- Not checked yetcites [31] Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A: Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005, 14: 26-33. 10.1136/qshc.2004.011155. source
- Not checked yetcites [69] Davies P, Walker AE, Grimshaw JM: A systematic review of the use of theory in the design of guideline dissemination and implementation strategies and interpretation of the results of rigorous evaluations. Implement Sci. 2010, 5: 14-10.1186/1748-5908-5-14. source
- Not checked yetcites [70] Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, Whitty P, Eccles MP, Matowe L, Shirran L, Wensing M, Dijkstra R, Donaldson C: Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004, 8: 1-72. source
- Not checked yetcites [15] Eccles M, Grimshaw J, Walker A, Johnston M, Pitts N: Changing the behavior of healthcare professionals: the use of theory in promoting the uptake of research findings. J Clin Epidemiol. 2005, 58: 107-112. 10.1016/j.jclinepi.2004.09.002. source
- Not checked yetcites [23] Michie S, Johnston M, Francis J, Hardeman W, Eccles M: From theory to intervention: mapping theoretically derived behavioural determinants to behaviour change techniques. Appl Psychol. 2008, 57: 660-680. 10.1111/j.1464-0597.2008.00341.x. source
- Not checked yetcites [26] French SD, Green SE, O’Connor DA, McKenzie JE, Francis JJ, Michie S, Buchbinder R, Schattner P, Spike N, Grimshaw JM: Developing theory-informed behaviour change interventions to implement evidence into practice: a systematic approach using the Theoretical Domains Framework. Implement Sci. 2012, 7: 38-10.1186/1748-5908-7-38. source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“this study represents an important first step in the thorough development of a questionnaire to measure TDF-based factors underlying healthcare professionals’ implementation behaviors”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Discussion > Strengths and limitations of our methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This approach is a degree of magnitude more challenging than how DCVs have typically been applied in the past (to a much smaller number of constructs).”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Discussion > Strengths and limitations of our methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“The results partly support Cane et al.’s validation of the TDF [37] and suggest that the 12-domain version [31] might be more applicable in developing a TDF-based questionnaire.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Conclusions
What the sources say
- Not checked yetcites [37] Cane J, O’Connor D, Michie S: Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012, 7: 37-10.1186/1748-5908-7-37. source
- Not checked yetcites [31] Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A: Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005, 14: 26-33. 10.1136/qshc.2004.011155. source
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The DCV exercise of allocating 79 items to 14 domains was a challenging task for judges, requiring consideration of multiple possible definitions.”
§ Discriminant content validity of a theoretical domains framework questionnaire for use in implementation research > Discussion > Strengths and limitations of our methods
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper relies on this without support.
The content of the TDF had not yet been validated at item level.
“Although the framework is suggested to be useful for the development of theory-based questionnaires for use in implementation research, the content of the TDF has not yet been validated on item level.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“There are three published validated questionnaire measures of the TDF to identify influences on the following behaviours: health care professionals' patient safety behaviours [48], physical activity in the general population [20] and generic health professional behaviours [49].”
From A guide to using the Theoretical Domains Framework of behaviour change to investigate implementation problems (2017) · cites this paper as [49]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Huijg and colleagues developed a TDF-based questionnaire [118] that could be tailored to study these under-explored barriers and assess their importance.”
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,893
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
The theory of planned behavior
Ajzen, I. (1991) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(91)90020-t · Cited by 90,587
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
Rogers, E. M., Singhal, A., & Quinlan, M. M. (2019) · Routledge eBooks · doi:10.4324/9780203710753-35 · Cited by 37,670
Advances In Experimental Social Psychology
(2022) · Advances in experimental social psychology · doi:10.1016/s0065-2601(22)x0002-6 · Cited by 22,559
Developing and evaluating complex interventions: the new Medical Research Council guidance
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008) · BMJ · doi:10.1136/bmj.a1655 · Cited by 10,995
Applied Multivariate Statistics for the Social Sciences
Stevens, J. P. (2012) · Routledge eBooks · doi:10.4324/9780203843130 · Cited by 10,652
Scheier, M. F., Carver, C. S., & Bridges, M. W. (1994) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.67.6.1063 · Cited by 7,456
Diffusion of Innovations in Service Organizations: Systematic Review and Recommendations
GREENHALGH, T., ROBERT, G., MACFARLANE, F., BATE, P., & KYRIAKIDOU, O. (2004) · The Milbank Quarterly · doi:10.1111/j.0887-378x.2004.00325.x · Cited by 7,448
Research works printed but not linked: 5
- Eakin EG, Glasgow RE, Riley KM: Review of primary care-based physical activity intervention studies. J Fam Practice. 2000, 49: 158-168.
- Huijg JM, van der Zouwe N, Gebhardt WA, Crone MR, Verheijden MW, Middelkoop BJC: Introducing physical activity interventions in primary health care: a qualitative study of perceived facilitators and barriers [abstract]. Int J Behav Med. 2012, 19: S1-S341.
- Maes S, Karoly P, de Gucht V, Ruehlman LS, Heiser W: The self regulation skills battery. 2006, Leiden/Phoenix: Leiden University/Arizona State University
- IBM Corp: IBM SPSS statistics for windows, version 19.0. 2010, Armonk, NY: IBM Corp
- Falissard B: Psy: Various Procedures Used in Psychometry. 2012, : R Package Version 1.1
Cited by
Sorted by citations
Atkins, L., Francis, J., Islam, R., O’Connor, D., Patey, A., Ivers, N., Foy, R., Duncan, E. M., Colquhoun, H., Grimshaw, J. M., et al. (2017) · Implementation Science · doi:10.1186/s13012-017-0605-9 · Cited by 3,819
“Published TDF questionnaires may be helpful in developing interview schedules [20, 48, 49].”
BackgroundInfluentialPsychometric assessment of three newly developed implementation outcome measures
Weiner, B. J., Lewis, C. C., Stanick, C., Powell, B. J., Dorsey, C. N., Clary, A. S., Boynton, M. H., & Halko, H. (2017) · Implementation Science · doi:10.1186/s13012-017-0635-3 · Cited by 2,819
Action, actor, context, target, time (AACTT): a framework for specifying behaviour
Presseau, J., McCleary, N., Lorencatto, F., Patey, A. M., Grimshaw, J. M., & Francis, J. J. (2019) · Implementation Science · doi:10.1186/s13012-019-0951-x · Cited by 411
Self-citationBirken, S. A., Powell, B. J., Presseau, J., Kirk, M. A., Lorencatto, F., Gould, N. J., Shea, C. M., Weiner, B. J., Francis, J., Yu, Y., et al. (2017) · Implementation Science · doi:10.1186/s13012-016-0534-z · Cited by 387
Self-citationHuijg, J. M., Gebhardt, W. A., Dusseldorp, E., Verheijden, M. W., van der Zouwe, N., Middelkoop, B. J., & Crone, M. R. (2014) · Implementation Science · doi:10.1186/1748-5908-9-33 · Cited by 378
Self-citation
Metadata
- Authors
- Huijg, J. M., Gebhardt, W. A., Crone, M. R., Dusseldorp, E., & Presseau, J.
- Affiliations
- Leiden University (NL)
- Leiden University Medical Center (NL)
- Netherlands Organisation for Applied Scientific Research (NL)
- Newcastle University (GB)
- Corresponding author
- Huijg, Johanna M
- Venue
- Implementation Science
- Published
- 2014-01-15
- Volume
- 9
- Issue
- 1
- Pages
- 11-11
- ISSN
- 1748-5908 (ISSN-L)
- PMC
- PMC3896680
- OpenAlex
- W2130770431
- Semantic Scholar
- d4ff79d66f85c061e8af489063ba90d48fbc076a
- Cited by
- 385
- References
- 78
- Citations by year
- 2026: 27 · 2025: 34 · 2024: 44 · 2023: 47 · 2022: 65 · 2021: 59 · 2020: 32 · 2019: 22 · 2018: 15 · 2017: 15 · 2016: 12 · 2015: 9 · 2014: 3
- FWCI
- 23.48
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Implementation Science (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Leiden Repository (Leiden University) (submittedVersion)
- TNO Repository (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Newcastle University ePrints (Newcastle Univesity) (cc-by)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Nursing › Research and Theory › Health Education and Validation
- Social Sciences › Psychology › Applied Psychology › Behavioral Health and Interventions
- Keywords
- Theoretical Domains Framework, questionnaire development, behavioral factors, healthcare workers, reinforcement, goals, behavioral regulation, Dutch translation, English translation
- MeSH terms
- Attitude of Health Personnel, Diffusion of Innovation, Environment, Health Knowledge, Attitudes, Practice, Language, Models, Theoretical, Motivation, Surveys and Questionnaires, Reproducibility of Results, Evidence-Based Medicine, Self Efficacy, Professional Role, Translational Research, Biomedical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
