How to … assess the quality of qualitative research

Classified as probably not a substantial claim.

Disclosing information about the researcher's role in the methodology section increases the transparency and trustworthiness of the research findings.

“Disclosing such information in the methodology section is important and increases the transparency and trustworthiness of the research findings.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The concept of saturation is widely debated, and information power might be a better concept.

“This concept is widely debated, however, and Varpio et al. provides a great description of why information power might be a better concept.6”

• cites [6] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

SRQR was developed especially for medical education, whereas Tracy and CASP present criteria for qualitative research applicable across genres and methodologies.

“Another checklist, developed especially for medical education, is Standards for Reporting Qualitative Research (SRQR), whereas Tracy and Critical Appraisal Skills Programme (CASP) present criteria for qualitative research applicable across genres and methodologies.[10][11][12]”

• cites [10] sourceNot checked yet

  Not checked in the free run.

• cites [11] sourceNot checked yet

  Not checked in the free run.

• cites [12]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

A less rich data sample (for example very short interviews) is likely to be of less value to the research than fewer in-depth interviews that provide rich insight.

“a less rich data sample (for example very short interviews) is likely to be of less value to the research than fewer in-depth interviews that provide rich insight.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Saturation or data sufficiency are commonly used terms to indicate credibility, suggesting that enough data were gathered to identify all relevant aspects needed to answer the research question.

“Similarly, saturation or data sufficiency are commonly used terms to indicate credibility, suggesting that enough data were gathered to identify all relevant aspects to answer the research question.”

Probably not a substantial claim: may not pass S2 (checkable).

A high score on a quality checklist may say little about whether the findings can be applied in a given setting, because authors' theoretical and philosophical frameworks and journal standards vary immensely.

“We would caution, however, that a high score on such a list may say little about whether the findings can be applied in your setting, bearing in mind that the authors' theoretical and philosophical frameworks and the standards of journals vary immensely.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Quantifying qualitative findings by expressions such as percentages may not strengthen the research.

“however, Monrouxe and Rees provide a compelling argument for why this may not be strengthening the research.8”

• cites [8] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

In qualitative research, the person collecting the data plays a central role.

“In qualitative research, the person collecting the data plays a central role (compare the roles of a researcher conducting a personal interview versus a researcher collecting data through an online questionnaire).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

COREQ is perhaps the most commonly used framework today, and some journals now require authors to use it when submitting an article.

“The perhaps most commonly used framework today is Consolidated Criteria for Reporting Qualitative Research (COREQ), and some journals now require authors to use this when submitting an article.9”

• cites [9] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

Whether the interviewer is an expert on the topic is likely to affect the interview, as are factors such as the interviewees' and respondents' respective ages, gender and professions.

“Whether the interviewer is an expert on the topic is also likely to affect the interview, as well as other factors such as the interviewees' and the respondents' respective ages, gender and professions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The concepts of triangulation or crystallisation are used as quality markers in somewhat different ways depending on the paradigm.

“Varpio et al. further explain how the concepts of triangulation or crystallisation are used as a quality marker in somewhat different ways, depending on the paradigm.6”

• cites [6] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Reflexivity - the researcher reflecting on his or her role in the study - is a key marker of quality.

“This process, reflexivity, is a key marker of quality.7”

• cites [7] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

A teacher interviewing his or her students is likely to get different answers than a student asking peers the same questions.

“A teacher interviewing his or her students is likely to get different answers than a student asking peers the same questions.”

Probably not a substantial claim: may not pass S4 (based).

The quality standards looked for in quantitative research do not apply to qualitative research.

“The quality standards you would look for in quantitative research do not apply.”

• AssumptionRests on an assumption: Qualitative research is conducted within a number of paradigms, each associated with different ways of defining, understanding and reporting quality.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

A detailed description of the research process - including context, aims, questions and design, theoretical underpinnings, methods of data collection and analysis, results, discussion, and their careful alignment - usually increases the quality of a qualitative study.

“In sum, a detailed description of the research process, including the research context, research aims, questions and design, theoretical underpinnings, and the methods of data collection and data analysis, results, discussion, and their careful alignment, usually increases the quality of a qualitative study.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

In the write-up of the findings, confirmability is enhanced by the inclusion of quotes or similar research data.

“In the write-up of the findings, confirmability is enhanced by the inclusion of quotes or similar research data.”

Probably not a substantial claim: may not pass S4 (based).

Describing the process of data collection and data analysis in enough detail for someone to follow the same steps supports the dependability of the research.

“Most importantly, the process of data collection and data analysis should be described in enough detail for someone to follow the same steps.This supports the dependability of the research.”

May combine two claims.

It is very difficult to know beforehand how many respondents will be needed in qualitative studies, and there is flexibility regarding this point.

“Taken together, this makes it very difficult to know beforehand how many respondents will be needed, and in qualitative studies there is flexibility regarding this point.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

Qualitative findings may inform theory and methodology development, but they may not be directly transferrable to other social or organisational settings.

“The findings may inform theory and methodology development, but they may not be directly transferrable to other social or organisational settings.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Morse proposes that the following factors be taken into consideration when judging data adequacy: the scope of the study; the nature of the topic; what is already known about the studied phenomenon; the study design; the quality of the data; the amount of useful information obtained from each participant; and the use of shadowed data.

“Morse proposes the following factors to be taken into consideration: the scope of the study; the nature of the topic; what is already known about the studied phenomenon; the study design; the quality of the data; the amount of useful information”

• cites [5] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In some studies, the credibility of the analysis can be enhanced through member reflections, in which preliminary findings are presented to participants for input and elaboration.

“In some studies, the credibility of the analysis can be enhanced through member reflections, which means that the preliminary findings are presented to the participants for input and elaboration.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

It may not be possible to analyse a huge volume of data in the appropriate depth required for certain methodologies.

“Moreover, it may not be possible to analyse a huge volume of data in the appropriate depth required for certain methodologies”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

The markers of quality differ between qualitative research and quantitatively oriented natural sciences, and variation also exists between different qualitative methodologies.

“It is important to note that these markers differ between qualitative research and quantitatively oriented natural sciences, albeit that variation also exists between different qualitative methodologies.”

• AssumptionRests on an assumption: Qualitative research is conducted within a number of paradigms, each associated with different ways of defining, understanding and reporting quality.