Cited by 1,238Health Policy Implementation ScienceInnovations in Medical EducationQualitative Research Methods and Applications

A Review of the Quality Indicators of Rigor in Qualitative Research

Open access
Johnson, Jessica L., Adkins, Donna, Chauvin, Sheila·2020·American Journal of Pharmaceutical Education

Abstract

Attributes of rigor and quality and suggested best practices for qualitative research design as they relate to the steps of designing, conducting, and reporting qualitative research in health professions educational scholarship are presented. A research question must be clear and focused and supported by a strong conceptual framework, both of which contribute to the selection of appropriate research methods that enhance trustworthiness and minimize researcher bias inherent in qualitative methodologies. Qualitative data collection and analyses are often modified through an iterative approach to answering the research question. Researcher reflexivity, essentially a researcher’s insight into their own biases and rationale for decision-making as the study progresses, is critical to rigor. This article reviews common standards of rigor, quality scholarship criteria, and best practices for qualitative research from design through dissemination.

Integrity signals

Registry

Crossref · 10.5688/ajpe7120

Cites

45 of 45 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

10 of 66 cited sources checked · 7 supported · 0 flagged · accuracy not yet measured

  • Premisecitation

    If the research participants or topic are too familiar or personal, this may influence data collection or analysis and interpretation of the results.

    “However, if the research participants or topic are too familiar or personal, this may also influence data collection or analysis and interpretation of the results.33”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [33] 33. Patton MQ. Enhancing the quality and credibility of qualitative analysis. Health Serv Res. 1999; 34(5 Pt 2):1189-1208.
  • Premisecitation

    Any researcher biases not adequately addressed or errors in judgement can affect the quality of data and subsequent research results.

    “Any researcher biases not adequately addressed or errors in judgement can affect the quality of data and subsequent research results.33”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [33] 33. Patton MQ. Enhancing the quality and credibility of qualitative analysis. Health Serv Res. 1999; 34(5 Pt 2):1189-1208.
  • Premisecitation

    The confirmability of the results is influenced by reducing or at a minimum explaining any researcher influence on the result by applying and meeting standards of rigor such as member checking, triangulation, and peer review.

    “The confirmability of the results is influenced by reducing or at a minimum explaining any researcher influence on the result by applying and meeting standards of rigor such as member checking, triangulation, and peer review.29,33”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [29] 29. Marshall MN. Sampling for qualitative research. Fam Pract. 1996;13(6):522-525. source
    • cites [33] 33. Patton MQ. Enhancing the quality and credibility of qualitative analysis. Health Serv Res. 1999; 34(5 Pt 2):1189-1208.
  • Premisecitation

    Researcher characteristics such as facial expressions, spoken language, body language, style of dress, age, race, gender, social status, culture, and relationship with participants may influence either participants' responses or how the researcher interprets those responses.

    “Facial expressions, spoken language, body language, style of dress, age, race, gender, social status, culture, and the researcher's relationship with the participants may potentially influence either participants' responses or how the researcher interprets those responses.33”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [33] 33. Patton MQ. Enhancing the quality and credibility of qualitative analysis. Health Serv Res. 1999; 34(5 Pt 2):1189-1208.

      Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured
  • Premisecitation

    If a theme or theory can be arrived at and validated using multiple sources of data, the result of the study has greater credibility and confirmability.

    “If a theme or theory can be arrived at and validated using multiple sources of data, the result of the study has greater credibility and confirmability.16,33,36”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites [16] 16. Creswell J. Qualitative, quantitative, and mixed methods approaches. Res Des. 2013:1-26.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [33] 33. Patton MQ. Enhancing the quality and credibility of qualitative analysis. Health Serv Res. 1999; 34(5 Pt 2):1189-1208.
    • cites [36] 36. Johnson RB. Examining the validity structure of qualitative research. Education. 1997;118(2):282.
  • Classified as probably not a substantial claim. 44
  • May combine two claims. 12
  • Classified as probably not a substantial claim.

    Premisecitation

    Some authors have raised concerns that strict adherence to universal checklists may stifle the diversity and multiplicity of practices in qualitative research.

    “Some authors have raised concerns about the use of such checklists and adherence to strict, universal criteria because they do not afford sufficient flexibility to accommodate the diverse approaches and multiple interpretive practices often represented in qualitative studies.[7][8][9][10][11]”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites [7] 7. Dixon-Woods M, Shaw RL, Agarwal S, Smith JA. The problem of appraising qualitative research. BMJ Quality & Safety. 2004;13:223-225 source
    • cites [8] 8. Stige B, Malterud K, Midtgarden T. Toward an agenda for evaluation of qualitative research. Qual Health Res. 2009;19(10):1504-16. source
    • cites [9] 9. Wasserman JA, Clair JM, Wilson KL. Problematics of grounded theory: Innovations for developing an increasingly rigorous qualitative method. Qual Res. 2009;9(3):355-381. source
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In recent years, multiple publications have synthesized quality criteria and recommendations for use by researchers and peer reviewers, often in the form of checklists.

    “In recent years, multiple publications have synthesized quality criteria and recommendations for use by researchers and peer reviewers alike, often in the form of checklists.[2][3][4][5][6]”
    Quote in the paperNot a substantial claim1 of 3 sources checked
    • cites [2] 2. Anderson C. Presenting and evaluating qualitative research. Am J Pharm Educ. 2010; 74(8):Article 141. source
    • cites [3] 3. Santiago-Delefosse M, Gavin A, Bruchez C, Roux P, Stephen SL. Quality of qualitative research in the health sciences: analysis of the common criteria present in 58 assessment guidelines by expert users. Soc Sci Med. 2016;148:142-51. source
    • cites [4] 4. O'Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med. 2014;89(9):1245-1251. source
  • Classified as probably not a substantial claim.

    Interpretationcitation

    Developing a FINER research question is critical to study rigor and quality.

    “Developing a FINER research question is critical to study rigor and quality and should not be rushed, as all other aspects of research design depend on the focus and clarity of the research question(s) guiding the study.15”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [15] 15. Agee J. Developing qualitative research questions: A reflective process. Int J Qual Stud Educ. 2009;22(4):431-447. source

      The Abstract says poorly constructed questions can affect all subsequent stages of a study, and the Conclusion says good qualitative questions shape study design and analysis. The paper supports careful development of research questions, but these passages do not establish the specific FINER framework named in the claim. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationcitation

    Developing and refining a primary research question focused on both the phenomena of interest and the context in which it is situated is essential to research rigor and quality.

    “Developing and refining a primary research question focused on both the phenomena of interest and the context in which it is situated is essential to research rigor and quality.”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites [15] 15. Agee J. Developing qualitative research questions: A reflective process. Int J Qual Stud Educ. 2009;22(4):431-447. source
    • cites [16] 16. Creswell J. Qualitative, quantitative, and mixed methods approaches. Res Des. 2013:1-26.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    The goal of rigor in qualitative research can be described as ensuring that the research design, method, and conclusions are explicit, public, replicable, open to critique, and free of bias.

    “The goal of rigor in qualitative research can be described as ensuring that the research design, method, and conclusions are explicit, public, replicable, open to critique, and free of bias.41”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [41] 41. Leung L. Validity, reliability, and generalizability in qualitative research. J Fam Med Prim Care. 2015;4(3):324. source

      The 'Overall Criteria for Quality in Qualitative Research' section supports making the design, methods, and conclusions transparent and systematic; the paper also discusses appraisal of their quality. But the 'Reliability' section says exact replicability is counter-intuitive for qualitative research, and 'Nature of Qualitative Research versus Quantitative Research' treats subjectivity as an essential element rather than something to eliminate as bias. Page numbers are unavailable in our copy. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Inclusion of best practice methods for assuring the rigor and trustworthiness of results is critical to study design.

    “Although researchers can employ great flexibility in the selection of study methods, inclusion of best practice methods for assuring the rigor and trustworthiness of results is critical to study design.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    Given the unique and often highly sensitive nature of data being collected by the researcher, trustworthiness is an essential component of the researcher-participant relationship.

    “Given the unique and often highly sensitive nature of data being collected by the researcher, trustworthiness is an essential component of the researcher-participant relationship.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The conceptual framework contributes directly to determining sample definitions, size, and recruitment of participants.

    “The conceptual framework contributes directly to determining sample definitions, size, and recruitment of participants.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Data saturation is probably the most common terminology used to describe the achievement of sufficient sample size.
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    Other considerations related to sampling bias may impact the quantity and quality of data generated and therefore the quality of the study result.

    “In addition to defining the sample and selecting participants, other considerations related to sampling bias may impact the quantity and quality of data generated and therefore the quality of the study result.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Given the unique and often highly sensitive nature of data being collected by the researcher, trustworthiness is an essential component of the researcher-participant relationship.
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Ringsted, Hodges, and Sherpbier describe three essential parts to an effective conceptual framework: theories and/or concepts and principles relevant to the phenomenon of interest; what is known and unknown from prior work, observations, and examples; and the researcher's observations, ideas, and suppositions regarding the research problem statement and question.

    “Ringsted, Hodges, and Sherpbier describe three essential parts to an effective conceptual framework: theories and/ or concepts and principles relevant to the phenomenon of interest; what is known and unknown from prior work, observations, and examples; and the researcher's observations, ideas, and suppositions regarding the research problem statement and question.21”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [21] 21. Ringsted C, Hodges B, Scherpbier A. "The research compass": an introduction to research in medical education: AMEE Guide No. 56. Med Teach. 2011;33(9):695-709. source
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Glassick, Huber, and Maeroff identified six criteria applicable to assessing the quality of scholarship.

    “Glassick, Huber, and Maeroff identified six criteria applicable to assessing the quality of scholarship.18,19”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [18] 18. Glassick CE, Huber MT, Maeroff GI, Boyer EL. Scholarship Assessed: Evaluation of the Professoriate. San Francisco, CA: Jossey-Bass; 1997.
    • cites [19] 19. Glassick CE. Boyer's expanded definitions of scholarship, the standards for assessing scholarship, and the elusiveness of the scholarship of teaching. Acad Med. 2000;75(9):877-880. source
  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    By being transparent in the process and reporting of results when saturation may not have been possible, the resulting data may still contribute to the field and to further inquiry.

    “By being transparent in the process and reporting of results when saturation may not have been possible, the resulting data may still contribute to the field and to further inquiry.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Any of these factors could potentially change the nature of the relationship between the researcher and the study participants and influence the trustworthiness of data collected or the study result.
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The 2015 second edition of the Review Criteria for Research Manuscripts and work by Ravitch and colleagues provide specific guidance for applying the conceptual framework to each stage of the research process to enhance rigor and quality.

    “The 2015 second edition of the Review Criteria for Research Manuscripts 14 and work by Ravitch and colleagues 23,24 provide specific guidance for applying the conceptual framework to each stage of the research process to enhance rigor and quality.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [14] 14. Durning SJ, Carline J, eds. Review criteria for research manuscripts, 2 nd ed. Washington, DC: Association of American Medical Colleges; 2015.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [23] 23. Ravitch S, Carl N. Qualitative Research: Bridging the Conceptual, Theoretical, and Methodological. Thousand Oaks, CA: Sage; 2016.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [24] 24. Ravitch SM, Riggan M. Reason & Rigor: How Conceptual Frameworks Guide Research. Los Angeles, CA: Sage; 2012. source
  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    Any of these factors could potentially change the nature of the relationship between the researcher and the study participants and influence the trustworthiness of data collected or the study result.

    “Any of these factors could potentially change the nature of the relationship between the researcher and the study participants and influence the trustworthiness of data collected or the study result.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Interview protocols and structure are often modified or refined, based on concurrent data collection and analysis processes to support or refute preliminary interpretations and refine focus and continuing inquiry.
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationnone given

    Interview methods range from highly structured to increase dependability or completely open-ended to allow for interviewers to clarify a participant's response for increased credibility and confirmability.

    “Interview methods range from highly structured to increase dependability or completely open-ended to allow for interviewers to clarify a participant's response for increased credibility and confirmability.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Member checking, a common standard of rigor, is a practice to increase study credibility and confirmability that involves asking a research subject to verify the transcription of an interview.

    “Member checking, a common standard of rigor, is a practice to increase study credibility and confirmability that involves asking a research subject to verify the transcription of an interview.1,16,28”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [1] 1. Denzin, Norman, Lincoln YS. The Sage Handbook of Qualitative Research. 4th ed. Los Angeles, CA: Sage Publications; 2011.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [16] 16. Creswell J. Qualitative, quantitative, and mixed methods approaches. Res Des. 2013:1-26.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [28] 28. Whittemore R, Chase SK, Mandle CL. Validity in qualitative research. Qual Health Res. 2001;11(4):522-537. source
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    Data saturation is probably the most common terminology used to describe the achievement of sufficient sample size, but it does not apply to all study designs.

    “Though data saturation is probably the most common terminology used to describe the achievement of sufficient sample size, it does not apply to all study designs.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: By being transparent in the process and reporting of results when saturation may not have been possible, the resulting data may still contribute to the field and to further inquiry.
  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Sampling design in qualitative research is not random but defined purposively to include the most appropriate participants in the most appropriate context for answering the research question.

    “Therefore, sampling design in qualitative research is not random but defined purposively to include the most appropriate participants in the most appropriate context for answering the research question.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Attrition and factors related to time and resources may contribute to not reaching data saturation within the limits of the study.
  • Classified as probably not a substantial claim.

    Premisecitation

    Lincoln and Guba outline four criteria for establishing the overall trustworthiness of qualitative research results: credibility, transferability, dependability, confirmability.

    “Lincoln and Guba outline four criteria for establishing the overall trustworthiness of qualitative research results: credibility, the researcher ensures and imparts to the reader supporting evidence that the results accurately represent what was studied; transferability, the researcher provides detailed contextual information such that readers can determine whether the results are applicable to”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [27] 27. Guba E, Lincoln Y. Sage Handbook of Qualitative Research. Thousand Oaks, CA: Sage; 1994.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Lingard describes four types of unknowns to pursue during literature review: what no one knows; what is not yet well understood; what controversy or conflicting results, understandings, or perspectives exist; and what are unproven assumptions.

    “Lingard describes four types of unknowns to pursue during literature review: what no one knows; what is not yet well understood; what controversy or conflicting results, understandings, or perspectives exist; and what are unproven assumptions.22”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [22] 22. Lingard L. Writing an effective literature review. Perspect Med Educ. 2018;7(1):47-49.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    Interview protocols and structure are often modified or refined, based on concurrent data collection and analysis processes to support or refute preliminary interpretations and refine focus and continuing inquiry.

    “Regardless, interview protocols and structure are often modified or refined, based on concurrent data collection and analysis processes to support or refute preliminary interpretations and refine focus and continuing inquiry.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Attrition and factors related to time and resources may contribute to not reaching data saturation within the limits of the study.

    “In other instances, attrition and factors related to time and resources may contribute to not reaching data saturation within the limits of the study.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    Developing a well-integrated conceptual framework is essential to establishing a research topic based upon a thorough and integrated review of relevant literature.

    “Developing a well-integrated conceptual framework is essential to establishing a research topic based upon a thorough and integrated review of relevant literature (addressing Glassick criteria #1 and #2: clear purpose and adequate preparation).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    The number of participants available for inclusion in a sample may be small and some risk of not reaching data saturation may be unavoidable.

    “Conversely, the number of participants available for inclusion in a sample may be small and some risk of not reaching data saturation may be unavoidable.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Quality criteria for assessing a study's problem statement, conceptual framework, and research question include: introduction builds a logical case and provides context for the problem statement; problem statement is clear and well-articulated; conceptual framework is explicit and justified; research purpose and/or question is clearly stated; and constructs being investigated are clearly presented

    “Quality criteria for assessing a study's problem statement, conceptual framework, and research question include the following: introduction builds a logical case and provides context for the problem statement; problem statement is clear and well-articulated; conceptual framework is explicit and justified; research purpose and/or question is clearly stated; and constructs being investigated are”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [14] 14. Durning SJ, Carline J, eds. Review criteria for research manuscripts, 2 nd ed. Washington, DC: Association of American Medical Colleges; 2015.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [24] 24. Ravitch SM, Riggan M. Reason & Rigor: How Conceptual Frameworks Guide Research. Los Angeles, CA: Sage; 2012. source
    • cites [25] 25. Miles MB, Huberman M. Qualitative Data Analysis: An Expanded Sourcebook, 2nd ed. Thousand Oaks, CA: Sage; 1994.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Recent contributions from Ravitch and colleagues present best practices in developing frameworks for conceptual and methodological coherence within a study design, regardless of the research approach.

    “Recent contributions from Ravitch and colleagues present best practices in developing frameworks for conceptual and methodological coherence within a study design, regardless of the research approach.23,24”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [23] 23. Ravitch S, Carl N. Qualitative Research: Bridging the Conceptual, Theoretical, and Methodological. Thousand Oaks, CA: Sage; 2016.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [24] 24. Ravitch SM, Riggan M. Reason & Rigor: How Conceptual Frameworks Guide Research. Los Angeles, CA: Sage; 2012. source
  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Interpretationcitation

    The development of a strong conceptual framework facilitates selection of appropriate study methods to minimize the bias inherent in qualitative studies and help readers to trust the research and the researcher.

    “The development of a strong conceptual framework facilitates selection of appropriate study methods to minimize the bias inherent in qualitative studies and help readers to trust the research and the researcher (see 23 Glassick criteria #3 in Table 1).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationcitation

    Convenience sampling, using the most accessible research participants, is the least rigorous approach to defining a study sample and may result in low accuracy, poor representativeness, low credibility, and lack of transferability of study results.

    “Convenience sampling, using the most accessible research participants, is the least rigorous approach to defining a study sample and may result in low accuracy, poor representativeness, low credibility, and lack of transferability of study results.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [29] 29. Marshall MN. Sampling for qualitative research. Fam Pract. 1996;13(6):522-525. source
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The first two Glassick criteria, clear purpose and adequate preparation, are directly related to formulating effective research questions and a strong conceptual framework.

    “The first two criteria, clear purpose and adequate preparation, are directly related to formulating effective research questions and a strong conceptual framework.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given

    Reflexivity, the idea that a researcher's preconceptions and biases can influence decisions and actions throughout qualitative research activities, is a critical aspect of rigor even at the earliest stages of the study.

    “Reflexivity, the idea that a researcher's preconceptions and biases can influence decisions and actions throughout qualitative research activities, is a critical aspect of rigor even at the earliest stages of the study.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Premisenone given

    As in any research paradigm, the goal of quality and rigor in qualitative research is to minimize the risk of bias and maximize the accuracy and credibility of research results.

    “As in any research paradigm, the goal of quality and rigor in qualitative research is to minimize the risk of bias and maximize the accuracy and credibility of research results.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The SRQR framework provides detailed explanations of what should be reported in each of 21 sections of a qualitative research manuscript.

    “This framework provides detailed explanations of what should be reported in each of 21 sections of a qualitative research manuscript.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [4] 4. O'Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med. 2014;89(9):1245-1251. source
  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Beginning the study with a well-developed conceptual framework and active use of both researcher reflexivity and rigorous peer review during study implementation can drive both study rigor and quality.

    “Beginning the study with a well-developed conceptual framework and active use of both researcher reflexivity and rigorous peer review during study implementation can drive both study rigor and quality.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    It is vital to the credibility and trustworthiness of the study that the author disclose and explore negative cases (competing or controversial theories) that emerge during data collection or analysis.

    “Should any competing or controversial theories emerge during data collection or analysis, it is vital to the credibility and trustworthiness of the study that the author disclose and explore those negative cases.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [16] 16. Creswell J. Qualitative, quantitative, and mixed methods approaches. Res Des. 2013:1-26.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Data triangulation is used to identify convergence of data obtained through multiple data sources and methods to avoid or minimize error or bias and optimize accuracy in data collection and analysis processes.

    “Data triangulation is used to identify convergence of data obtained through multiple data sources and methods (eg, observation field notes and interview transcripts) to avoid or minimize error or bias and optimize accuracy in data collection and analysis processes.33,35,36”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [33] 33. Patton MQ. Enhancing the quality and credibility of qualitative analysis. Health Serv Res. 1999; 34(5 Pt 2):1189-1208.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [35] 35. Creswell J, Miller D. Determining validity in qualitative inquiry. Theory Pract. 2000;39(3):124-130. source
    • cites [36] 36. Johnson RB. Examining the validity structure of qualitative research. Education. 1997;118(2):282.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning

    The peer reviewer offers a critique of the study methods and validation of the conclusions drawn by the author as a thorough check on researcher bias.

    “The peer reviewer offers a critique of the study methods and validation of the conclusions drawn by the author as a thorough check on researcher bias.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Premisecitation

    A study's conceptual framework is critical to achieving rigor and quality and provides guidance in developing predetermined notions or plans for what to observe, how to record, and how to minimize the influence of potential bias.

    “A study's conceptual framework is critical to achieving rigor and quality and provides guidance in developing predetermined notions or plans for what to observe, how to record, and how to minimize the influence of potential bias.34”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [34] 34. Mulhall A. In the field: Notes on observation in qualitative research. J Adv Nurs. 2003;41(3):306-313. source
  • Classified as probably not a substantial claim.

    Premisecitation

    To ensure dependability, the research method must be reported in detail such that the reader can determine proper research practices have been followed and that future researchers can repeat the study.

    “To ensure dependability, the research method must be reported in detail such that the reader can determine proper research practices have been followed and that future researchers can repeat the study.40”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [40] 40. Shenton AK. Strategies for ensuring trustworthiness in qualitative research projects. Educ Inf. 2004;22(2):63-75. source
  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Premisenone given

    The use of best practices applying to data collection and data analysis facilitates the full examination of data relative to the study purpose and research question and helps to prevent premature closure of the study.

    “The use of best practices applying to data collection and data analysis facilitates the full examination of data relative to the study purpose and research question and helps to prevent premature closure of the study.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    Rigor is best achieved through thoughtful and deliberate planning, diligent and ongoing application of researcher reflexivity, and honest communication between the researcher and the audience regarding the study and its results.

    “Rigor is best achieved through thoughtful and deliberate planning, diligent and ongoing application of researcher reflexivity, and honest communication between the researcher and the audience regarding the study and its results.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Premisecitation

    Regardless of the researcher's degree of visibility to study participants, persistent observation of the targeted sample is critical to the confirmability standard and to achieving data saturation.

    “Regardless of the researcher's degree of visibility to the study participants, persistent observation of the targeted sample is critical to the confirmability standard and to achieving data saturation.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [28] 28. Whittemore R, Chase SK, Mandle CL. Validity in qualitative research. Qual Health Res. 2001;11(4):522-537. source
  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    Study conclusions must be clearly grounded in persistent phenomena witnessed during the study, rather than on a fluke event.

    “That is, study conclusions must be clearly grounded in persistent phenomena witnessed during the study, rather than on a fluke event.28”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [28] 28. Whittemore R, Chase SK, Mandle CL. Validity in qualitative research. Qual Health Res. 2001;11(4):522-537. source
  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Due to the creative interpretative and contextually bound nature of qualitative studies, the application of standards of rigor and adherence to systematic processes well-documented in an audit trail are essential.

    “Thus, due to the creative interpretative and contextually bound nature of qualitative studies, the application of standards of rigor and adherence to systematic processes welldocumented in an audit trail are essential.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [33] 33. Patton MQ. Enhancing the quality and credibility of qualitative analysis. Health Serv Res. 1999; 34(5 Pt 2):1189-1208.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • Rests on an assumption: Qualitative studies are creative, interpretative, and contextually bound in nature.
  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Premisenone given

    Thick and rich descriptions, where robust descriptive language is used to provide sufficient contextual information, enable the reader to determine credibility, transferability, dependability, and confirmability.

    “Thick and rich descriptions, where robust descriptive language is used to provide sufficient contextual information, enable the reader to determine credibility, transferability, dependability, and confirmability.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisereasoning

    The process of demonstrating the credibility of research is rooted in honest and transparent reporting of how biases and other possible confounders were identified and addressed throughout study processes.

    “The process of demonstrating the credibility of research is rooted in honest and transparent reporting of how biases and other possible confounders were identified and addressed throughout study processes.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    Rigor in the research process and results are achieved when each element of study methodology is systematic and transparent through complete, methodical, and accurate reporting.

    “Rigor in the research process and results are achieved when each element of study methodology is systematic and transparent through complete, methodical, and accurate reporting.33”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [33] 33. Patton MQ. Enhancing the quality and credibility of qualitative analysis. Health Serv Res. 1999; 34(5 Pt 2):1189-1208.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

46 of 46
  • The paper relies on this without support.

    Qualitative research in health professions education has increased significantly, both in practice and publication, within the past 20 years.

    “Within the past 20 years, qualitative research in health professions education has increased significantly, both in practice and publication.”
  • The paper relies on this without support.

    There have been recurrent calls for enhancing rigor and quality in qualitative research.

    “Simultaneously, there have been recurrent calls for enhancing rigor and quality in qualitative research.”
  • The paper relies on this without support.

    Qualitative data collection and analyses are often modified through an iterative approach to answering the research question.

    “Qualitative data collection and analyses are often modified through an iterative approach to answering the research question.”
  • The paper relies on this without support.

    Qualitative studies typically reflect designs in which data collection and analysis are done concurrently, with results of ongoing analysis informing continuing data collection.

    “Qualitative studies typically reflect designs in which data collection and analysis are done concurrently, with results of ongoing analysis informing continuing data collection.”
  • The paper relies on this without support.

    Common qualitative data collection methods used in health professions education include interview, direct observation methods, and textual/document analysis.

    “Common qualitative data collection methods used in health professions education include interview, direct observation methods, and textual/document analysis.”
  • The paper relies on this without support.

    The conceptual framework contributes directly to determining sample definitions, size, and recruitment of participants.

    “The conceptual framework contributes directly to determining sample definitions, size, and recruitment of participants.”
  • The paper relies on this without support.

    Sampling design in qualitative research is not random but defined purposively to include the most appropriate participants in the most appropriate context for answering the research question.

    “Therefore, sampling design in qualitative research is not random but defined purposively to include the most appropriate participants in the most appropriate context for answering the research question.”
  • The paper relies on this without support.

    Data saturation is probably the most common terminology used to describe the achievement of sufficient sample size.

    “Though data saturation is probably the most common terminology used to describe the achievement of sufficient sample size, it does not apply to all study designs.”
  • The paper relies on this without support.

    The idea of fully achieving data saturation may be unrealistic when applied to some populations or research questions.

    “Thus, the idea of fully achieving data saturation may be unrealistic when applied to some populations or research questions.”
  • The paper relies on this without support.

    Attrition and factors related to time and resources may contribute to not reaching data saturation within the limits of the study.

    “In other instances, attrition and factors related to time and resources may contribute to not reaching data saturation within the limits of the study.”
  • The paper relies on this without support.

    By being transparent in the process and reporting of results when saturation may not have been possible, the resulting data may still contribute to the field and to further inquiry.

    “By being transparent in the process and reporting of results when saturation may not have been possible, the resulting data may still contribute to the field and to further inquiry.”
  • The paper relies on this without support.

    Other considerations related to sampling bias may impact the quantity and quality of data generated and therefore the quality of the study result.

    “In addition to defining the sample and selecting participants, other considerations related to sampling bias may impact the quantity and quality of data generated and therefore the quality of the study result.”
  • The paper relies on this without support.

    Any of these factors could potentially change the nature of the relationship between the researcher and the study participants and influence the trustworthiness of data collected or the study result.

    “Any of these factors could potentially change the nature of the relationship between the researcher and the study participants and influence the trustworthiness of data collected or the study result.”
  • The paper relies on this without support.

    Given the unique and often highly sensitive nature of data being collected by the researcher, trustworthiness is an essential component of the researcher-participant relationship.

    “Given the unique and often highly sensitive nature of data being collected by the researcher, trustworthiness is an essential component of the researcher-participant relationship.”
  • The paper relies on this without support.

    Interview methods range from highly structured to increase dependability or completely open-ended to allow for interviewers to clarify a participant's response for increased credibility and confirmability.

    “Interview methods range from highly structured to increase dependability or completely open-ended to allow for interviewers to clarify a participant's response for increased credibility and confirmability.”
  • The paper relies on this without support.

    Interview protocols and structure are often modified or refined, based on concurrent data collection and analysis processes to support or refute preliminary interpretations and refine focus and continuing inquiry.

    “Regardless, interview protocols and structure are often modified or refined, based on concurrent data collection and analysis processes to support or refute preliminary interpretations and refine focus and continuing inquiry.”
  • The paper relies on this without support.

    Prolonged engagement supports reflexivity by allowing the researcher to determine how they themselves may be a source of bias during data collection by altering how individuals behave or interact in the researcher's presence.

    “This strategy supports reflexivity, allowing the researcher to determine how they themselves may be a source of bias during the data collection process by altering the nature of how individuals behave or interact with others in the presence of the researcher.”
  • The paper relies on this without support.

    In observational methods, it is impossible to separate the researcher from the environment.

    “In both observation scenarios, it is impossible to separate the researcher from the environment, and researcher reflexivity is essential.”
  • The paper relies on this without support.

    Observational methodologies are limited by the researcher's bias in determining what is observed, what is recorded, how it is recorded, and how it is transcribed for analysis.

    “Researchers acknowledge that observational methodologies are limited by the reality that the researcher carries a bias in determining what is observed, what is recorded, how it is recorded, and how it is transcribed for analysis.”
  • The paper relies on this without support.

    In qualitative studies, the researcher is often the primary instrument for data collection.

    “In qualitative studies, the researcher is often the primary instrument for data collection.”
  • The paper relies on this without support.

    Qualitative studies are creative, interpretative, and contextually bound in nature.

    “Thus, due to the creative interpretative and contextually bound nature of qualitative studies, the application of standards of rigor and adherence to systematic processes welldocumented in an audit trail are essential.”
21 of 21

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Cited author's surname differs from reference 21

    Minor

    The text spells the third author's surname "Sherpbier," while reference 21 spells it "Scherpbier."

1 of 1

Cites

Flagged first, then by citations

1 of the 46 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

  1. Creswell, 2013, Qualitative, quantitative, and mixed methods approaches, Res Des, 1

Cited by

Sorted by citations

Authors
Johnson, J. L., Adkins, D., & Chauvin, S.
Affiliations
  • William Carey University (US)
  • Louisiana State University (US)
Venue
American Journal of Pharmaceutical Education
Publisher
Elsevier BV
Published
2020-01
Volume
84
Issue
1
Pages
7120
ISSN
0002-9459 (ISSN-L)
OpenAlex
W2936015247
Cited by
1,238
References
46
Citations by year
2026: 228 · 2025: 348 · 2024: 316 · 2023: 178 · 2022: 94 · 2021: 40 · 2020: 7 · 2019: 2
FWCI
86.37
Type
journal article
Language
en
Open access
Locations
  • American Journal of Pharmaceutical Education (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (submittedVersion)
Topics
  • Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
  • Social Sciences › Education › Qualitative Research Methods and Applications
Keywords
qualitative research rigor, research quality indicators, health professions education, research design, trustworthiness, researcher bias, reflexivity, conceptual framework
MeSH terms
Data Collection, Education, Pharmacy, Fellowships and Scholarships, Humans, Research Design, Research Personnel, Quality Indicators, Health Care

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-11.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-11.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review