Endometriosis diagnosis and staging by operating surgeon and expert review using multiple diagnostic tools: an inter-rater agreement study
Open accessAbstract
Objective: To determine agreement on endometriosis diagnosis between real‐time laparoscopy and subsequent expert review of digital images, operative reports, magnetic resonance imaging ( MRI ), and histopathology, viewed sequentially. Design: Inter‐rater agreement study. Setting: Five urban surgical centres. Population: Women, aged 18–44 years, who underwent a laparoscopy regardless of clinical indication. A random sample of 105 women with and 43 women without a postoperative endometriosis diagnosis was obtained from the ENDO study. Methods: Laparoscopies were diagnosed, digitally recorded, and reassessed. Main outcome measures: Inter‐observer agreement of endometriosis diagnosis and staging according to the revised American Society for Reproductive Medicine criteria. Prevalence and bias‐adjusted kappa values ( κ ) were calculated for diagnosis, and weighted κ values were calculated for staging. Results: Surgeons and expert reviewers had substantial agreement on diagnosis and staging after viewing digital images ( n = 148; mean κ = 0.67, range 0.61–0.69; mean κ = 0.64, range 0.53–0.78, respectively) and after additionally viewing operative reports ( n = 148; mean κ = 0.88, range 0.85–0.89; mean κ = 0.85, range 0.84–0.86, respectively). Although additionally viewing MRI findings ( n = 36) did not greatly impact agreement, agreement substantially decreased after viewing histological findings ( n = 67), with expert reviewers changing their assessment from a positive to a negative diagnosis in up to 20% of cases. Conclusion: Although these findings suggest that misclassification bias in the diagnosis or staging of endometriosis via visualised disease is minimal, they should alert gynaecologists who review operative images in order to make decisions on endometriosis treatment that operative reports/drawings and histopathology, but not necessarily MRI , will improve their ability to make sound judgments. Tweetable abstract: Endometriosis diagnosis and staging agreement between expert reviewers and operating surgeons was substantial.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Our previous work showed high agreement for endometriosis diagnosis among 8 expert surgeons viewing operative digital images (κ=0.69) or digital images in addition to operative reports (κ=0.88).
“Our previous work showed high agreement for endometriosis diagnosis among 8 expert surgeons viewing operative digital images (κ=0.69) or digital images in addition to operative reports (κ=0.88).26,27”
From Examining the co-occurrence of endometriosis and polycystic ovarian syndrome (2023) · cites this paper as [27]
The Results report κ=0.88 for diagnosis after reviewers saw digital images and operative reports, but κ=0.67 for images alone. The Methods say this study recruited four expert surgeons, not eight; the Introduction attributes κ=0.69 between expert consultants viewing recorded laparoscopies to earlier work. Page numbers are not available in our copy. (LLM, conf=0.90)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Statistical Methods for Rates and Proportions
Fleiss, J. L., Levin, B., & Paik, M. C. (2003) · Wiley Series in Probability and Statistics · doi:10.1002/0471445428 · Cited by 18,646 · see citing works
(2005) · PubMed · Cited by 7,011 · see citing works
The kappa statistic in reliability studies: use, interpretation, and sample size requirements
Sim, J., & Wright, C. C. (2005) · Physical Therapy · doi:10.1093/ptj/85.3.257 · Cited by 4,093 · see citing works
ESHRE guideline: management of women with endometriosis
Dunselman, G. A. J., Vermeulen, N., Becker, C., Calhaz-Jorge, C., D'Hooghe, T., De Bie, B., Heikinheimo, O., Horne, A. W., Kiesel, L., Nap, A., et al. (2014) · Human Reproduction · doi:10.1093/humrep/det457 · Cited by 2,211 · see citing works
Giudice, L. C. (2010) · New England Journal of Medicine · doi:10.1056/nejmcp1000274 · Cited by 1,924 · see citing works
Eskenazi, B., & Warner, M. L. (1997) · Obstetrics and Gynecology Clinics of North America · doi:10.1016/s0889-8545(05)70302-8 · Cited by 1,617 · see citing works
ESHRE guideline for the diagnosis and treatment of endometriosis
Kennedy, S., Bergqvist, A., Chapron, C., D’Hooghe, T., Dunselman, G., Greb, R., Hummelshoj, L., Prentice, A., & Saridogan, E. (2005) · Human Reproduction · doi:10.1093/humrep/dei135 · Cited by 1,584 · see citing works
Endometriosis and infertility: a committee opinion
(2012) · Fertility and Sterility · doi:10.1016/j.fertnstert.2012.05.031 · Cited by 633 · see citing works
Incidence of endometriosis by study population and diagnostic method: the ENDO study
Buck Louis, G. M., Hediger, M. L., Peterson, C. M., Croughan, M., Sundaram, R., Stanford, J., Chen, Z., Fujimoto, V. Y., Varner, M. W., Trumble, A., et al. (2011) · Fertility and Sterility · doi:10.1016/j.fertnstert.2011.05.087 · Cited by 309 · see citing works
Self-citationBalasch, J., Creus, M. T., Fábregues, F., Carmona, F., Ordï, J., Martínez‐Román, S., & Vanrell, J. A. (1996) · Human Reproduction · doi:10.1093/humrep/11.2.387 · Cited by 278 · see citing works
(1991) · PubMed · Cited by 202 · see citing works
Endometriosis: Correlation between histologic and visual findings at laparoscopy
Walter, A. J., Hentz, J. G., Magtibay, P. M., Cornella, J. L., & Magrina, J. F. (2001) · American Journal of Obstetrics and Gynecology · doi:10.1067/mob.2001.115747 · Cited by 197 · see citing works
REVIEW: Accuracy of laparoscopy in the diagnosis of endometriosis: a systematic quantitative review
Wykes, C. B., Clark, T. J., & Khan, K. S. (2004) · BJOG: An International Journal of Obstetrics & Gynaecology · doi:10.1111/j.1471-0528.2004.00433.x · Cited by 177 · see citing works
Nnoaham, K. E., Webster, P., Kumbang, J., Kennedy, S. H., & Zondervan, K. T. (2012) · Fertility and Sterility · doi:10.1016/j.fertnstert.2012.05.035 · Cited by 174 · see citing works
Stratton, P., Winkel, C. A., Premkumar, A., Chow, C., Wilson, J., Hearns-Stokes, R., Heo, S., Merino, M. J., & Nieman, L. K. (2003) · Fertility and Sterility · doi:10.1016/s0015-0282(03)00155-9 · Cited by 143 · see citing works
The revised American Fertility Society classification of endometriosis: reproducibility of scoring
Rock, J. A., & Zoladex Endometriosis Study Group (1995) · Fertility and Sterility · doi:10.1016/s0015-0282(16)57556-6 · Cited by 97 · see citing works
Diagnosis of pelvic endometriosis: fat-suppressed T1-weighted vs conventional MR images.
Ha, H. K., Lim, Y.-T., Kim, H. S., et al. (1994) · American Journal of Roentgenology · doi:10.2214/ajr.163.1.8010198 · Cited by 95 · see citing works
Relating Pelvic Pain Location to Surgical Findings of Endometriosis
Hsu, A. L., Sinaii, N., Segars, J. H., Nieman, L. K., & Stratton, P. (2011) · Obstetrics and Gynecology · doi:10.1097/aog.0b013e318223fed0 · Cited by 82 · see citing works
Histologic confirmation of endometriosis in different peritoneal lesions
Moen, M. H., & Halvorsen, T. B. (1992) · Acta Obstetricia Et Gynecologica Scandinavica · doi:10.3109/00016349209021069 · Cited by 66 · see citing works
Yang, H., Zhu, L., Wang, S., Lang, J., & Xu, T. (2015) · Journal of Minimally Invasive Gynecology · doi:10.1016/j.jmig.2013.06.003 · Cited by 37 · see citing works
MR staging of pelvic endometriosis: role of fat-suppression T1-weighted images.
Tanaka, Y., Itai, Y., Anno, I., et al. (1996) · PubMed · Cited by 33 · see citing works
Interrater and intrarater reliability in the diagnosis and staging of endometriosis
Schliep, K. C., Stanford, J. B., Chen, Z., Zhang, B., Dorais, J. K., Boiman Johnstone, E., Hammoud, A. O., Varner, M. W., Louis, G. M. B., & Peterson, C. M. (2012) · Obstetrics & Gynecology · doi:10.1097/aog.0b013e31825bc6cf · Cited by 33 · see citing works
Self-citationInterobserver variability in the diagnosis of minimal and mild endometriosis
Buchweitz, O., Wülfing, P., & Malik, E. (2005) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2005.02.002 · Cited by 30 · see citing works
Weijenborg, P. T. M., ter Kuile, M. M., & Jansen, F. W. (2006) · Fertility and Sterility · doi:10.1016/j.fertnstert.2006.06.052 · Cited by 18 · see citing works
Biopsy in laparoscopically diagnosed endometriosis.
Chatman, D. L., & Zbella, E. A. (1987) · PubMed · Cited by 15 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Clinical diagnosis of endometriosis: a call to action
Agarwal, S. K., Chapron, C., Giudice, L. C., Laufer, M. R., Leyland, N., Missmer, S. A., Singh, S. S., & Taylor, H. S. (2019) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2018.12.039 · Cited by 843 · see citing works
Adiposity and endometriosis severity and typology
Byun, J., Peterson, C. M., Backonja, U., Taylor, R. N., Stanford, J. B., Allen-Brady, K. L., Smith, K. R., Louis, G. M. B., & Schliep, K. C. (2020) · Journal of Minimally Invasive Gynecology · doi:10.1016/j.jmig.2020.01.002 · Cited by 27 · see citing works
“While we could not confirm whether these patients truly had less severe disease than originally assigned by the operating surgeon or whether the biopsy failed to sample and/or detect disease, we cannot rule out misclassification of disease severity in our study [36].”
Self-citationExamining the co-occurrence of endometriosis and polycystic ovarian syndrome
Schliep, K. C., Ghabayen, L., Shaaban, M., Hughes, F. R., Pollack, A. Z., Stanford, J. B., Brady, K. A., Kiser, A., & Peterson, C. M. (2023) · AJOG Global Reports · doi:10.1016/j.xagr.2023.100259 · Cited by 9 · see citing works
Self-citation
- Authors
- Schliep, K., Chen, Z., Stanford, J., Xie, Y., Mumford, S., Hammoud, A., Boiman Johnstone, E., Dorais, J., Varner, M., Buck Louis, G., & Peterson, C.
- Affiliations
- National Institutes of Health (US)
- Department of Family and Preventive Medicine University of Utah Salt Lake City Utah USA
- Division of Intramural Population Health Research <i>Eunice Kennedy Shriver</i> National Institute of Child Health and Human Development National Institutes of Health Rockville Maryland USA
- University of Utah (US)
- Department of Obstetrics and Gynecology University of Utah Salt Lake City Utah USA
- Corresponding author
- Schliep, KC
- ORCID
- Venue
- BJOG: An International Journal of Obstetrics & Gynaecology
- Publisher
- Wiley
- Published
- 2015-10-05
- Volume
- 124
- Issue
- 2
- Pages
- 220-229
- ISSN
- 0306-5456 (ISSN-L)
- 1470-0328
- 1471-0528
- PMC
- PMC4821828
- OpenAlex
- W1910949098
- Cited by
- 39
- References
- 25
- Citations by year
- 2026: 2 · 2025: 4 · 2024: 4 · 2023: 7 · 2022: 4 · 2021: 7 · 2020: 3 · 2019: 3 · 2018: 4 · 2016: 1
- FWCI
- 1.90
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- BJOG An International Journal of Obstetrics & Gynaecology (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- Eunice Kennedy Shriver National Institute of Child Health and Human Development
- National Institutes of Health (NO1‐DK‐6‐3428, NO1‐DK‐6‐3427, 10001406‐02)
- Topics
- Health Sciences › Medicine › Reproductive Medicine › Endometriosis Research and Treatment
- Health Sciences › Medicine › Obstetrics and Gynecology › Gynecological conditions and treatments
- Health Sciences › Medicine › Obstetrics and Gynecology › Endometrial and Cervical Cancer Treatments
- Keywords
- endometriosis, endometriosis staging, interobserver agreement, diagnostic laparoscopy, operative reports, histopathology, magnetic resonance imaging, misclassification bias, expert review
- MeSH terms
- Adolescent, Adult, Endometriosis, Female, Humans, Laparoscopy, Reproducibility of Results, Observer Variation, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
