Validation of an algorithm-based definition of treatment resistance in patients with schizophrenia
Open accessAbstract
Large-scale pharmacoepidemiological research on treatment resistance relies on accurate identification of people with treatment-resistant schizophrenia (TRS) based on data that are retrievable from administrative registers. This is usually approached by operationalising clinical treatment guidelines by using prescription and hospital admission information. We examined the accuracy of an algorithm-based definition of TRS based on clozapine prescription and/or meeting algorithm-based eligibility criteria for clozapine against a gold standard definition using case notes. We additionally validated a definition entirely based on clozapine prescription. 139 schizophrenia patients aged 18-65years were followed for a mean of 5years after first presentation to psychiatric services in South-London, UK. The diagnostic accuracy of the algorithm-based measure against the gold standard was measured with sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV). A total of 45 (32.4%) schizophrenia patients met the criteria for the gold standard definition of TRS; applying the algorithm-based definition to the same cohort led to 44 (31.7%) patients fulfilling criteria for TRS with sensitivity, specificity, PPV and NPV of 62.2%, 83.0%, 63.6% and 82.1%, respectively. The definition based on lifetime clozapine prescription had sensitivity, specificity, PPV and NPV of 40.0%, 94.7%, 78.3% and 76.7%, respectively. Although a perfect definition of TRS cannot be derived from available prescription and hospital registers, these results indicate that researchers can confidently use registries to identify individuals with TRS for research and clinical practices.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Cites
Clozapine for the Treatment-Resistant Schizophrenic: A Double-blind Comparison With Chlorpromazine
Kane, J. (1988) · Archives of General Psychiatry · doi:10.1001/archpsyc.1988.01800330013001 · Cited by 4,156 · see citing works
Understanding and using sensitivity, specificity and predictive values
Parikh, R. S., Mathai, A., Parikh, S. R., Sekhar, G. C., & Thomas, R. (2008) · Indian Journal of Ophthalmology · doi:10.4103/0301-4738.37595 · Cited by 1,395 · see citing works
Howes, O. D., McCutcheon, R. A., Agid, O., de Bartolomeis, A., van Beveren, N. J. M., Birnbaum, M. L., Bloomfield, M., Bressan, R. A., Buchanan, R. W., Carpenter, W. T., et al. (2016) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2016.16050503 · Cited by 1,145 · see citing works
Self-citationDi Forti, M., Sallis, H., Allegri, F., Trotta, A., Ferraro, L., Stilo, S. A., Marconi, A., La Cascia, C., Reis Marques, T., Pariante, C., et al. (2014) · Schizophrenia Bulletin · doi:10.1093/schbul/sbt181 · Cited by 470 · see citing works
Self-citationStewart, R., Soremekun, M., Perera, G., Broadbent, M., Callard, F., Denis, M., Hotopf, M., Thornicroft, G., & Lovestone, S. (2009) · BMC Psychiatry · doi:10.1186/1471-244x-9-51 · Cited by 445 · see citing works
The social and economic burden of treatment-resistant schizophrenia: a systematic literature review
Kennedy, J. L., Altar, C. A., Taylor, D. L., Degtiar, I., & Hornberger, J. C. (2014) · International Clinical Psychopharmacology · doi:10.1097/yic.0b013e32836508e6 · Cited by 392 · see citing works
Dose Equivalents for Second-Generation Antipsychotics: The Minimum Effective Dose Method
Leucht, S., Samara, M. T., Heres, S., Patel, M. X., Woods, S. W., & Davis, J. M. (2014) · Schizophrenia Bulletin · doi:10.1093/schbul/sbu001 · Cited by 373 · see citing works
Howes, O. D., Vergunst, F., Gee, S., McGuire, P. K., Kapur, S., & Taylor, D. M. (2012) · The British Journal of Psychiatry · doi:10.1192/bjp.bp.111.105833 · Cited by 355 · see citing works
Lally, J., Ajnakina, O., Stubbs, B., Cullinane, M., Murphy, K. C., Gaughran, F., & Murray, R. M. (2017) · British Journal of Psychiatry · doi:10.1192/bjp.bp.117.201475 · Cited by 335 · see citing works
Self-citationTreatment-Resistant Schizophrenia
Elkis, H. (2007) · Psychiatric Clinics of North America · doi:10.1016/j.psc.2007.04.001 · Cited by 332 · see citing works
Lally, J., Ajnakina, O., Di Forti, M., Trotta, A., Demjaha, A., Kolliakou, A., Mondelli, V., Reis Marques, T., Pariante, C., Dazzan, P., et al. (2016) · Psychological Medicine · doi:10.1017/s0033291716002014 · Cited by 298 · see citing works
Self-citationReappraising the long-term course and outcome of psychotic disorders: the AESOP-10 study
Morgan, C., Lappin, J., Heslin, M., Donoghue, K., Lomas, B., Reininghaus, U., Onyejiaka, A., Croudace, T., Jones, P. B., Murray, R. M., et al. (2014) · Psychological Medicine · doi:10.1017/s0033291714000282 · Cited by 248 · see citing works
Self-citationTreatment-Resistant Schizophrenia
Elkis, H., & Buckley, P. F. (2016) · Psychiatric Clinics of North America · doi:10.1016/j.psc.2016.01.006 · Cited by 184 · see citing works
Predictors of treatment resistance in patients with schizophrenia: a population-based cohort study
Wimberley, T., Støvring, H., Sørensen, H. J., Horsdal, H. T., MacCabe, J. H., & Gasse, C. (2016) · The Lancet Psychiatry · doi:10.1016/s2215-0366(15)00575-1 · Cited by 153 · see citing works
Self-citationSchoeler, T., Petros, N., Forti, M. D., Klamerus, E., Foglia, E., Murray, R. M., & Bhattacharyya, S. (2017) · The Lancet Psychiatry · doi:10.1016/s2215-0366(17)30233-x · Cited by 138 · see citing works
Self-citationMortality and Self-Harm in Association With Clozapine in Treatment-Resistant Schizophrenia
Wimberley, T., MacCabe, J. H., Laursen, T. M., Sørensen, H. J., Astrup, A., Horsdal, H. T., Gasse, C., & Støvring, H. (2017) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2017.16091097 · Cited by 133 · see citing works
Self-citationReliability of the Life Chart Schedule for assessment of the long-term course of schizophrenia
Susser, E., Finnerty, M. T., Mojtabai, R., Yale, S. A., Conover, S., Goetz, R. R., & AMADOR, X. F. (2000) · Schizophrenia Research · doi:10.1016/s0920-9964(99)00088-2 · Cited by 93 · see citing works
Polygenic Risk Score for Schizophrenia and Treatment-Resistant Schizophrenia
Wimberley, T., Gasse, C., Meier, S. M., Agerbo, E., MacCabe, J. H., & Horsdal, H. T. (2017) · Schizophrenia Bulletin · doi:10.1093/schbul/sbx007 · Cited by 89 · see citing works
Self-citationRemission and relapse in psychosis: operational definitions based on case-note data
Bebbington, P., Craig, T., Garety, P. A., Fowler, D. G., Dunn, G., Colbert, S., Fornells‐Ambrojo, M., & Kuipers, E. (2006) · Psychological Medicine · doi:10.1017/s0033291706008579 · Cited by 57 · see citing works
Huber, C. G., Naber, D., & Lambert, M. (2008) · Expert Opinion on Pharmacotherapy · doi:10.1517/14656566.9.12.2027 · Cited by 48 · see citing works
Ajnakina, O., Lally, J., Forti, M. D., Kolliakou, A., Gardner-Sood, P., López-Moríñigo, J.-D., Dazzan, P., Pariante, C. M., Mondelli, V., MacCabe, J. H., et al. (2017) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-017-1417-6 · Cited by 36 · see citing works
Self-citationFactors Associated With Initiation on Clozapine and on Other Antipsychotics Among Medicaid Enrollees
Manuel, J. I., Essock, S. M., Wu, Y., Pangilinan, M., & Stroup, T. S. (2012) · Psychiatric Services · doi:10.1176/appi.ps.201100435 · Cited by 34 · see citing works
Wheeler, A., Feetam, C., & Harrison, J. (2014) · Clinical Drug Investigation · doi:10.1007/s40261-013-0166-x · Cited by 23 · see citing works
Inverse association between urbanicity and treatment resistance in schizophrenia
Wimberley, T., Pedersen, C. B., MacCabe, J. H., Støvring, H., Astrup, A., Sørensen, H. J., Horsdal, H. T., Mortensen, P. B., & Gasse, C. (2016) · Schizophrenia Research · doi:10.1016/j.schres.2016.03.021 · Cited by 20 · see citing works
Self-citationHorsdal, H. T., Wimberley, T., Benros, M. E., & Gasse, C. (2017) · Human Psychopharmacology Clinical and Experimental · doi:10.1002/hup.2632 · Cited by 18 · see citing works
Self-citation
Flagged first, then by citations
1 of the 26 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
DNA methylation and regulation of gene expression: Guardian of our health
Dhar, G. A., Saha, S., Mitra, P., & Chaudhuri, R. N. (2021) · The Nucleus · doi:10.1007/s13237-021-00367-y · Cited by 252 · see citing works
Hannon, E., Dempster, E. L., Mansell, G., Burrage, J., Bass, N., Bohlken, M. M., Corvin, A., Curtis, C. J., Dempster, D., Di Forti, M., et al. (2021) · eLife · doi:10.7554/elife.58430 · Cited by 132 · see citing works
“…from DNA methylation data Up to 25% of schizophrenia patients are resistant to the most commonly prescribed antipsychotic medications, and clozapine is a second-generation antipsychotic often prescribed to patients with such TRS who may represent a more severe subgroup ( Ajnakina et al., 2018 ).”
Self-citationPolese, D., Fornaro, M., Palermo, M., Luca, V. D., & de Bartolomeis, A. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00210 · Cited by 70 · see citing works
Labonté, C., Zhand, N., Park, A., & Harvey, P. D. (2021) · Psychiatry Research · doi:10.1016/j.psychres.2021.114382 · Cited by 47 · see citing works
Kowalec, K., Lu, Y., Sariaslan, A., Song, J., Ploner, A., Dalman, C., Hultman, C. M., Larsson, H., Lichtenstein, P., & Sullivan, P. F. (2019) · Molecular Psychiatry · doi:10.1038/s41380-019-0575-1 · Cited by 44 · see citing works
Schizophrenia polygenic risk scores, urbanicity and treatment-resistant schizophrenia
Gasse, C., Wimberley, T., Wang, Y., Mors, O. N., Børglum, A. D., Als, T. D., Werge, T. M., Nordentoft, M., HOUGAARD, D. M., & Horsdal, H. T. (2019) · Schizophrenia Research · doi:10.1016/j.schres.2019.08.008 · Cited by 33 · see citing works
Self-citationAjnakina, O., Agbedjro, D., Lally, J., Forti, M. D., Trotta, A., Mondelli, V., Pariante, C. M., Dazzan, P., Gaughran, F., Fisher, H. L., et al. (2020) · Psychiatry Research · doi:10.1016/j.psychres.2020.113527 · Cited by 26 · see citing works
Self-citationAjnakina, O., Das, T. K., Lally, J., Forti, M. D., Pariante, C. M., Marques, T. R., Mondelli, V., David, A. S., Murray, R. M., Palaniyappan, L. K., et al. (2021) · Schizophrenia Bulletin · doi:10.1093/schbul/sbab035 · Cited by 26 · see citing works
Self-citationNasyrova, R., Khasanova, A. K., Altynbekov, K. S., Asadullin, A., Markina, E. A., Gayduk, A., Shipulin, G., Petrova, M. M., & Shnayder, N. (2022) · Nutrients · doi:10.3390/nu14235142 · Cited by 16 · see citing works
de Freitas, R. R., dos Santos, B., Altamura, C. A., Bernasconi, C., Corral, R. M., Evans, J., MALLA, A. K., Krebs, M., Nordstroem, A.-L., Zink, M., et al. (2019) · Psychiatry Research · doi:10.1016/j.psychres.2019.05.002 · Cited by 18 · see citing works
Ursodeoxycholic acid augmentation in treatment-refractory schizophrenia: a case report
Khosravi, M. (2020) · Journal of Medical Case Reports · doi:10.1186/s13256-020-02484-9 · Cited by 14 · see citing works
A predictor model of treatment resistance in schizophrenia using data from electronic health records
Kadra-Scalzo, G., de Freitas, D. F., Agbedjro, D., Francis, E., Ridler, I., Pritchard, M., Shetty, H., Segev, A., Casetta, C., Smart, S. E., et al. (2022) · PLoS ONE · doi:10.1371/journal.pone.0274864 · Cited by 14 · see citing works
Self-citationMuramyldipeptide - based compounds in current medicine: focus on glucosaminylmuramyl dipeptide
Ушкалова, Е. А., Zyryanov, S. K., & Затолочина, К. Э. (2019) · Terapevticheskii arkhiv · doi:10.26442/00403660.2019.12.000471 · Cited by 6 · see citing works
Ochi, S., Inoue, S., Yoshino, Y., Shimizu, H., Iga, J., & Ueno, S. (2019) · Clinical Psychopharmacology and Neuroscience · doi:10.9758/cpn.2019.17.4.559 · Cited by 6 · see citing works
Aguirre, J. M., Dellarossa, C. D., Barbagelata, D., Núñez, J. V., Mena, C., Tepper, Á., Ramírez-Mahaluf, J. P., Aceituno, D., Nachar, R., Undurraga, J. P., et al. (2025) · Schizophrenia Research · doi:10.1016/j.schres.2025.01.017 · Cited by 5 · see citing works
Tepper, Á., Núñez, J. V., Dellarossa, C. D., Ramírez-Mahaluf, J. P., Aguirre, J. M., Barbagelata, D., Aceituno, D., Undurraga, J. P., Nachar, R., Mena, C., et al. (2025) · Schizophrenia · doi:10.1038/s41537-025-00653-7 · Cited by 3 · see citing works
Conejeros‐Pavez, J., Núñez, J. V., Díaz, C., Mena, C., Undurraga, J. P., González‐Valderrama, A., Claro, S., Undurraga, E. A., & Crossley, N. A. (2025) · Psychological Medicine · doi:10.1017/s0033291725101062 · Cited by 1 · see citing works
Hannon, E. J., Dempster, E. L., Mansell, G., Burrage, J., Bass, N. J., Bohlken, M. M., Corvin, A. P., Curtis, C. J. C., Dempster, D. W., Forta, M. D., et al. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.04.05.026211 · Cited by 1 · see citing works
Self-citation
- Authors
- Ajnakina, O., Horsdal, H. T., Lally, J., MacCabe, J. H., Murray, R. M., Gasse, C., & Wimberley, T.
- Affiliations
- King's College London (GB)
- Aarhus University (DK)
- University College Dublin (IE)
- Corresponding author
- Ajnakina, Olesya
- ORCID
- Venue
- Schizophrenia Research
- Publisher
- Elsevier BV
- Published
- 2018-07
- Volume
- 197
- Pages
- 294-297
- ISSN
- 0920-9964 (ISSN-L)
- OpenAlex
- W2794259381
- Cited by
- 20
- References
- 26
- Citations by year
- 2025: 3 · 2022: 2 · 2021: 4 · 2020: 4 · 2019: 6 · 2018: 1
- FWCI
- 2.95
- Type
- journal article
- Language
- en
- Open access
- green
- kclpure.kcl.ac.uk
- Licence
- Locations
- Schizophrenia Research (publishedVersion)
- PubMed (publishedVersion)
- Research Portal (King's College London) (publishedVersion, cc-by-nc-nd)
- UCL Discovery (University College London) (submittedVersion)
- Figshare (submittedVersion, cc-by-nc-sa)
- Scopus (Elsevier) (submittedVersion)
- Funders
- European Community's Seventh Framework Programme (FP7/2007-2013)
- Maudsley Charitable Fund
- King's College London
- UK National Institute of Health Research (RP-PG-0606-1049)
- BRC-SLAM
- The Lundbeck Foundation (R155-2014-1724)
- Topics
- Health Sciences › Medicine › Psychiatry and Mental health › Schizophrenia research and treatment
- Social Sciences › Arts and Humanities › Philosophy › Mental Health and Psychiatry
- Social Sciences › Psychology › Social Psychology › Mental Health Treatment and Access
- Keywords
- treatment-resistant schizophrenia, clozapine
- MeSH terms
- Adolescent, Adult, Aged, Algorithms, Clozapine, Drug Resistance, Female, Humans, London, Male, Middle Aged, Drug Prescriptions, Registries, Schizophrenia, Sensitivity and Specificity, Antipsychotic Agents, Cohort Studies, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
