Using Complexity Assessment to Inform the Development and Deployment of a Digital Dashboard for Schizophrenia Care: Case Study
Open accessPublished version of a preprint
Abstract
Background: Health care is becoming more complex. For an increasing number of individuals, interacting with health care means addressing more than just one illness or disorder, engaging in more than one treatment, and interacting with more than one care provider. Individuals with severe mental illnesses such as schizophrenia are disproportionately affected by this complexity. Characteristic symptoms can make it harder to establish and maintain relationships. Treatment failure is common even where there is access to effective treatments, increasing suicide risk. Knowledge of complex adaptive systems has been increasingly recognized as useful in understanding and developing health care. A complex adaptive system is a collection of interconnected agents with the freedom to act based on their own internalized rules, affecting each other. In a complex health care system, relevant feedback is crucial in enabling continuous learning and improvement on all levels. New technology has potential, but the failure rate of technology projects in health care is high, arguably due to complexity. The Nonadoption, Abandonment, and challenges to Scale-up, Spread, and Sustainability (NASSS) framework and complexity assessment tool (NASSS-CAT) have been developed specifically to help identify and manage complexity in technology-related development projects in health care. Objective: This study aimed to use a pilot version of the NASSS-CAT instrument to inform the development and deployment of a point-of-care dashboard supporting schizophrenia care in west Sweden. Specifically, we report on the complexity profile of the project, stakeholders’ experiences with using NASSS-CAT, and practical implications. Methods: We used complexity assessment to structure data collection and feedback sessions with stakeholders, thereby informing an emergent approach to the development and deployment of the point-of-care dashboard. We also performed a thematic analysis, drawing on observations and documents related to stakeholders' use of the NASSS-CAT to describe their views on its usefulness. Results: Application of the NASSS framework revealed different types of complexity across multiple domains, including the condition, technology, value proposition, organizational tasks and pathways, and wider system. Stakeholders perceived the NASSS-CAT tool as useful in gaining perspective and new insights, covering areas that might otherwise have been neglected. Practical implications derived from feedback sessions with managers and developers are described. Conclusions: This case study shows how stakeholders can identify and plan to address complexities during the introduction of a technological solution. Our findings suggest that NASSS-CAT can bring participants a greater understanding of complexities in digitalization projects in general.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The NASSS has been utilized prospectively and retrospectively to assess patient-oriented technologies and tools for decision-making.
“The surge in interest reflects the widespread adoption of the NASSS, which has been utilized prospectively and retrospectively to assess patient-oriented technologies and tools for decision-making purposes [12,13].”
From The NASSS (Non-Adoption, Abandonment, Scale-Up, Spread and Sustainability) framework use over time: A scoping review (2025) · cites this paper as [12]
The Methods (Overall Design; page unavailable) describe using NASSS-CAT to inform the development and deployment of a patient-facing schizophrenia-care dashboard. The Methods (NASSS Framework; page unavailable) also describe the framework as guiding and evaluating technology deployment. This supports prospective assessment for project decisions, but the paper does not establish retrospective use or widespread adoption. (LLM, conf=0.84)
Experimental — accuracy not yet measuredGremyr et al. applied the NASSS and identified various implementation complexities across several domains for their point-of-care dashboard supporting schizophrenia care.
“For example, Gremyr et al. [12] applied the NASSS and identified various implementation complexities across several domains for their point-of-care dashboard supporting schizophrenia care.”
From The NASSS (Non-Adoption, Abandonment, Scale-Up, Spread and Sustainability) framework use over time: A scoping review (2025) · cites this paper as [12]
Gremyr et al.'s analysis revealed the need for a clear value proposition that includes detailed information on costs, benefits, and risks.
“Their analysis revealed the need for a clear value proposition that includes detailed information on costs, benefits, and risks.”
From The NASSS (Non-Adoption, Abandonment, Scale-Up, Spread and Sustainability) framework use over time: A scoping review (2025) · cites this paper as [12]
Gremyr et al. used the NASSS-CAT to generate recommendations for the development and deployment of the dashboard.
“Then, they used the NASSS-CAT to generate recommendations for the development and deployment of the dashboard [12].”
From The NASSS (Non-Adoption, Abandonment, Scale-Up, Spread and Sustainability) framework use over time: A scoping review (2025) · cites this paper as [12]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414 · see citing works
Research Design: Qualitative, Quantitative, and Mixed Methods Approaches
ภู่พันธ์, ศ. (2018) · วารสารวิธีวิทยาการวิจัย (Journal of Research Methodology: JRM) · Cited by 12,777 · see citing works
(2017) · Cited by 9,643 · see citing works
Greenhalgh, T., Wherton, J., Papoutsi, C., Lynch, J., Hughes, G., A'Court, C., Hinder, S., Fahy, N., Procter, R., & Shaw, S. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.8775 · Cited by 2,887 · cited 3× · see citing works
“Greenhalgh et al [23] argued that adoption, scale-up, and spread of new technologies often fail due to complexity.”
BackgroundMethodSelf-citationThe challenge of complexity in health care
Plsek, P. E., & Greenhalgh, T. (2001) · BMJ · doi:10.1136/bmj.323.7313.625 · Cited by 2,578 · see citing works
Self-citationPerson-centered care—ready for prime time
Ekman, I., Swedberg, K., Taft, C., Lindseth, A., Norberg, A., Brink, E., Carlsson, J., Dahlin-Ivanoff, S., Johansson, I.-L., Kjellgren, K., et al. (2011) · European Journal of Cardiovascular Nursing · doi:10.1016/j.ejcnurse.2011.06.008 · Cited by 1,681 · see citing works
Studying complexity in health services research: desperately seeking an overdue paradigm shift
Greenhalgh, T., & Papoutsi, C. (2018) · BMC Medicine · doi:10.1186/s12916-018-1089-4 · Cited by 962 · see citing works
Self-citationCoping with complexity: educating for capability
Fraser, S. W., & Greenhalgh, T. (2001) · BMJ · doi:10.1136/bmj.323.7316.799 · Cited by 884 · see citing works
Self-citationTiihonen, J., Mittendorfer-Rutz, E., Majak, M., Mehtälä, J., Hoti, F., Jedenius, E., Enkusson, D., Leval, A., Sermon, J., Tanskanen, A., et al. (2017) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2017.1322 · Cited by 670 · see citing works
Handbook of Disease Burdens and Quality of Life Measures
Bradley, C., Mitchell, J., & Woodcock, A. J. (2009) · Springer eBooks · doi:10.1007/978-0-387-78665-0 · Cited by 652 · see citing works
Mair, F. S., May, C., O’Donnell, C., Finch, T., Sullivan, F., & Murray, E. (2012) · Bulletin of the World Health Organization · doi:10.2471/blt.11.099424 · Cited by 553 · cited 1× · see citing works
“Moreover, the failure rate of technology projects in health care is high; large and complex projects often tend to fail to deliver anticipated results [20-22].”
BackgroundUnderstanding Excess Mortality in Persons With Mental Illness
Druss, B. G., Zhao, L., von Esenwein, S. A., Morrato, E. H., & Marcus, S. C. (2011) · Medical Care · doi:10.1097/mlr.0b013e31820bf86e · Cited by 438 · see citing works
Spreading and scaling up innovation and improvement
Greenhalgh, T., & Papoutsi, C. (2019) · BMJ · doi:10.1136/bmj.l2068 · Cited by 382 · see citing works
Self-citationSligo, J. L., Gauld, R. D. C., Roberts, V., & Villa, L. (2016) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2016.09.007 · Cited by 277 · see citing works
Jimison, H. B., Gorman, P., Woods, S. S., Nygren, P., Walker, M., Norris, S. L., & Hersh, W. (2008) · PubMed Central · Cited by 249 · see citing works
Greenhalgh, T., Wherton, J., Papoutsi, C., Lynch, J., Hughes, G., A’Court, C., Hinder, S., Procter, R., & Shaw, S. (2018) · BMC Medicine · doi:10.1186/s12916-018-1050-6 · Cited by 232 · see citing works
Self-citationToward an Information Infrastructure for Global Health Improvement
Friedman, C. P., Rubin, J. C., & Sullivan, K. (2017) · Yearbook of Medical Informatics · doi:10.15265/iy-2017-004 · Cited by 214 · see citing works
Why Business Modeling is Crucial in the Development of eHealth Technologies
van Limburg, M., van Gemert-Pijnen, J. E., Nijland, N., Ossebaard, H. C., Hendrix, R. M., & Seydel, E. R. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1674 · Cited by 179 · see citing works
Relevant models and elements of integrated care for multi-morbidity: Results of a scoping review
Struckmann, V., Leijten, F., van Ginneken, E., Kraus, M., Reiss, M., Spranger, A., Boland, M. R. S., Czypionka, T., Busse, R., & Mölken, M. R. (2017) · Health Policy · doi:10.1016/j.healthpol.2017.08.008 · Cited by 174 · see citing works
Policy implications of big data in the health sector
Vayena, E., Dzenowagis, J., Brownstein, J. S., & Sheikh, A. (2018) · Bulletin of the World Health Organization · doi:10.2471/blt.17.197426 · Cited by 164 · see citing works
Greenhalgh, T., Maylor, H., Shaw, S., Wherton, J., Papoutsi, C., Betton, V., Nelissen, N., Gremyr, A., Rushforth, A., Koshkouei, M., et al. (2020) · JMIR Research Protocols · doi:10.2196/16861 · Cited by 152 · see citing works
Self-citationThe contribution of case study research to knowledge of how to improve quality of care
Baker, G. R. (2011) · BMJ Quality & Safety · doi:10.1136/bmjqs.2010.046490 · Cited by 131 · see citing works
Greenhalgh, T., Shaw, S., Wherton, J., Hughes, G., Lynch, J., A'Court, C., Hinder, S., Fahy, N., Byrne, E., Finlayson, A., et al. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-010208 · Cited by 85 · see citing works
Self-citationHow Hard Can It Be?: Actively Managing Complexity in Technology Projects
Maylor, H. R., Turner, N. W., & Murray-Webster, R. (2013) · Research-Technology Management · doi:10.5437/08956308x5602125 · Cited by 83 · see citing works
The Role of Technology and Engineering Models in Transforming Healthcare
Misha Pavel, C., Jimison, H. B., Wactlar, H. D., Hayes, T. L., Barkis, W., Skapik, J. L., & Kaye, J. A. (2013) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2012.2222636 · Cited by 76 · see citing works
How to improve success of technology projects in health and social care
Greenhalgh, T. (2018) · Public Health Research & Practice · doi:10.17061/phrp2831815 · Cited by 68 · see citing works
Self-citationSacristán, J. A., & Dilla, T. (2015) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12350 · Cited by 53 · see citing works
Bridging the Health Data Divide
Celi, L. A., Davidzon, G., Johnson, A. E. W., Komorowski, M., Marshall, D. C., Nair, S., Phillips, C. T., Pollard, T., Raffa, J. D., Salciccioli, J. D., et al. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.6400 · Cited by 46 · see citing works
Building a learning health system using clinical registers: a non-technical introduction
Øvretveit, J., Nelson, E. C., & James, B. C. (2016) · Journal of Health Organization and Management · doi:10.1108/jhom-06-2016-0110 · Cited by 42 · see citing works
Forsman, J., Taipale, H. T., Masterman, T., Tiihonen, J., & Tanskanen, A. J. (2019) · European Journal of Clinical Pharmacology · doi:10.1007/s00228-019-02707-z · Cited by 33 · see citing works
Action evaluation of health programmes and changes: a handbook for a user-focused approach.
Hughes, J. M. (2002) · Family Practice · doi:10.1093/fampra/19.5.569-a · Cited by 23 · see citing works
Gremyr, A., Malm, U., Lundin, L., & Andersson, A. (2019) · Digital Psychiatry · doi:10.1080/2575517x.2019.1622397 · Cited by 21 · see citing works
Self-citationSchizophrenia. Current science and clinical practice
Lauber, C. (2011) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.2011.01782.x · Cited by 17 · see citing works
Information When and Where Needed for Safe and Effective Integrated Care
Fitzsimons, M. B., Power, K., White, M., et al. (2017) · International Journal of Integrated Care · doi:10.5334/ijic.3796 · Cited by 1 · see citing works
Greenhalgh, T., Maylor, H. R., Shaw, S. E., et al. (2019) · doi:10.2196/preprints.16861 · Cited by 1 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Greenhalgh, T., Maylor, H., Shaw, S., Wherton, J., Papoutsi, C., Betton, V., Nelissen, N., Gremyr, A., Rushforth, A., Koshkouei, M., et al. (2020) · JMIR Research Protocols · doi:10.2196/16861 · Cited by 152 · see citing works
Self-citationDyb, K., Berntsen, G. R., & Kvam, L. (2021) · BMC Health Services Research · doi:10.1186/s12913-021-06262-1 · Cited by 67 · see citing works
Kadesjö Banck, J., & Bernhardsson, S. (2020) · BMC Health Services Research · doi:10.1186/s12913-020-05596-6 · Cited by 35 · see citing works
Shin, H. D., Hamovitch, E., Gatov, E., MacKinnon, M., Samawi, L., Boateng, R., Thorpe, K. E., & Barwick, M. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0000418 · Cited by 40 · see citing works
“For example, Gremyr et al. [12] applied the NASSS and identified various implementation complexities across several domains for their point-of-care dashboard supporting schizophrenia care.”
Influential
- Authors
- Gremyr, A., Andersson Gäre, B., Greenhalgh, T., Malm, U., Thor, J., & Andersson, A.-C.
- Affiliations
- Sahlgrenska University Hospital (SE)
- University of Oxford (GB)
- University of Gothenburg (SE)
- Jönköping University (SE)
- Corresponding author
- Gremyr, Andreas
- ORCID
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2020-04-23
- Volume
- 22
- Issue
- 4
- Pages
- e15521
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC7206515
- OpenAlex
- W3018710111
- Cited by
- 31
- References
- 35
- Citations by year
- 2025: 4 · 2024: 10 · 2023: 7 · 2022: 4 · 2021: 3 · 2020: 2 · 2019: 1
- FWCI
- 3.51
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Wellcome Trust (104830/Z/14/Z)
- National Institute for Health and Care Research (NF-SI-0512-10031)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Social Sciences › Sociology and Political Science › Information Systems Theories and Implementation
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Keywords
- complexity assessment, NASSS framework, complex adaptive systems, digital dashboard, health technology adoption, stakeholder feedback, digitalization of healthcare, severe mental illness, technology deployment
- MeSH terms
- Humans, Schizophrenia, Sweden, Technology Assessment, Biomedical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
