The Development of Delta: Using Agile to Develop a Decision Aid for Pediatric Oncology Clinical Trial Enrollment
Open accessPublished version of a preprint
Abstract
BACKGROUND: The internet is increasingly being used to disseminate health information. Given the complexity of pediatric oncology clinical trials, we developed Delta, a Web-based decision aid to support families deciding whether or not to enroll their child with cancer in a clinical trial. OBJECTIVE: This paper details the Agile development process of Delta and user testing results of Delta. METHODS: Development was iterative and involved 5 main stages: a requirements analysis, planning, design, development, and user testing. For user testing, we conducted 13 eye-tracking analyses and think-aloud interviews with health care professionals (n=6) and parents (n=7). RESULTS: Results suggested that there was minimal rereading of content and a high level of engagement in content. However, there were some navigational problems. Participants reported high acceptability (12/13) and high usability of the website (8/13). CONCLUSIONS: Delta demonstrates the utility for the use of Agile in the development of a Web-based decision aid for health purposes. Our study provides a clear step-by-step guide to develop a Web-based psychosocial tool within the health setting.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Decision aids for people facing health treatment or screening decisions
Stacey, D., Légaré, F., Lewis, K., Barry, M. J., Bennett, C. L., Eden, K. B., Holmes-Rovner, M., Llewellyn-Thomas, H., Lyddiatt, A., Thomson, R., et al. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001431.pub5 · Cited by 6,685 · see citing works
Nutbeam, D. (2000) · Health Promotion International · doi:10.1093/heapro/15.3.259 · Cited by 5,205 · see citing works
Empirical studies of agile software development: A systematic review
Dybå, T., & Dingsøyr, T. (2008) · Information and Software Technology · doi:10.1016/j.infsof.2008.01.006 · Cited by 2,668 · cited 1× · see citing works
“Multiple case studies have acknowledged the challenges of implementing Agile development, such as highly intensive interactions with stakeholders [19], however many have reported an overall positive impact [20-23].”
Elwyn, G. (2006) · BMJ · doi:10.1136/bmj.38926.629329.ae · Cited by 1,824 · see citing works
Refining the Test Phase of Usability Evaluation: How Many Subjects Is Enough?
Virzi, R. A. (1992) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/001872089203400407 · Cited by 1,237 · see citing works
Agile software development: the business of innovation
Highsmith, J. A., & Cockburn, A. (2001) · Computer · doi:10.1109/2.947100 · Cited by 1,216 · see citing works
A decade of agile methodologies: Towards explaining agile software development
Dingsøyr, T., Nerur, S., Balijepally, V., & Moe, N. B. (2012) · Journal of Systems and Software · doi:10.1016/j.jss.2012.02.033 · Cited by 1,118 · see citing works
Agile software development, the people factor
Cockburn, A., & Highsmith, J. A. (2001) · Computer · doi:10.1109/2.963450 · Cited by 1,105 · see citing works
Patients’ use of the internet for medical information
Diaz, J. A., Griffith, R. A., Ng, J. J., Reinert, S. E., Friedmann, P. D., & Moulton, A. W. (2002) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2002.10603.x · Cited by 1,032 · see citing works
Challenges of migrating to agile methodologies
Nerur, S., Mahapatra, R., & Mangalaraj, G. (2005) · Communications of the ACM · doi:10.1145/1060710.1060712 · Cited by 1,018 · see citing works
A systematic development process for patient decision aids
Coulter, A., Stilwell, D., Kryworuchko, J., Mullen, P. D., Ng, C. J., & van der Weijden, T. (2013) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-13-s2-s2 · Cited by 615 · see citing works
Nilsen, E. S., Myrhaug, H. T., Johansen, M., Oliver, S. R., & Oxman, A. D. (2006) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004563.pub2 · Cited by 565 · see citing works
The think aloud method: a guide to user interface design
JASPERS, M., STEEN, T., BOS, C., & GEENEN, M. (2004) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2004.08.003 · Cited by 544 · see citing works
A teamwork model for understanding an agile team: A case study of a Scrum project
Moe, N. B., Dingsøyr, T., & Dybå, T. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2009.11.004 · Cited by 543 · see citing works
Estimating the number of subjects needed for a thinking aloud test
Nielsen, J. (1994) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1994.1065 · Cited by 476 · see citing works
Elwyn, G. J., O'Connor, A. M. C., Bennett, C. L., Newcombe, R. G., Politi, M. C., Durand, M., Drake, E. R., Joseph‐Williams, N., Khangura, S. D., Saarimaki, A., et al. (2009) · PLoS ONE · doi:10.1371/journal.pone.0004705 · Cited by 467 · see citing works
Amante, D. J., Hogan, T. P., Pagoto, S., English, T. M., & Lapane, K. L. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4126 · Cited by 350 · see citing works
A systematic review on communicating with patients about evidence
Trevena, L. J., BPsych, H. M. D., Barratt, A. L., Butow, P. N., & Caldwell, P. H. Y. (2005) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2005.00596.x · Cited by 329 · see citing works
Usage and Perceptions of Agile Software Development in an Industrial Context: An Exploratory Study
Begel, A., & Nagappan, N. (2007) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2007.12 · Cited by 173 · see citing works
Health literacy and the Internet: a study on the readability of Australian online health information
Cheng, C. C., & Dunn, M. (2015) · Australian and New Zealand Journal of Public Health · doi:10.1111/1753-6405.12341 · Cited by 158 · see citing works
Eye Tracking and Web Experience
Djamasbi, S. (2014) · AIS Transactions on Human-Computer Interaction · doi:10.17705/1thci.00060 · Cited by 155 · see citing works
End-of-Life Decision Making by Adolescents, Parents, and Healthcare Providers in Pediatric Oncology
Hinds, P. S., Oakes, L. L., Furman, W. L., Quargnenti, A. G., Olson, M. S., Foppiano, P., & Srivastava, D. K. (2001) · Cancer Nursing · doi:10.1097/00002820-200104000-00007 · Cited by 150 · see citing works
Using Scrum in Distributed Agile Development: A Multiple Case Study
Paasivaara, M., Durasiewicz, S., & Lassenius, C. (2009) · IEEE International Conference on Global Software Engineering (ICGSE) · doi:10.1109/icgse.2009.27 · Cited by 150 · see citing works
Examining Information Processing on the World Wide Web Using Think Aloud Protocols
Eveland, W. P., & Dunwoody, S. (2000) · Media Psychology · doi:10.1207/s1532785xmep0203_2 · Cited by 149 · see citing works
Design Features of Explicit Values Clarification Methods
Witteman, H. O., Scherer, L. D., Gavaruzzi, T., Pieterse, A. H., Fuhrel-Forbis, A., Dansokho, S. C., Exe, N. L., Kahn, V. C., Feldman‐Stewart, D., Col, N. F., et al. (2016) · Medical Decision Making · doi:10.1177/0272989x15626397 · Cited by 124 · see citing works
Feldman‐Stewart, D., Tong, C., Siemens, R., Alibhai, S. M. H., Pickles, T., Robinson, J. W., & Brundage, M. (2012) · Medical Decision Making · doi:10.1177/0272989x11434601 · Cited by 112 · see citing works
Decisional Regret Among Parents of Children With Cancer
Mack, J. W., Cronin, A. M., & Kang, T. I. (2016) · Journal of Clinical Oncology · doi:10.1200/jco.2016.69.1634 · Cited by 112 · see citing works
Participation in psychosocial oncology and quality-of-life research: A systematic review
Wakefield, C. E., Fardell, J. E., Doolan, E. L., Aaronson, N. K., Jacobsen, P. B., Cohn, R. J., & King, M. (2017) · The Lancet Oncology · doi:10.1016/s1470-2045(17)30100-6 · Cited by 89 · see citing works
Self-citation(2014) · PubMed · Cited by 86 · see citing works
Roth, S. P., Tuch, A. N., Mekler, E. D., Bargas-Avila, J. A., & Opwis, K. (2012) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2012.09.001 · Cited by 79 · see citing works
Communicating and Understanding the Purpose of Pediatric Phase I Cancer Trials
Cousino, M. K., Zyzanski, S. J., Yamokoski, A. D., Hazen, R. A., Baker, J. N., Noll, R. B., Rheingold, S. R., Geyer, J. R., Alexander, S. C., Drotar, D., et al. (2012) · Journal of Clinical Oncology · doi:10.1200/jco.2012.42.3004 · Cited by 77 · see citing works
Adolescent perspectives on phase I cancer research
Miller, V. A., Baker, J. N., Leek, A. C., Hizlan, S., Rheingold, S. R., Yamokoski, A. D., Drotar, D., & Kodish, E. D. (2012) · Pediatric Blood & Cancer · doi:10.1002/pbc.24326 · Cited by 75 · see citing works
Estimating, planning and managing Agile Web development projects under a value-based perspective
Torrecilla-Salinas, C. J., Sedeño, J., Escalona, M. J., & Mejías, M. (2015) · Information and Software Technology · doi:10.1016/j.infsof.2015.01.006 · Cited by 74 · see citing works
Mattsson, S., Olsson, E. M. G., Johansson, B., & Carlsson, M. E. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6830 · Cited by 58 · see citing works
Parents' Experiences in Decision Making With Childhood Cancer Clinical Trials
Woodgate, R. L., & Yanofsky, R. (2009) · Cancer Nursing · doi:10.1097/ncc.0b013e3181b43389 · Cited by 56 · see citing works
Decision Aids Can Support Cancer Clinical Trials Decisions: Results of a Randomized Trial
Politi, M. C., Kuzemchak, M. D., Kaphingst, K. A., Perkins, H., Liu, J., & Byrne, M. M. (2016) · The Oncologist · doi:10.1634/theoncologist.2016-0068 · Cited by 56 · see citing works
Salmon, P., Hill, J., Ward, J., Gravenhorst, K. C., Eden, T. O. B., & Young, B. (2012) · The Oncologist · doi:10.1634/theoncologist.2011-0308 · Cited by 54 · see citing works
Robertson, E. G., Wakefield, C. E., Signorelli, C., Cohn, R. J., Patenaude, A. F., Foster, C. L., Pettit, T., & Fardell, J. E. (2018) · Patient Education and Counseling · doi:10.1016/j.pec.2018.02.001 · Cited by 44 · see citing works
Self-citation(2013) · Routledge eBooks · doi:10.4324/9780203810675 · Cited by 40 · see citing works
Informed consent for pediatric phase 1 cancer trials: Physicians' perspectives
Yap, T. Y., Yamokoski, A. D., Hizlan, S., Zyzanski, S. J., Angiolillo, A. L., Rheingold, S. R., Baker, J. N., & Kodish, E. D. (2010) · Cancer · doi:10.1002/cncr.25158 · Cited by 35 · see citing works
Cline, R. J. W., Penner, L. A., Harper, F. W. K., Foster, T. S., Ruckdeschel, J. C., & Albrecht, T. L. (2007) · Journal of Oncology Practice · doi:10.1200/jop.0737001 · Cited by 34 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- [19]Abrahamsson P Salo O Ronkainen J Warsta J VTT 2002 2018-04-12 Espoo, VTT Publications Agile software development methods: review and analysis http://www.vtt.fi/inf/pdf/publications/2002/P478.pdf
- [21]Olsson HH Alahyari H Bosch J Climbing the Stairway to Heaven--A Mulitiple-Case Study Exploring Barriers in the Transition from Agile Development towards Continuous Deployment of Software 2012 09 05 38th Euromicro Conference on Software Engineering and Advanced Applications September 5-8, 2012 Cesme, Izmir, Turkey
- [35]Pignone M Fagerlin A Abhyankar P Col N Feldman-Stewart D Kryworuchko J Levin CA Pieterse A Reyna V Stiggelbout A Scherer L Wills C Witteman H International Patient Decision Aid Standards Collaboration 2012 2018-04-12 Clarifying and expressing values (2012 Update of the International Patient Decision Aids Standards (IPDAS) Collaboration's Background Document) http://ipdas.ohri.ca/ipdas-chapter-d.pdf
- [48]Coulter A Engaging Patients in Healthcare 2006 Berkshire McGraw-Hill Education
Cited by
Sorted by citations
Systematic Development of Patient Decision Aids: An Update from the IPDAS Collaboration
Witteman, H. O., Maki, K. G., Vaisson, G., Finderup, J., Lewis, K. B., Steffensen, K. D., Beaudoin, C., Comeau, S., & Volk, R. J. (2021) · Medical Decision Making · doi:10.1177/0272989x211014163 · Cited by 196 · see citing works
Decision-making in childhood cancer: parents’ and adolescents’ views and perceptions
Robertson, E. G., Wakefield, C. E., Shaw, J., Darlington, A., McGill, B. C., Cohn, R. J., & Fardell, J. E. (2019) · Supportive Care in Cancer · doi:10.1007/s00520-019-04728-x · Cited by 80 · see citing works
Self-citationSin, J., Henderson, C., Woodham, L. A., Sesé Hernández, A., & Gillard, S. (2019) · Journal of Medical Internet Research · doi:10.2196/14374 · Cited by 52 · see citing works
Papachristou, N., Kotronoulas, G., Δικαίος, Ν., Allison, S. J., Eleftherochorinou, H., Rai, T., Kunz, H. R., Barnaghi, P. M., Miaskowski, C. A., & Bamidis, P. D. (2023) · Seminars in Oncology Nursing · doi:10.1016/j.soncn.2023.151433 · Cited by 43 · see citing works
Application of Information Technology to Clinical Trial Evaluation and Enrollment: A Review
von Itzstein, M. S., Hullings, M., Mayo, H., Beg, M. S., Williams, E. L., & Gerber, D. E. (2021) · JAMA Oncology · doi:10.1001/jamaoncol.2021.1165 · Cited by 36 · see citing works
User stories as lightweight requirements for agile clinical decision support development
Kannan, V., Basit, M. A., Bajaj, P., Carrington, A., Donahue, I. B., Flahaven, E. L., Medford, R. J., Melaku, T., Moran, B., Saldana, L., et al. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz123 · Cited by 34 · see citing works
Robertson, E. G., Wakefield, C. E., Cohn, R. J., Battisti, R. A., Donoghoe, M. W., Ziegler, D. S., Fardell, J. E., Mitchell, R., & O'Brien, T. A. (2019) · Psycho-Oncology · doi:10.1002/pon.5109 · Cited by 25 · see citing works
Self-citationParental decision making regarding consent to randomization on Children's Oncology Group AALL0932
Parker, K. N., Cottrell, E., Stork, L. C., & Lindemulder, S. (2021) · Pediatric Blood & Cancer · doi:10.1002/pbc.28907 · Cited by 18 · see citing works
Medeiros, C., Buckley, L., Metcalfe, K. A., Narayanan, U. G., & Widger, K. A. (2020) · Journal of Paediatrics and Child Health · doi:10.1111/jpc.14885 · Cited by 11 · see citing works
External validation of a prediction model for timely implementation of innovations in radiotherapy
Swart, R., Fijten, R., Boersma, L. J., Kalendralis, P., Behrendt, M., Ketelaars, M., Roumen, C., & Jacobs, M. (2023) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2022.109459 · Cited by 6 · see citing works
Robertson, E. G., Cohen, J. E., Signorelli, C., Grant, D. M., Fardell, J. E., & Wakefield, C. E. (2019) · Journal of Child Health Care · doi:10.1177/1367493519869717 · Cited by 5 · see citing works
Self-citationDuffy, A., Christie, G., & Moreno, S. (2021) · JMIR Research Protocols · doi:10.2196/28083 · Cited by 3 · see citing works
Halilaj, I., Lieverse, R. I. Y., Oberije, C., Hendriks, L. E. L., Billiet, C., Joye, I., van Eeckhout, B., Wind, A., Ankolekar, A., & Lambin, P. (2025) · Journal of Medical Internet Research · doi:10.2196/71817 · Cited by 2 · see citing works
Pearson, H., Darlington, A., Gibson, F., & Myall, M. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03313-z
Using agile development methodology to develop a decision aid for complex rectal cancer
Williams, A., Woods, F. E. R., O’Neill, C., Hutchings, H. A., Quyn, A., Gilbert, A., Duff, M., Jenkins, J. T., Taylor, C., Griffiths, B., et al. (2026) · European Journal of Surgical Oncology · doi:10.1016/j.ejso.2026.111813
Wyst, K. B. V., Emerick, L. M., Lane, M., Gottenborg, E., & Echalier, B. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6897820
Thiel, J., Schewe, D. M., Schedel, A., Gotter, M., Beste, C., Zeh, J., & Rempel, S. (2026) · doi:10.2196/preprints.107667
- Authors
- Robertson, E. G., Wakefield, C. E., Cohn, R. J., O'Brien, T., Ziegler, D. S., & Fardell, J. E.
- Affiliations
- UNSW Sydney (AU)
- ORCID
- Venue
- JMIR Research Protocols
- Publisher
- JMIR Publications Inc.
- Published
- 2018-05-04
- Volume
- 7
- Issue
- 5
- Pages
- e119
- ISSN
- 1929-0748 (ISSN-L)
- PMC
- PMC5960036
- OpenAlex
- W2800684347
- Cited by
- 17
- References
- 41
- Citations by year
- 2026: 3 · 2025: 2 · 2023: 2 · 2021: 4 · 2020: 1 · 2019: 5
- FWCI
- 2.49
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Research Protocols (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- UNSWorks (University of New South Wales, Sydney, Australia) (submittedVersion, cc-by-nc-nd)
- Funders
- Cancer Institute NSW (10/TPG/1-13)
- Kids Cancer Alliance
- Medical Research Council
- National Health and Medical Research Council (1067501)
- Cancer Council NSW (PG16‐02)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Health Sciences › Health Professions › General Health Professions › Health Literacy and Information Accessibility
- Health Sciences › Health Professions › General Health Professions › Patient-Provider Communication in Healthcare
- Keywords
- web-based decision aid, pediatric oncology, clinical trial recruitment, agile software development, parents of children with cancer, eye tracking, think-aloud interviews, usability, acceptability, health information dissemination
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
