Evaluating the Benefits of Aphasia Intervention Delivered in Virtual Reality: Results of a Quasi-Randomised Study
Open accessThe DOI matches its Crossref record; 79 of the 82 references it cites were found in a registry or an index; 17 of the 68 sources its claims cite were checked for support.
Abstract
INTRODUCTION: This study evaluated an intervention for people with aphasia delivered in a novel virtual reality platform called EVA Park. EVA Park contains a number of functional and fantastic locations and allows for interactive communication between multiple users. Twenty people with aphasia had 5 weeks' intervention, during which they received daily language stimulation sessions in EVA Park from a support worker. The study employed a quasi randomised design, which compared a group that received immediate intervention with a waitlist control group. Outcome measures explored the effects of intervention on communication and language skills, communicative confidence and feelings of social isolation. Compliance with the intervention was also explored through attrition and usage data. RESULTS: There was excellent compliance with the intervention, with no participants lost to follow up and most (18/20) receiving at least 88% of the intended treatment dose. Intervention brought about significant gains on a measure of functional communication. Gains were achieved by both groups of participants, once intervention was received, and were well maintained. Changes on the measures of communicative confidence and feelings of social isolation were not achieved. Results are discussed with reference to previous aphasia therapy findings.
Overview
CoChat Summary
TL;DR
This quasi-randomised waitlist-controlled study evaluated a 5-week aphasia intervention delivered in the multi-user virtual reality platform EVA Park. It found excellent compliance and significant gains on functional communication (CADL-2), but no intervention-specific gains in communicative confidence or social isolation.
Key findings
- Compliance was excellent: no participants were lost to follow-up, 18/20 received at least 88% of the intended treatment dose, and mean time logged into EVA Park was 40.85 hours (range just under 14 to just over 100; median 34.89).
- Functional communication improved on the CADL-2: mixed ANOVA showed a significant group × time interaction, F(1,18)=5.236, p=.034, ηp²=.225; the immediate group improved from week 1 to week 7 and maintained gains at week 13, while the waitlist group improved once intervention was received (Friedman χ²=10.14, p=.006).
- Verbal fluency improved over time but was not tied to intervention; conversation, narrative, communicative confidence (CCRSA), and Friendship Scale outcomes showed no significant intervention-specific effects.
- Time logged into EVA Park correlated with prior computer use (rs=.652, p=.002), positively with CCRSA gain (rs=.446, p=.048), and negatively with conversation percentage of content words (rs=-.639, p=.002); ANCOVA for CADL-2 with log time as covariate was not significant (F=.672, p=.42).
Methods
- Quasi-randomised controlled design comparing immediate EVA Park intervention with a waitlist control group; 20 people with aphasia recruited in 4 cohorts, with testing at weeks 1, 7, and 13.
- Intervention consisted of 5 weeks of daily supported language stimulation in EVA Park (25 one-hour sessions) plus unlimited independent access; participants used loaned laptops from home.
- Outcomes included CADL-2, verbal fluency, POWERS conversation analysis, Cinderella narrative production, CCRSA, and Friendship Scale; analyses used mixed between-within ANOVAs, one-factor within-group ANOVAs, and non-parametric tests where assumptions were not met.
Limitations
- Stated: group assignment was not fully randomised; the waitlist control group scored lower on all measures at recruitment and had a large, non-significant discrepancy in time post stroke; sample size was small, raising type 1 and type 2 error concerns; the sample was younger than typical UK first-stroke populations (mean 57.8 years vs over 70); some measures (verbal fluency, CCRSA) were unstable before intervention, suggesting test-retest reliability problems; and the comparison was EVA Park vs no intervention, so no conclusions can be drawn about virtual vs conventional face-to-face therapy.
- Evident from the text: outcome assessors were not blinded to time point or group allocation, though some scoring was blind; participants were not blind to condition; independent EVA Park use was unmonitored, and usage data reflected only time logged in, not what participants did during that time.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
15 Valid · 2 Not valid · 24 Not checked · 102 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“This may be a result of the design process, which ensured that the creation of EVA Park was informed by user opinion [53].”
§ Evaluating the Benefits of Aphasia Intervention Delivered in Virtual Reality: Results of a Quasi-Randomised Study > Discussion
What the sources say
- Supportedcites [53] Wilson S, Roper A, Marshall J, Galliers J, Devane N, Booth T, et al Codesign for People with Aphasia through Tangible Design Languages. CoDesign, 2015; 11: 21– 34. 10.1080/15710882.2014.997744 source
The EVA Park section (p. 7) reports ten codesign workshops with people with aphasia. The Impact on Tools section (p. 15) says those activities directly influenced EVA Park’s settings, interactions and story themes, supporting the claim that user opinion informed its creation. (LLM, conf=0.94)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“However, gains typically follow highly targeted interventions, for example involving repeated cued naming of a set of pictures [65], or semantic feature analysis [66, 67].”
§ Evaluating the Benefits of Aphasia Intervention Delivered in Virtual Reality: Results of a Quasi-Randomised Study > Discussion
What the sources say
- Supportedcites [65] Conroy P, Sage K, Lambon Ralph M. The effects of decreasing and increasing cue therapy on improving naming speed and accuracy for verbs and nouns in aphasia. Aphasiology, 2009; 23: 707– 730. sourceExperimental — accuracy not yet measured
The Therapies section describes repeated picture-naming attempts with cues for a defined set of treatment targets. The results section reports significant naming gains for treated items, while control items showed no significant improvement. This citation supports the repeated cued-naming example; the sentence cites other sources for semantic feature analysis. (LLM, conf=0.93)
- Not checked — no full text of the sourcecites [66] Boyle M. Semantic feature analysis treatment for anomia in two fluent aphasia syndromes. American Journal of Speech-Language Pathology, 2004; 13: 236– 249. 15339233 source
- Not checked — no full text of the sourcecites [67] Wambaugh J, Mauszycki S, Wright S. Semantic feature analysis: Application to confrontation naming of actions in aphasia. Aphasiology, 2014; 28: 1– 24. source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Previous studies have reported gains in aphasic narrative production [70, 71].”
§ Evaluating the Benefits of Aphasia Intervention Delivered in Virtual Reality: Results of a Quasi-Randomised Study > Discussion
What the sources say
- Supportedcites [70] Kempler D, Goral M. A comparison of drill- and communication-based treatment for aphasia. Aphasiology, 2011; 25: 1327– 1346. 22582002 PMC3349434 source
The Analysis and Results section reports that participants produced better narratives after Generative treatment, though not after Drill treatment. The Summary and Discussion specifies improvements in lexical diversity, sentence structure, and discourse in both participants’ narratives. (LLM, conf=0.93)
- Not checked — no full text of the sourcecites [71] McCall D, Virata T, Linebarger M, Berndt R. Integrating technology and targeted treatment to improve narrative production in aphasia: A case study. Aphasiology, 2009; 23: 438– 461. source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Data for the number of content words per turn were not normally distributed, so were analysed with separate Friedman Tests (for the immediate group and waitlist controls). There were no significant findings.”
§ Evaluating the Benefits of Aphasia Intervention Delivered in Virtual Reality: Results of a Quasi-Randomised Study > Results > Outcome measures
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The percentage of content words against all speech units was analysed through mixed and one factor ANOVAs, with no significant results.”
§ Evaluating the Benefits of Aphasia Intervention Delivered in Virtual Reality: Results of a Quasi-Randomised Study > Results > Outcome measures
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
59 more claims
Create a free account to see all 143 claims and to verify the 1 reference still open.
- Probably not a substantial claim. 35
- It may state two separate claims as one. 63
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The use of avatars and immersive settings, as shown in EVA Park (Marshall et al. 2016), highlights how VR can address aspects of motivation, and engagement that CSLT alone may not provide.”
From Exploring the Effectiveness of Technology‐Based Interventions in Aphasia Rehabilitation: A Systematic Review (2026) · cites this paper as Marshall et al. 2016
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“EVA Park is a virtual world designed specifically for PWA, offering a range of real‐world locations such as cafes, health centres and restaurants (Marshall et al. 2016).”
From Exploring the Effectiveness of Technology‐Based Interventions in Aphasia Rehabilitation: A Systematic Review (2026) · cites this paper as Marshall et al. 2016
The cited source says what the claim attributes to it.
In the citing paper
“A quasi‐RCT using EVA Park reported significant improvements in Communication Activities of Daily Living scores, maintained at follow‐up, while a waitlist group showed no change during the waiting period before receiving the intervention (Marshall et al. 2016).”
From Exploring the Effectiveness of Technology‐Based Interventions in Aphasia Rehabilitation: A Systematic Review (2026) · cites this paper as Marshall et al. 2016
The cited source says what the claim attributes to it.
In the citing paper
“Twenty patients took part in a wait-list controlled, cross-over study. Over 5 weeks, they logged an average of 41 h of engagement. The intervention produced significant gains in a functional communication measure that persisted for at least 5 weeks after the therapy finished.”
From Aphasia Recovery: When, How and Who to Treat? (2018) · cites this paper as [34]
The cited source says what the claim attributes to it.
In the citing paper
“Development of VR applications for rehabilitation of aphasia is still in its early stages ([29–32]; see for a review [17]).”
From Conversational Therapy through Semi-Immersive Virtual Reality Environments for Language Recovery and Psychological Well-Being in Post Stroke Aphasia (2020) · cites this paper as [32]
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- ConsistentF(1, 18) = 5.236reported p = .034 · recomputed p = .034
“F (1,18) = 5.236, p = .034”
- ConsistentF(2, 18) = 8.26reported p < .003 · recomputed p = .003
“F (2, 18) = 8.26, p < .003”
- ConsistentF(1, 18) = 6.54reported p = .02 · recomputed p = .020
“F (1,18) = 6.54, p = .02”
- ConsistentF(2, 18) = 6.1reported p = .009 · recomputed p = .009
“F (2,18) = 6.10, p = .009”
- ConsistentF(2, 18) = 11.28reported p = .001 · recomputed p < .001
“F (2,18) = 11.28, p = .001”
4 more recomputed tests
Create a free account to see every test the Stats check recomputed.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Self-check results appear here once an independent check has confirmed them.
Citations
Cites
Speech and language therapy for aphasia following stroke
Brady, M. C., Kelly, H., Godwin, J., Enderby, P., & Campbell, P. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000425.pub4 · Cited by 1,159
Social Networks in Adult Life and a Preliminary Examination of the Convoy Model
Antonucci, T. C., & Akiyama, H. (1987) · Journal of Gerontology · doi:10.1093/geronj/42.5.519 · Cited by 841 · cited 1×
“One further measure, the Social Network Analysis [63] was specified in our protocol.”
Method(2008) · Cited by 793
Swinburn, K., Porter, G., & Howard, D. (2012) · PsycTESTS Dataset · doi:10.1037/t13733-000 · Cited by 601 · cited 1×
“They met the following criteria: all had a diagnosis of aphasia following a stroke that occurred at least 4 months prior to the study; they were fluent users of English prior to their stroke (self report); they had no uncorrected visual impairment (self report) and no hearing loss above 40Db (screened via pure tone au…”
MethodHilari, K., Byng, S., Lamping, D. L., & Smith, S. C. (2003) · Stroke · doi:10.1161/01.str.0000081987.46660.ed · Cited by 509 · cited 1×
“Aphasia has profound consequences for a person’s quality of life [3, 4, 5].”
BackgroundSelf-citationLohse, K. R., Hilderman, C. G. E., Cheung, K. L., Tatla, S., & Van der Loos, H. F. M. (2014) · PLoS ONE · doi:10.1371/journal.pone.0093318 · Cited by 501
Hawthorne, G. (2006) · Social Indicators Research · doi:10.1007/s11205-005-7746-y · Cited by 450 · cited 5×
“Feelings of social isolation were probed by the Friendship Scale [52].”
MethodInfluentialApplication of semantic feature analysis as a treatment for aphasic dysnomia
Boyle, M., & Coelho, C. A. (1995) · American Journal of Speech-Language Pathology · doi:10.1044/1058-0360.0404.94 · Cited by 431
Klinger, E., Bouchard, S., Légeron, P., Roy, S., Lauer, F., Chemin, I., & Nugues, P. (2005) · CyberPsychology & Behavior · doi:10.1089/cpb.2005.8.76 · Cited by 327 · cited 1×
“There have been several promising attempts to treat emotional and cognitive disorders, including posttraumatic stress disorder [29], social anxiety [30, 31] anxiety in autism [32] and paediatric disorders of attention [33].”
BackgroundCommunication Partner Training in Aphasia: A Systematic Review
Simmons-Mackie, N., Raymer, A., Armstrong, E., Holland, A., & Cherney, L. R. (2010) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2010.08.026 · Cited by 325 · cited 1×
“However, improving conversation in aphasia is difficult, even following conventional face to face therapy [68]; and when there is success in this area it typically follows the training of conversational partners [69].”
Background
Research works printed but not linked: 1
- Berndt R, Wayland S, Rochon E, Saffran E, Schwartz M. Quantitative Production Analysis (QPA) Hove: Psychology Press 2000.
Cited by
Virtual reality for stroke rehabilitation
Laver, K. E., George, S., Thomas, S., Deutsch, J. E., & Crotty, M. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008349.pub3 · Cited by 1,131
Virtual reality for stroke rehabilitation
Laver, K. E., Lange, B., George, S., Deutsch, J. E., Saposnik, G., & Crotty, M. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008349.pub4 · Cited by 800
A meta-analysis of virtual reality training programs for social skill development
Howard, M. C., & Gutworth, M. B. (2020) · Computers & Education · doi:10.1016/j.compedu.2019.103707 · Cited by 234
Virtual reality tourism experiences: Addiction and isolation
Merkx, C., & Nawijn, J. (2021) · Tourism Management · doi:10.1016/j.tourman.2021.104394 · Cited by 166
Bryant, L., Brunner, M., & Hemsley, B. (2020) · Disability and Rehabilitation: Assistive Technology · doi:10.1080/17483107.2018.1549276 · Cited by 131
“However, the intervention had minimal effect on specific language outcomes including verbal fluency, word finding, narrative production, communication confidence, and feelings of social isolation [27].”
BackgroundDiagnosing and managing post-stroke aphasia
Sheppard, S. M., & Sebastian, R. (2021) · Expert Review of Neurotherapeutics · doi:10.1080/14737175.2020.1855976 · Cited by 126
Aphasia Recovery: When, How and Who to Treat?
Doogan, C., Dignam, J., Copland, D., & Leff, A. (2018) · Current Neurology and Neuroscience Reports · doi:10.1007/s11910-018-0891-x · Cited by 110
Virtual Reality for the Rehabilitation of Acquired Cognitive Disorders: A Narrative Review
Catania, V., Rundo, F., Panerai, S., & Ferri, R. (2023) · Bioengineering · doi:10.3390/bioengineering11010035 · Cited by 109
“After a 5-week training program, significant improvements in functional communication were noted; there was excellent compliance with the intervention, with no participant lost to follow-up [90].”
BackgroundTelerehabilitation for people with aphasia: A systematic review and meta-analysis
Cacciante, L., Kiper, P., Garzon, M., Baldan, F., Federico, S., Turolla, A., & Agostini, M. (2021) · Journal of Communication Disorders · doi:10.1016/j.jcomdis.2021.106111 · Cited by 81
Maresca, G., Maggio, M. G., Latella, D., Cannavò, A., De Cola, M. C., Portaro, S., Stagnitti, M. C., Silvestri, G., Torrisi, M., Bramanti, A., et al. (2019) · Journal of Stroke and Cerebrovascular Diseases · doi:10.1016/j.jstrokecerebrovasdis.2019.104303 · Cited by 67
Delivering word retrieval therapies for people with aphasia in a virtual communication environment
Marshall, J., Devane, N., Edmonds, L., Talbot, R., Wilson, S., Woolf, C., & Zwart, N. (2018) · Aphasiology · doi:10.1080/02687038.2018.1488237 · Cited by 66
“Outcome measures showed significant benefits for functional communication (Marshall et al., 2016) and interviews revealed very positive user views about the intervention (Amaya et al., 2018).”
Self-citationMarshall, J., Devane, N., Talbot, R., Caute, A., Cruice, M., Hilari, K., MacKenzie, G., Maguire, K., Patel, A., Roper, A., et al. (2020) · PLOS ONE · doi:10.1371/journal.pone.0239715 · Cited by 65
“included one group session per week [39].”
BackgroundResultSelf-citationInnovative technology-based interventions in aphasia rehabilitation: a systematic review
Repetto, C., Paolillo, M. P., Tuena, C., Bellinzona, F., & Riva, G. (2021) · Aphasiology · doi:10.1080/02687038.2020.1819957 · Cited by 63
“Three studies employed VR (Grechuta et al., 2017, 2019; Marshall et al., 2016).”
BackgroundMethodResultInfluentialGiachero, A., Calati, M., Pia, L., La Vista, L., Molo, M., Rugiero, C., Fornaro, C., & Marangolo, P. (2020) · Behavioural Neurology · doi:10.1155/2020/2846046 · Cited by 58
“As reported in Introduction, in the past, several VR systems have been developed for cognitive rehabilitation; some of which have only gone through studies with a small number of participants [43, 47, 48] and/or without control groups [32].”
BackgroundInfluentialDose effects in behavioural treatment of post-stroke aphasia: a systematic review and meta-analysis
Harvey, S., Carragher, M., Dickey, M. W., Pierce, J. E., & Rose, M. L. (2020) · Disability and Rehabilitation · doi:10.1080/09638288.2020.1843079 · Cited by 50
“The included studies comprise four RCTs [42–45], one quasi-RCT in which participants were sequentially allocated to cohorts that had been randomly assigned to different treatment arms [46], and three quasi-experimental “AB with follow up” single-case designs [31,32,47].”
BackgroundTreatment dose in post-stroke aphasia: A systematic scoping review
Harvey, S. R., Carragher, M., Dickey, M. W., Pierce, J. E., & Rose, M. L. (2021) · Neuropsychological Rehabilitation · doi:10.1080/09602011.2020.1786412 · Cited by 47
The use of virtual reality in the rehabilitation of aphasia: a systematic review
Devane, N., Behn, N., Marshall, J., Ramachandran, A., Wilson, S., & Hilari, K. (2023) · Disability and Rehabilitation · doi:10.1080/09638288.2022.2138573 · Cited by 43
“In two studies, the primary outcome was communication activity [46,74], and one aimed to improve well-being [51].”
BackgroundMethodInfluentialSelf-citationExperiencing EVA Park, a Multi-User Virtual World for People with Aphasia
Galliers, J., Wilson, S., Marshall, J., Talbot, R., Devane, N., Booth, T., Woolf, C., & Greenwood, H. (2017) · ACM Transactions on Accessible Computing · doi:10.1145/3134227 · Cited by 40
“However, many approaches are digital versions of traditional speech therapy tasks (see Reference [60] for a review).”
MethodInfluentialSelf-citationAdjunctive Approaches to Aphasia Rehabilitation: A Review on Efficacy and Safety
Picano, C., Quadrini, A., Pisano, F., & Marangolo, P. (2021) · Brain Sciences · doi:10.3390/brainsci11010041 · Cited by 47
“, 2016 [113] quasi-RTC 20 chronic Free communication VR (EVA Park) Twenty-five one-hour therapy session over five weeks.”
BackgroundJesus, L. M. T., Martinez, J., Santos, J., Hall, A., & Joffe, V. (2019) · Journal of Speech, Language, and Hearing Research · doi:10.1044/2019_jslhr-s-18-0301 · Cited by 39
“…and enhance speech and language therapy, in both assessment and intervention, in populations as diverse as those with aphasia (Abad et al., 2013; Marshall et al., 2016), autism (Khowaja & Salim, 2013; Ramdoss et al., 2011), cerebral palsy (Hawley et al., 2013; Kanitkar et al., 2017), Down…”
BackgroundBu, X., Ng, P. H. F., Tong, Y., Chen, P. Q., Fan, R., Tang, Q., Cheng, Q., Li, S., Cheng, A. S. K., & Liu, X. (2022) · JMIR Serious Games · doi:10.2196/30196 · Cited by 48
“It is designed to enable patients with aphasia to communicate successfully with 1 or more conversation partners via speech using a headset and microphone in real time [13].”
BackgroundVaezipour, A., Aldridge, D., Koenig, S. T., Theodoros, D., & Russell, T. G. (2021) · Disability and Rehabilitation · doi:10.1080/09638288.2021.1895333 · Cited by 43
“Various small-scale studies have also revealed that communication interventions delivered via VR platforms were useful and able to effect positive changes in communication performance for individuals with aphasia [36,69].”
BackgroundNorthcott, S., Thomas, S., James, K., Simpson, A., Hirani, S. P., Barnard, R., & Hilari, K. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-050308 · Cited by 41
Self-citationRemacle, A., Bouchard, S., Étienne, A.-M., Rivard, M., & Morsomme, D. (2021) · Virtual Reality · doi:10.1007/s10055-020-00491-1 · Cited by 44
Receiving aphasia intervention in a virtual environment: the participants’ perspective
Amaya, A., Woolf, C., Devane, N., Galliers, J., Talbot, R., Wilson, S., & Marshall, J. (2018) · Aphasiology · doi:10.1080/02687038.2018.1431831 · Cited by 36
“However, feelings of social isolation were measured by the Friendship Scale (Hawthorne, 2006), with no significant findings (Marshall et al., 2016).”
BackgroundMethodInfluentialSelf-citationNichol, L., Wallace, S. J., Pitt, R., Rodriguez, A. D., Diong, Z. Z., & Hill, A. J. (2021) · Disability and Rehabilitation · doi:10.1080/09638288.2021.1989501 · Cited by 36
Treatment fidelity in aphasia randomised controlled trials
Brogan, E., Ciccone, N., & Godecke, E. (2019) · Aphasiology · doi:10.1080/02687038.2019.1576442 · Cited by 32
Telehealth Assessments and Interventions for Individuals With Poststroke Aphasia: A Scoping Review
Teti, S. D., Murray, L. L., Orange, J. B., Page, A. D., & Kankam, K. S. (2023) · American Journal of Speech-Language Pathology · doi:10.1044/2023_ajslp-22-00324 · Cited by 38
Effects of virtual reality in post-stroke aphasia: a systematic review and meta-analysis
Cao, Y., Huang, X., Zhang, B., Kranz, G. S., Zhang, D., Li, X., & Chang, J. (2021) · Neurological Sciences · doi:10.1007/s10072-021-05202-5 · Cited by 38
Bryant, L., Sedlarevic, N., Stubbs, P. W., Bailey, B., Nguyen, V. A., Bluff, A., Barnett, D., Estela, M., Hayes, C., Jacobs, C., et al. (2022) · Disability and Rehabilitation Assistive Technology · doi:10.1080/17483107.2022.2063423 · Cited by 36
“Marshall and colleagues [19] reported improved functional, everyday communication in users completing therapy using the non-immersive VR program EVA Park [20].”
BackgroundRogalski, E. J., Roberts, A., Salley, E., Saxon, M., Fought, A. J., Esparza, M., Blaze, E., Coventry, C., Mesulam, M. M., Weıntraub, S., et al. (2021) · The Journals of Gerontology Series B · doi:10.1093/geronb/gbab165 · Cited by 25
Telerehabilitation of aphasia: A systematic review of the literature
Cetinkaya, B., Twomey, K. E., Bullard, B., Kouaissi, S. E., & Conroy, P. (2023) · Aphasiology · doi:10.1080/02687038.2023.2274621 · Cited by 31
“Marshall et al. (2016) treated 20 individuals with aphasia for five weeks using the virtual reality platform EVA Park.”
BackgroundMethodInfluentialAI-assisted assessment and treatment of aphasia: a review
Zhong, X. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1401240 · Cited by 25
From individual to global: Human rights and aphasia
Hersh, D. (2017) · International Journal of Speech-Language Pathology · doi:10.1080/17549507.2018.1397749 · Cited by 21
Adapting therapy for a new world: storytelling therapy in EVA Park
Carragher, M., Steel, G., Talbot, R., Devane, N., Rose, M. L., & Marshall, J. (2020) · Aphasiology · doi:10.1080/02687038.2020.1812249 · Cited by 21
“Studies of intervention within EVA Park suggest this method of delivery is acceptable to participants (Amaya et al., 2018; Marshall et al., 2016; Marshall et al., 2018) and effective in improving functional communication (Marshall et al., 2016) and noun retrieval (Marshall et al., 2018).”
BackgroundMethodSelf-citationDelivering storytelling intervention in the virtual world of EVA Park
Carragher, M., Talbot, R., Devane, N., Rose, M., & Marshall, J. (2018) · Aphasiology · doi:10.1080/02687038.2018.1484880 · Cited by 19
“This study explored the remote delivery of storytelling intervention in EVA Park, a virtual world designed to create interactional and therapeutic opportunities for PWA (Marshall et al., 2016).”
BackgroundSelf-citationPaterson, H. L. (2017) · Augmentative and Alternative Communication · doi:10.1080/07434618.2016.1275789 · Cited by 21
“Marshall et al. (2016) found that an intervention program delivered remotely with self-directed computer practice was accessible and acceptable to participants with aphasia, with high levels of compliance and improvements in communication.”
BackgroundInfluentialBehn, N., Harrison, M., Brady, M., Breitenstein, C., Carragher, M., Fridriksson, J., Godecke, E., Hillis, A. E., Kelly, H., Palmer, R., et al. (2022) · Aphasiology · doi:10.1080/02687038.2022.2037502 · Cited by 21
Self-citationBrogan, E., Kim, J., Grimley, R., Wallace, S. J., Baker, C., Thayabaranathan, T., Andrew, N. E., Kilkenny, M. F., Godecke, E., Rose, M. L., et al. (2023) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2023.01.015 · Cited by 19
Carragher, M., Ryan, B., Worrall, L. E., Thomas, S., Rose, M. L., Simmons‐Mackie, N. N., Khan, A., Hoffmann, T., Power, E., Togher, L., et al. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-023560 · Cited by 20
“reported on treatment fidelity processes and one article addressed all receommended treatment fidelity components.(17)”
BackgroundSelf-reported feedback in ICT-delivered aphasia rehabilitation: a literature review
Kearns, Á., Kelly, H., & Pitt, I. J. (2019) · Disability and Rehabilitation · doi:10.1080/09638288.2019.1655803 · Cited by 20
“[41] identified that neither age or gender were found to influence the amount of time spent logged into the virtual world in their study.”
BackgroundInfluentialOpportunities for the Implementation of Immersive Virtual Reality in Rehabilitation
Bryant, L., Hemsley, B., Bailey, B., Bluff, A., Nguyen, V. A., Stubbs, P. W., Barnett, D., Jacobs, C., Lucas, C., & Power, E. (2020) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2020.437 · Cited by 17
Reversing the Ruin: Rehabilitation, Recovery, and Restoration After Stroke
Stockbridge, M. D., Bunker, L. D., & Hillis, A. E. (2022) · Current Neurology and Neuroscience Reports · doi:10.1007/s11910-022-01231-5 · Cited by 21
Brassel, S., Brunner, M., Power, E., Campbell, A., & Togher, L. (2022) · American Journal of Speech-Language Pathology · doi:10.1044/2022_ajslp-22-00077 · Cited by 18
Clunne, S. J., Ryan, B., Hill, A. J., Brandenburg, C., & Kneebone, I. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.9864 · Cited by 18
“However, there are numerous examples of people with poststroke aphasia successfully partaking in participatory design studies to enhance the usability of aphasia-specific programs [19,78-82].”
BackgroundKearns, Á., & Kelly, H. (2022) · Aphasiology · doi:10.1080/02687038.2022.2030462 · Cited by 19
“People with aphasia who have higher levels of prior computer experience have been noted to spend more time working with an ICT-delivered rehabilitation programme compared to those with lower experience (Marshall et al., 2016).”
BackgroundGeorgiou, A. M., & Kambanaros, M. (2023) · Medicina · doi:10.3390/medicina59091674 · Cited by 25
“The “EVA PARK” technology is considered accessible, acceptable, and engaging to PWA, and has the potential to drive language gains [61].”
BackgroundCaute, A., Cruice, M. N., Devane, N., Patel, A. K., Roper, A., Talbot, R., Wilson, S., & Marshall, J. (2021) · Disability and Rehabilitation · doi:10.1080/09638288.2021.2011436 · Cited by 18
Self-citationVaezipour, A., Aldridge, D., Koenig, S. T., Burns, C. L., Baghaei, N., Theodoros, D., & Russell, T. G. (2023) · JMIR Rehabilitation and Assistive Technologies · doi:10.2196/46959 · Cited by 19
“Other studies delivering intervention via VR platforms have also shown improved communication outcomes for this group measured by formal functional communication assessment and participants’ reports of maintained communicative confidence up to 1-year posttreatment [24,53,54].”
BackgroundLee, S. A. S. (2019) · Current Developmental Disorders Reports · doi:10.1007/s40474-019-00169-7 · Cited by 19
De Luca, R., Leonardi, S., Maresca, G., Marilena, F. C., Latella, D., Impellizzeri, F., Maggio, M. G., Naro, A., & Calabrò, R. S. (2023) · Aphasiology · doi:10.1080/02687038.2021.1998882 · Cited by 12
“…therapies) for the rehabilitation of patients with post-stroke aphasia (Basso et al., 2013; De Luca, Aragona et al., 2018; Maresca et al., 2019; Marshall et al., 2016; Peach, 2017; Van Vuuren & CONTACT Rocco Salvatore Calabrò salbro77@tiscali.it IRCCS Centro Neurolesi Bonino Pulejo; Via…”
BackgroundQuality of life outcomes from aphasia telepractice: A scoping review
Gauch, M., Leinweber, J., Plath, A., Spelter, B., & Corsten, S. (2022) · Aphasiology · doi:10.1080/02687038.2022.2079604 · Cited by 15
“The RCT of Marshall et al. (2016) met 8 of 11 criteria.”
BackgroundInfluentialAccessibility Feature Implementation Within Free VR Experiences
Naikar, V. H., Subramanian, S., & Tigwell, G. W. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3650935 · Cited by 21
“Our analysis took place May to July 2023.”
Investigating the Impact of Visual Input on Voice Production in Virtual Reality
Nudelman, C. J., & Bottalico, P. (2023) · Journal of Voice · doi:10.1016/j.jvoice.2023.07.016 · Cited by 16
Computerbasiertes Hörtraining in der Hörrehabilitation Erwachsener nach Cochleaimplantation
Völter, C., Schirmer, C., Stöckmann, C., et al. (2020) · HNO · doi:10.1007/s00106-020-00898-x · Cited by 11
Nichol, L., Wallace, S. J., Pitt, R., Rodriguez, A. D., & Hill, A. J. (2021) · Disability and Rehabilitation · doi:10.1080/09638288.2021.1988153 · Cited by 13
App-based data collection, mental imagery, and naming performance in adults with aphasia
Laures-Gore, J., Stache, M., Moore, E., & Tullis, C. (2021) · Complementary Therapies in Clinical Practice · doi:10.1016/j.ctcp.2021.101422 · Cited by 11
Behavioral interventions for poststroke aphasia
Cherney, L. R., & Carpenter, J. (2022) · Handbook of clinical neurology · doi:10.1016/b978-0-12-823384-9.00010-4 · Cited by 11
Ramos, V. M., Vásquez-Correa, J. C., Cremers, R., den Steen, L. V., Nöth, E., Bodt, M. S. D., & Nuffelen, G. V. (2021) · International Journal of Language & Communication Disorders · doi:10.1111/1460-6984.12647 · Cited by 12
“…(e.g., Abad et al. 2013, Thompson et al. 2010, Van Vuuren and Cherney 2014, Cherney et al. 2008, Lee et al. 2009, Kalinyak-Fliszar et al. 2015, Marshall et al. 2016, Amaya et al. 2018) and children with hearing impairment (Chaisanit et al. 2010, Eriksson et al. 2005, Massaro and Light 2004,…”
BackgroundElhakeem, E. S., Saeed, S. S. G. M., Elsalakawy, R. N. A.-E., Elmaghraby, R. M., & Ashmawy, G. A. H. O. (2021) · The Egyptian Journal of Otolaryngology · doi:10.1186/s43163-021-00144-3 · Cited by 8
Franco, J., Glize, B., & Laganaro, M. (2025) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2025.109069 · Cited by 12
“It therefore leads to higher motivation and interest (Marshall et al., 2016).”
Hickin, J., Cruice, M. N., & Dipper, L. T. (2022) · Aphasiology · doi:10.1080/02687038.2022.2116928 · Cited by 10
“Examples include the use of virtual realities to improve the functionality of treatment by simulating real-life communication (e.g., the virtual world of Eva Park investigated by Carragher et al., 2021 and Marshall et al., 2016).”
BackgroundEricson, C., Latysheva, A., Poirier, S.-È., & Fossard, M. (2025) · Brain Sciences · doi:10.3390/brainsci15020122 · Cited by 7
“The emerging use of virtual reality, with immersive or semi-immersive environments that simulate real-life contexts, enhances the ecological validity of language rehabilitation [87,88].”
Langford, T., Fleming, V., Upton, E., Doogan, C., Leff, A. P., & Romano, D. M. (2022) · JMIR Neurotechnology · doi:10.2196/39855 · Cited by 9
“It was found that users responded positively to the novelty of the co-designed therapy, as evaluated in terms of both a zero percent therapy dropout rate [24] and high acceptability deduced from qualitative interviews [27].”
BackgroundMethodKhoza‐Shangase, K. (2025) · Journal of Speech Language and Hearing Research · doi:10.1044/2025_jslhr-24-00895 · Cited by 9
Delivering Script Therapy for people with aphasia in EVA Park: Two single case treatment studies
Marshall, J., Devane, N., Berraondo, J., Talbot, R., Temponera, P., Clegg, K., & Wilson, S. (2023) · Advances in Communication and Swallowing · doi:10.3233/acs-220014 · Cited by 7
“EVA Park is the only multi-user, online VR world that was developed with and for people with aphasia (Wilson et al., 2015; Marshall et al., 2016).”
Self-citationThe Relationship of Virtual Reality and Interpersonal Communication: Bibliometric Overview
Hadi, H. K., Irawan, M. I., & Nadlifatin, R. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.03.121 · Cited by 7
An automated speech-language therapy tool with interactive virtual agent and peer-to-peer feedback
Das, M., & Saha, A. (2017) · International Conference on Advances in Electrical Engineering · doi:10.1109/icaee.2017.8255409 · Cited by 7
“evaluated the effects of a virtual reality platform on people with aphasia in [32].”
BackgroundRehabilitation of Communication Disorders
Palmer, R., & Pauranik, A. (2021) · Springer eBooks · doi:10.1007/978-3-030-58505-1_10 · Cited by 9
Vaezipour, A., Aldridge, D., Wall, K. J., Koenig, S. T., & Theodoros, D. (2019) · IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) · doi:10.1109/ismar-adjunct.2019.00043 · Cited by 6
“Non-immersive platforms have been used to provide a virtual world environment (EVA Park) [22], and virtual speech pathologists to provide opportunities for people with aphasia to practise scripted dialogues [23, 24].”
BackgroundEmbracing virtual reality in rehabilitation of post-stroke aphasia
Kljajević, V. (2024) · Digital Health · doi:10.1177/20552076241271823 · Cited by 9
“47,52 The results further suggest that using EVA Park considerably improved PWA ’ s functional communication, but not their communicative con fi dence or feeling of social isolation.”
BackgroundResultCuperus, P., de Kok, D., de Aguiar, V., & Nickels, L. (2024) · Aphasiology · doi:10.1080/02687038.2024.2384542 · Cited by 4
Is Aphasia Treatment Beneficial for the Elderly? A Review of Recent Evidence
Fabian, R., Bunker, L. D., & Hillis, A. E. (2020) · Current Physical Medicine and Rehabilitation Reports · doi:10.1007/s40141-020-00287-z · Cited by 6
Stark, B. C. (2025) · Perspectives of the ASHA Special Interest Groups · doi:10.1044/2025_persp-25-00035 · Cited by 5
Anchuri, K., Steiner, L., Rabet, R., Craig-Neil, A., Antonio, E. S., Ogundele, O. J., Seabrook, M. S. S., Pope, C., Dai, S., Schuler, A., et al. (2024) · BJGP Open · doi:10.3399/bjgpo.2023.0119 · Cited by 5
Devane, N., Marshall, J., Wilson, S., & Hilari, K. (2025) · Disability and Rehabilitation · doi:10.1080/09638288.2025.2470401 · Cited by 7
“There is now a large body of evidence for positive psychology interventions, with six meta-analyses published over the last 10 years (see [43] for a summary).”
InfluentialSelf-citation“Difficult but Good”: enjoying accessible digital creativity
Roper, A., Barnard, R., Cruice, M. N., Marshall, J., Mulholland, A., Neate, T., & Wilson, S. (2023) · Aphasiology · doi:10.1080/02687038.2022.2160624 · Cited by 5
Self-citationEvaluating the intervention fidelity of self-managed computer therapy for aphasia post-stroke
Harrison, M. (2019) · White Rose eTheses Online (University of Leeds, The University of Sheffield, University of York) · Cited by 4
Shenoy, R., Intiaz, C., Tiwari, S., & Krishnan, G. (2024) · Technology and Disability · doi:10.3233/tad-230008 · Cited by 5
Talbot, R., Farrington-Douglas, C., Mummery, C. J., Warren, J. D., & Volkmer, A. (2025) · Aphasiology · doi:10.1080/02687038.2025.2495618 · Cited by 2
“The CCRSA has been used as an outcome measure in PPA intervention studies (Rogalski et al., 2022; Volkmer, Walton, et al., 2023) and in stroke aphasia telehealth intervention studies (Marshall et al., 2016; Steele et al., 2015).”
Self-citationConnections that Care: WhatsApp as a Support Space for Stroke Survivors
Rodrigues, M., Silva, I. D. S., & Soares, L. (2025) · Advances in Geriatric Medicine and Research · doi:10.20900/agmr20250020 · Cited by 4
Repetto, C., Scerrati, E., Amighetti, G., & Cancer, A. (2026) · Aphasiology · doi:10.1080/02687038.2026.2627314 · Cited by 2
iReadMore: A Reading Therapy App Co-Designed by People with Aphasia and Alexia
Langford, T., Leff, A. P., & Romano, D. M. (2021) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3441852.3476518 · Cited by 3
“Co-design has been applied to a growing number of aphasia therapies in recent years [8, 9, 11, 16].”
BackgroundA Study on Addiction and Isolation in Virtual Reality Tourism Experiences
Kamal, R., Vijaykumar, & Retheesh, P. T. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-031-08954-1_32 · Cited by 2
Yu, M., Zhu, L., Chen, W., Liu, J., Liu, Z., & Song, A. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789090 · Cited by 2
“Speech therapy applications typically focus on acoustic analysis [9], conversational platforms emphasize social interaction without systematic performance measurement [29], and reading-focused tools assess word recognition rather than production mechanics.”
Telehealth use in Speech-Language Pathology: an exploratory scoping review
Krikheli, L., Carey, L. B., McDonald, C. E., & Abdu, N. M. (2023) · Figshare · doi:10.26181/22240210 · Cited by 2
Roper, A. (2019) · Sage eBooks · doi:10.4135/9781483380810.n150 · Cited by 2
Kodani, Y., Nagami, S., Yokozeki, A., Fukunaga, S., Nakamura, K., & Nakamura, H. (2025) · PLOS ONE · doi:10.1371/journal.pone.0319805 · Cited by 1
“Training studies targeted language function (n = 8) [32–34,36, 39– 42], The Web-based dual card game [36], Rehabilitation Gaming System for aphasia [39,41], and NeuroVR 2.0 [40] were used for training.”
InfluentialVR-Based Rehabilitation in a Virtual Home: A Perspective on Cognitive and Motor Recovery
Perrucci, T., Celino, M., D’Errico, G., Pellegrino, G., Barba, M. C., & Paolis, L. T. D. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-97778-7_34 · Cited by 2
Greiner, N., Lauer, N., Schedel, V., & Pfingsten, A. (2025) · PM&R · doi:10.1002/pmrj.70006 · Cited by 1
“87 In a crossover RCT 67 on non-immersive VR-based conversational group therapy, significant gains in functional communication were demonstrated, both in the group with immediate and deferred treat-ment.”
InfluentialAlhejji, B., Cunningham, S., Dorney, S., & Palmer, R. (2026) · International Journal of Language & Communication Disorders · doi:10.1111/1460-6984.70322 · Cited by 1
“A quasi-RCT using EVA Park reported significant improvements in Communication Activities of Daily Living scores, maintained at follow-up, while a waitlist group showed no change during the waiting period before receiving the intervention (Marshall et al. 2016 ).”
(2023) · doi:10.5121/csit.2023.131408 · Cited by 1
“defprepare_corpus(doc_clean): """ Input : clean document Purpose: create term dictionary of our courpus and Converting list of documents (corpus) into Document Term Matrix [8, 9] Output : term dictionary and Document Term Matrix """ # Creating the term dictionary of our courpus, where every unique term is assigned an…”
BackgroundInfluentialKekes-Szabo, S., Clough, S., Brown‐Schmidt, S., & Duff, M. C. (2025) · American Journal of Speech-Language Pathology · doi:10.1044/2025_ajslp-24-00151 · Cited by 1
“VR is utilized for aphasia rehabilitation (EVA Park; Marshall et al., 2016) and acquired brain injury rehabilitation (Brassel et al., 2021), and can be used as a research and clinical tool for communication disorders and psycholinguistics (Bryant et al., 2020; Peeters, 2019).”
Holmes, D. (2019) · Ulster University Research Portal (Ulster University) · Cited by 1
An Exploratory Study on Supporting Persons with Aphasia in Pakistan: Challenges and Opportunities
Riaz, W., Ali, G., Abid, M., Butt, I. N., Shahzad, A., & Shahid, S. (2020) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3373625.3418024 · Cited by 3
Sims, K. M., & Buehler, C. (2021) · The Mathematics Enthusiast
Kim, N. Y., Yu, J., Yoon, S.-Y., Park, K., Min, Y.-S., Yoon, J., Bae, Y.-S., Yang, Y.-M., Jung, S.-J., & Ha, J.-W. (2023) · Eon'eo cheong'gag jang'ae yeon'gu/Communication sciences & disorders · doi:10.12963/csd.23968
“대상은 실어증 (Marshall et al., 2016), 뇌성마비 (Kanitkar et al., 2017), 자폐스펙트럼장애 (Hair, Monroe, Ahmed, Ballard, & Osuna, 2018; Khowaja & Salim, 2013), 말소리장애 (Chen et al., 2016) Littlejohns, & Thompson, 1992; Brindley, Cope-land, Demain, & Martyn, 1989).”
VIRTUAL REALITY IN SPEECH SOUND DISORDERS THERAPY
(2023) · Education and new developments · doi:10.36315/2023v2end106
“Therapists used Eva Park and Second Life to provide the therapy setting for the speech language intervention sessions (Marshall et al., 2016; Stendal, Balandin, Molka-Danielsen, 2011).”
MethodHübner, L. C., Carthery‐Goulart, M. T., & Rodrigues, E. D. S. (2023) · Springer eBooks · doi:10.1007/978-3-031-45190-4_25 · Cited by 1
Advances in Neuropsychology and Speech and Language Pathology in Patients with Stroke
Gobbo, M., & Marini, A. (2024) · Springer eBooks · doi:10.1007/978-3-031-63604-2_4
Metadata
- Authors
- Marshall, J., Booth, T., Devane, N., Galliers, J., Greenwood, H., Hilari, K., Talbot, R., Wilson, S., & Woolf, C.
- Affiliations
- City, University of London (GB)
- Corresponding author
- Marshall, Jane
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2016-08-12
- Volume
- 11
- Issue
- 8
- Pages
- e0160381
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4982664
- OpenAlex
- W2518102671
- Cited by
- 127
- References
- 82
- Citations by year
- 2026: 15 · 2025: 16 · 2024: 11 · 2023: 15 · 2022: 13 · 2021: 19 · 2020: 12 · 2019: 13 · 2018: 7 · 2017: 5
- FWCI
- 19.84
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- City Research Online (City University London) (acceptedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- UCL Discovery (University College London) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- Funders
- The Stroke Association (TSA 2011/2012)
- Topics
- Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
- Health Sciences › Medicine › Rehabilitation › Stroke Rehabilitation and Recovery
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
- Keywords
- aphasia, virtual reality, language stimulation, communication skills, communication confidence, social isolation, intervention adherence, aphasia rehabilitation
- MeSH terms
- Telerehabilitation, Aphasia, Communication, Female, Humans, Language Therapy, Male, Middle Aged, Speech Therapy, Case-Control Studies, Outcome Assessment, Health Care, Early Intervention, Educational, Virtual Reality Exposure Therapy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
