The goal of explaining black boxes in EEG seizure prediction is not to explain models' decisions
Open accessAbstract
Many state‐of‐the‐art methods for seizure prediction, using the electroencephalogram, are based on machine learning models that are black boxes, weakening the trust of clinicians in them for high‐risk decisions. Seizure prediction concerns a multidimensional time‐series problem that performs continuous sliding window analysis and classification. In this work, we make a critical review of which explanations increase trust in models' decisions for predicting seizures. We developed three machine learning methodologies to explore their explainability potential. These contain different levels of model transparency: a logistic regression, an ensemble of 15 support vector machines, and an ensemble of three convolutional neural networks. For each methodology, we evaluated quasi‐prospectively the performance in 40 patients (testing data comprised 2055 hours and 104 seizures). We selected patients with good and poor performance to explain the models' decisions. Then, with grounded theory, we evaluated how these explanations helped specialists (data scientists and clinicians working in epilepsy) to understand the obtained model dynamics. We obtained four lessons for better communication between data scientists and clinicians. We found that the goal of explainability is not to explain the system's decisions but to improve the system itself. Model transparency is not the most significant factor in explaining a model decision for seizure prediction. Even when using intuitive and state‐of‐the‐art features, it is hard to understand brain dynamics and their relationship with the developed models. We achieve an increase in understanding by developing, in parallel, several systems that explicitly deal with signal dynamics changes that help develop a complete problem formulation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Five studies did not provide sufficient detail on their qualitative data analysis process, making it difficult to determine the validity of the findings.
“Five studies did not provide sufficient detail on their qualitative data analysis process, and this made it difficult to determine the validity of the findings [31,32,34,37,38].”
From Explainable AI in hospital clinical decision support systems: A scoping review of healthcare professionals’ perspectives (2026) · cites this paper as [34]
In Materials and Methods, under “Evaluating explanations” (page unavailable), the paper says interviews were transcribed and analyzed using grounded theory, but gives only a general description of iterative coding and theme identification. This supports the concern about limited analytic detail; the paper does not itself say that the detail was insufficient to assess the validity of its findings. The other four studies cited in the sentence are outside this source. (LLM, conf=0.76)
Experimental — accuracy not yet measuredSimple, interpretable visualizations, such as bar charts and line graphs, were preferred for understanding predictions at a glance.
“Simple, interpretable visualizations, such as bar charts and line graphs, were preferred for understanding predictions at a glance.”
From Explainable AI in hospital clinical decision support systems: A scoping review of healthcare professionals’ perspectives (2026) · cites this paper as [34]
The Results section, “Time plots are the most intuitive explanations” (page not available), describes time plots as intuitive and useful for examining seizure-prediction behavior. It does not report a preference for bar charts or say that the visualizations enabled predictions to be understood at a glance. (LLM, conf=0.85)
Experimental — accuracy not yet measuredFour studies used more than one explanation technique in their model.
“Four studies used more than one explanation technique in their model [24,32,34,38].”
From Explainable AI in hospital clinical decision support systems: A scoping review of healthcare professionals’ perspectives (2026) · cites this paper as [34]
Several studies lacked detail on the recruitment of participants.
“Several studies lacked detail on the recruitment of participants [32,34,37,38].”
From Explainable AI in hospital clinical decision support systems: A scoping review of healthcare professionals’ perspectives (2026) · cites this paper as [34]
Perceptions on the model's performance were reported in 13 studies.
“Perceptions on the model’s performance was reported in 13 studies [23–26,29–37].”
From Explainable AI in hospital clinical decision support systems: A scoping review of healthcare professionals’ perspectives (2026) · cites this paper as [34]
XAI model predictions were reportedly more likely to be accepted when they aligned with health professionals' knowledge or with the patient context.
“XAI model predictions were reportedly more likely to be accepted when they aligned with health professionals’ knowledge, or with the patient context [23,25,27–30,32–36]”
From Explainable AI in hospital clinical decision support systems: A scoping review of healthcare professionals’ perspectives (2026) · cites this paper as [34]
Five studies specifically described using SHAP.
“Five studies [23,25,27,28,34] specifically described using SHapley Additive exPlanations (SHAP)”
From Explainable AI in hospital clinical decision support systems: A scoping review of healthcare professionals’ perspectives (2026) · cites this paper as [34]
Of the four studies that implemented multiple explanation techniques, only one directly examined participants' perceptions of the different methods.
“Of the four studies that implemented multiple explanation techniques within their models [24,32,34,38], only one directly examined participants’ perceptions of the different methods [34].”
From Explainable AI in hospital clinical decision support systems: A scoping review of healthcare professionals’ perspectives (2026) · cites this paper as [34]
The study that examined multiple explanation techniques was unable to determine a preferred explanation approach.
“The study was unable to determine a preferred explanation approach, likely because of the clinical context where there was a limited understanding of the electroencephalogram (EEG) mechanisms that occur prior to a seizure.”
From Explainable AI in hospital clinical decision support systems: A scoping review of healthcare professionals’ perspectives (2026) · cites this paper as [34]
LIME were used by models in three studies.
“Local Interpretable Model-Agnostic Explanations (LIME) were also used by models in three studies [33,34,37]”
From Explainable AI in hospital clinical decision support systems: A scoping review of healthcare professionals’ perspectives (2026) · cites this paper as [34]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vasileiou, K., Barnett, J., Thorpe, S., & Young, T. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0594-7 · Cited by 3,115 · see citing works
European Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
Goodman, B., & Flaxman, S. (2017) · AI Magazine · doi:10.1609/aimag.v38i3.2741 · Cited by 2,131 · cited 1× · see citing works
“39 Explaining seizure prediction models led us to consider each case scenario individually.”
BackgroundSeizure prediction – ready for a new era
Kuhlmann, L., Lehnertz, K., Richardson, M. P., Schelter, B., & Zaveri, H. P. (2018) · Nature Reviews Neurology · doi:10.1038/s41582-018-0055-2 · Cited by 450 · see citing works
Flagged first, then by citations
Research works printed but not linked: 39
- 10.1212/WNL.0000000000002707 · DOI
- 10.1136/jnnp-2016-313831 · DOI
- 10.1212/WNL.0000000000003509 · DOI
- 10.1016/j.seizure.2016.10.002 · DOI
- 10.1093/brain/awl241 · DOI
- 10.1097/WCO.0000000000000429 · DOI · Paper page
- 10.1016/j.jneumeth.2015.06.010 · DOI
- Doshi‐VelezF KimB.Towards a rigorous science of interpretable machine learning. arXiv Preprint arXiv:170208608.2017.
- GleavesLP SchwartzR BroniatowskiDA.The role of individual user differences in interpretable and explainable machine learning systems. arXiv Preprint arXiv:200906675.2020.
- 10.1109/DSAA.2018.00018 · DOI · Paper page
- 10.1038/s41598-021-82828-7 · DOI
- 10.1038/s41598-022-08322-w · DOI
- 10.1093/brain/awx173 · DOI
- 10.1109/IEMBS.2006.260931 · DOI
- 10.1186/s40708-020-00105-1 · DOI
- 10.1016/j.jneumeth.2005.04.009 · DOI
- 10.1016/j.artint.2018.07.007 · DOI · Paper page
- 10.1111/j.1528-1167.2012.03564.x · DOI
- 10.1109/ACCESS.2021.3125728 · DOI
- 10.1016/j.bspc.2017.02.001 · DOI
- 10.1007/978-3-319-33383-0_5 · DOI
- 10.1109/EMBC.2012.6347490 · DOI
- 10.1016/S1525-5050(03)00105-7 · DOI
- 10.1103/PhysRevE.67.010901 · DOI
- 10.1145/2939672.2939778 · DOI · Paper page
- 10.4997/jrcpe.2015.305 · DOI
- Charmaz K, 2007, Grounded theory. The Blackwell encyclopedia of sociology, 2023
- 10.1016/S0896-6273(01)00262-8 · DOI
- 10.1038/nature08227 · DOI · Paper page
- 10.1111/epi.17311 · DOI
- 10.1016/S0013-4694(98)00052-2 · DOI
- 10.3389/fnhum.2013.00138 · DOI · Paper page
- 10.1002/epi4.12597 · DOI
- 10.1038/s42256-019-0048-x · DOI · Paper page
- VermaS DickersonJ HinesK.Counterfactual explanations for machine learning: a review. arXiv preprint arXiv:201010596.2020.
- 10.1016/j.artmed.2021.102113 · DOI
- 10.1016/S1474-4422(13)70075-9 · DOI
- 10.1016/j.yebeh.2010.05.008 · DOI
- 10.1016/j.yebeh.2018.05.044 · DOI · Paper page
Cited by
Sorted by citations
Van Dort, B. A., Engelsma, T., Sneekes, P., Peute, L., & Medlock, S. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001417 · Cited by 9 · see citing works
- Authors
- Pinto, M. F., Batista, J., Leal, A., Lopes, F., Oliveira, A., Dourado, A., Abuhaiba, S. I., Sales, F., Martins, P., & Teixeira, C. A.
- Affiliations
- University of Coimbra (PT)
- University of Freiburg (DE)
- Unidade Local de Saúde de Coimbra (PT)
- Corresponding author
- Pinto, Mauro F.
- ORCID
- Venue
- Epilepsia Open
- Publisher
- Wiley
- Published
- 2023-04-27
- Volume
- 8
- Issue
- 2
- Pages
- 285-297
- ISSN
- 2470-9239 (ISSN-L)
- PMC
- PMC10235568
- OpenAlex
- W4366351911
- Cited by
- 23
- References
- 3
- Citations by year
- 2026: 5 · 2025: 5 · 2024: 9 · 2023: 4
- FWCI
- 0.48
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Epilepsia Open (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Fundação para a Ciência e a Tecnologia
- European Regional Development Fund (POCI‐01‐0247‐FEDER070365)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Health Sciences › Medicine › Clinical Neurology › Epilepsy research and treatment
- Keywords
- seizure prediction, EEG, explainable AI, model interpretability, black-box models, epilepsy, model transparency, convolutional neural networks, support vector machine, logistic regression, brain dynamics, time series classification
- MeSH terms
- Brain, Electroencephalography, Epilepsy, Goals, Humans, Seizures
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
