A hybrid fuzzy logic–Random Forest model to predict psychiatric treatment order outcomes: an interpretable tool for legal decision support

May combine two claims.

Fuzzy logic systems allow for expert rule-based reasoning that accommodates uncertainty and vagueness, which are common characteristics in psychiatric data.

“Fuzzy logic systems, by contrast, allow for expert rule-based reasoning that accommodates uncertainty and vagueness which are common characteristics in psychiatric data”

• cites Torres and Nieto, 2006 sourcePartly supported

  The Introduction says fuzzy logic handles uncertainty and vague medical descriptions, and “Fuzzy Logic in Medicine” notes imprecise patient information and difficult classification of mental disorders. Our copy does not describe expert rule-based reasoning, so that part of the claim is unsupported. (LLM, conf=0.84)

  Experimental — accuracy not yet measured
• cites Shoaip et al., 2024 sourcePartly supported

  The Fuzzy inference section says diagnosis rules are created or advised by an expert and that fuzzy inference manages imprecise knowledge through approximate reasoning. The Introduction describes uncertainty and ambiguity in medical data. The paper studies Alzheimer’s disease, however, and does not establish that uncertainty and vagueness are common characteristics of psychiatric data specifically. (LLM, conf=0.84)

  Experimental — accuracy not yet measured
• cites Modai et al., 2004 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

A 10-fold stratified cross-validation procedure was employed during development.

“A 10-fold stratified cross-validation procedure was employed during development”

• cites Jung and Hu, 2015 sourcePartly supported

  Section 4 (Simulations; page not provided) considers 10 folds and describes K-fold cross-validation, supporting the 10-fold part of the method. It specifies a random, even split but does not describe stratification. The source also cannot establish what the citing authors employed during their own development. (LLM, conf=0.86)

  Experimental — accuracy not yet measured

May combine two claims.

Traditional machine learning models often lack interpretability, making them less suited for ethically sensitive applications in mental health law.

“Furthermore, traditional machine learning models often lack interpretability, making them less suited for ethically sensitive applications in mental health law”

• cites Goktas and Grzybowski, 2025 sourcePartly supported

  Section 3.1 says that opaque AI models, especially deep-learning systems, can lack interpretability and pose risks in clinical practice. But the Introduction says traditional rule-based models and decision trees offer greater transparency. The paper does not assess suitability for applications in mental health law; no page numbers are available in our copy. (LLM, conf=0.88)

  Experimental — accuracy not yet measured
• cites Ennab and Mcheick, 2024 sourcePartly supported

  The Introduction (page unavailable) describes deep-learning algorithms as often being black boxes, and §7 (page unavailable) says interpretability is especially important in high-risk healthcare settings. The paper does not extend that assessment to traditional machine-learning models generally—§7.1.1 calls a decision tree easy to interpret—nor does it assess their suitability for mental health law. (LLM, conf=0.85)

  Experimental — accuracy not yet measured

Classified as probably not a substantial claim.

The dataset was split into training and testing subsets using a 70/30 ratio, with 70% for model development and 30% held out for independent performance assessment.

“To ensure validation, the dataset was split into training and testing subsets using a 70/30 ratio, with 70% of the data reserved for model development and 30% held out for independent performance assessment”

• Also stated in: § Random Forest classifier · plain

  “The model was trained on 70% of the dataset and evaluated on the remaining 30% using both predicted labels and prediction probabilities.”

• cites Collins et al., 2024 sourcePartly supported

  The “Random split” section uses a 70/30 development–test split to illustrate performance evaluation, but it does not document how the citing authors split their dataset. It also says that calling the resulting subsets “independent” is a misnomer and generally advises against split-sample validation, so it does not support the claim that this split ensured independent validation. (LLM, conf=0.86)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Although legal frameworks may vary internationally, the core challenges (balancing patient autonomy, clinical risk, and procedural fairness) are widely shared.

“Although legal frameworks may vary internationally, the core challenges (balancing patient autonomy, clinical risk, and procedural fairness) are widely shared (Mikellides et al., 2019).”

• cites Mikellides et al., 2019 sourcePartly supported

  The ‘International community treatment orders use and outcomes’ section (page unavailable) says CTO legislation and administration vary across jurisdictions. ‘Clinical decision-making’ discusses consent and risk, while the international comparison lists oversight and tribunal procedures in some jurisdictions. The paper does not establish procedural fairness as a widely shared challenge across jurisdictions, so the full characterization is not supported. (LLM, conf=0.83)

  Experimental — accuracy not yet measured

Classified as probably not a substantial claim.

Fuzzy logic provides a formal framework to model uncertainty and expert knowledge, which is especially valuable in psychiatric decision-making.

“Fuzzy logic, by contrast, provides a formal framework to model uncertainty and expert knowledge, which is especially valuable in psychiatric decision-making (Torres and Nieto, 2006).”

• cites Torres and Nieto, 2006 sourcePartly supported

  The Introduction describes fuzzy logic as a formalism for medical uncertainty, and the Drug Addictions section describes an expert assigning fuzzy degrees of addiction. The Fuzzy Logic in Medicine section notes difficulties classifying mental disorders and lists related applications, but does not establish that fuzzy logic is especially valuable in psychiatric decision-making. (LLM, conf=0.82)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

The inclusion of the Burden_Score aligns with literature advocating for holistic, multi-dimensional indicators in forensic psychiatry.

“Importantly, the inclusion of the Burden_Score also aligns with literature advocating for holistic, multi-dimensional indicators in forensic psychiatry (Raharjanti et al., 2021).”

• cites Raharjanti et al., 2021 sourcePartly supported

  The Background describes forensic psychiatric analysis as integrating information across psychiatry, psychology, medicine, and law. Clinical Reasoning in Forensic Assessments also says an evaluee’s developmental history, social context, and chain of events must be considered. These passages support a broad, multidimensional approach to assessment, but they do not advocate multidimensional indicators or address a Burden_Score specifically (page numbers unavailable in our copy). (LLM, conf=0.83)

  Experimental — accuracy not yet measured

Classified as probably not a substantial claim.

Prior work in clinical decision support and mental health law has emphasized the limitations of purely statistical or black-box models in capturing the ambiguity of psychiatric documentation and judicial interpretation.

“Prior work in clinical decision support and mental health law has emphasized the limitations of purely statistical or black-box models in capturing the ambiguity of psychiatric documentation and judicial interpretation (Chekroud et al., 2021; McCradden et al., 2023).”

• cites Chekroud et al., 2021 sourcePartly supported

  The “HOW CLOSE WE HAVE COME TO REAL-WORLD IMPLEMENTATION” section (page not specified) says black-box models can be difficult for clinicians to interpret and that accurate algorithms alone are insufficient for clinical decision support. The paper mentions text-based clinical notes, but does not say such models fail to capture ambiguity in psychiatric documentation or judicial interpretation. (LLM, conf=0.88)

  Experimental — accuracy not yet measured
• cites McCradden et al., 2023 sourcePartly supported

  The ‘EBM, clinical judgement and AI’ section (page unavailable) says that ambiguity in psychiatric classifications makes reliance on ML predictions tenuous. The paper also discusses biased descriptors in electronic health records, but it does not address models capturing ambiguity in psychiatric documentation or judicial interpretation under mental health law. (LLM, conf=0.86)

  Experimental — accuracy not yet measured

May combine two claims.

Among the 53 cases in the test set, 56.6% were categorized as Medium and 43.4% as High, with no cases falling into the Low category.

“Among the 53 cases in the test set, 56.6% were categorized as Medium and 43.4% as High, with no cases falling into the Low category.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The fuzzy outputs (Expanded_Score) ranged between 0.3 and 0.85 and were defuzzified using the centroid method.

“The resulting fuzzy outputs (Expanded_Score) ranged between 0.3 and 0.85 and were defuzzified using the centroid method.”

May combine two claims.

The hybrid fuzzy logic-Random Forest model achieved an accuracy of 98.1%, precision of 93.3%, recall of 100%, and an F1 score of 96.6.

“The hybrid model achieved an accuracy of 98.1%, precision of 93.3%, recall of 100%, and a F1 score of 96.6.”

Classified as probably not a substantial claim.

The main outcome variable is a binary indicator reflecting the court's decision to either entirely or partially accept the treatment order.

“The main outcome variable used for model training and evaluation is a binary indicator reflecting the court’s decision to either entirely or partially accept the treatment order.”

May combine two claims.

The most influential predictor was the duration of time granted by the court (35.2%), followed by the duration requested by the treating team (22.6%) and the age of the defendant (13.7%).

“The most influential predictor was the duration of time granted by the court (35.2%), followed by the duration requested by the treating team (22.6%) and the age of the defendant (13.7%).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Ethics approval was not required because all the material used to construct the dataset is publicly available, according to Tri-Council Policy Statement 2.

“According to Tri-Council Policy Statement 2 on Ethical Conduct for Research Involving Humans, ethics approval was not required because all the material used to construct the dataset is publicly available”

• cites Conseil de recherches en sciences humaines du Canada, n.d.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

All personal identifiers were removed in accordance with SOQUIJ's database, and the dataset was processed in full compliance with applicable privacy standards.

“In terms of data privacy, all personal identifiers were removed in accordance with SOQUIJ’s database, and the dataset was processed in full compliance with applicable privacy standards.”

Classified as probably not a substantial claim.

With only 176 cases, adding many low-variance or categorical features to a correlation matrix inflates the familywise Type I error without providing interpretable effect sizes.

“With only 176 cases, adding many low-variance or categorical features to a correlation matrix inflates the familywise Type I error without providing interpretable effect sizes.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Stratification ensured that class imbalance did not affect any fold disproportionately, thus preserving the fairness and reliability of the evaluation.

“Stratification ensured that class imbalance did not affect any fold disproportionately, thus preserving the fairness and reliability of the evaluation.”

May combine two claims.

The semi-structured nature of psychiatric records and the complex language of legal documentation pose barriers to standard predictive modeling.

“However, the semi-structured nature of psychiatric records and the complex language of legal documentation pose barriers to standard predictive modeling”

• cites London, 2019 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S2 (checkable).

This exploratory study offers a novel approach for decision support in forensic psychiatric contexts.

“This exploratory study offers a novel approach for decision support in forensic psychiatric contexts.”

• AssumptionRests on an assumption: No transparent, evidence-informed predictive tools currently exist to estimate the likelihood of full treatment order acceptance.

Probably not a substantial claim: may not pass S3 (load-bearing).

The Random Forest classifier was implemented with class_weight = 'balanced' to address potential class imbalance between cases that were entirely versus partially accepted.

“The model was implemented using the RandomForestClassifier class from the sklearn.ensemble library, with the parameter class_weight = ‘balanced’ to address potential class imbalance between cases that were entirely versus partially accepted”

• cites Pedregosa et al., 2011Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

The study did not include a formal expert evaluation of interpretability.

“the study did not include a formal expert evaluation of interpretability.”

Classified as probably not a substantial claim. May combine two claims.

By encoding rules such as "Severe + Non-compliant → High likelihood of acceptance" the system provided logic paths that could be validated or critiqued by legal or clinical stakeholders, enhancing the model's usability and trustworthiness.

“By encoding rules such as “Severe + Non-compliant → High likelihood of acceptance” the system provided logic paths that could be validated or critiqued by legal or clinical stakeholders, enhancing the model’s usability and trustworthiness.”

May combine two claims.

The machine learning classifier captured non-linear relationships and enhanced overall performance.

“while the machine learning classifier captured non-linear relationships and enhanced overall performance.”

May combine two claims.

The ability of the model to accurately distinguish between fully and partially accepted orders (while simultaneously offering an interpretable rationale for its predictions) addresses a need in both forensic psychiatry and legal medicine.

“The ability of the model to accurately distinguish between fully and partially accepted orders (while simultaneously offering an interpretable rationale for its predictions) addresses a need in both forensic psychiatry and legal medicine.”

Probably not a substantial claim: may not pass S2 (checkable).

The illustrative cases highlight how the model's reasoning aligns with typical clinical and legal expectations.

“These examples highlight how the model’s reasoning aligns with typical clinical and legal expectations.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Not all treatment orders that took place in Quebec in 2024 were reported in the SOQUIJ database, which may underrepresent certain areas.

“not all the treatment orders that took place in Quebec in 2024 were reported in the SOQUIJ’s database which may underrepresent certain areas.”

Classified as probably not a substantial claim.

The present model's ability to integrate fuzzy outputs with structured variables (e.g., age, treatment duration, legal aid status) and semantic indicators (e.g., compliance, severity) reflects an "augmented" approach.

“The present model’s ability to integrate fuzzy outputs with structured variables (e.g., age, treatment duration, legal aid status) and semantic indicators (e.g., compliance, severity) reflects this “augmented” approach.”

May combine two claims.

While the model achieved a recall rate of 100% in the held-out test set, this result should be interpreted in light of the relatively small sample size and the potential for overfitting.

“While the model achieved a recall rate of 100% in the held-out test set, this result should be interpreted in light of the relatively small sample size and the potential for overfitting.”

May combine two claims.

By integrating clinical, legal, and semantic information within a hybrid fuzzy-Random Forest framework, the model achieved strong predictive performance and offered interpretable outputs aligned with expert reasoning.

“By integrating clinical, legal, and semantic information within a hybrid fuzzy–Random Forest framework, the model achieved strong predictive performance and offered interpretable outputs aligned with expert reasoning.”

• AssumptionRests on an assumption: The decision-making process in Quebec’s Superior Court is informed not only by medical evaluations but also by judicial discretion shaped by case law and social values.

Classified as probably not a substantial claim.

The fuzzy inference system was explicitly designed to enhance interpretability via rule-based logic.

“the fuzzy inference system was explicitly designed to enhance interpretability via rule-based logic”

Probably not a substantial claim: may not pass S2 (checkable).

The fuzzy inference system provided a transparent layer that mirrored clinical and legal judgment processes.

“The fuzzy inference system provided a transparent layer that mirrored clinical and legal judgment processes”

May combine two claims.

Duration contributes to the burden calculation but does not dominate it, reinforcing the multidimensional nature of the composite score.

“indicating that while duration contributes to the burden calculation, it does not dominate it which reinforces the multidimensional nature of the composite score.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The construction of semantic features such as severity and compliance flags relied on keyword matching from clinical narratives, which may not fully capture the nuance of psychiatric assessments and could introduce misclassification bias.

“the construction of semantic features such as severity and compliance flags relied on keyword matching from clinical narratives, which may not fully capture the nuance of psychiatric assessments and could introduce misclassification bias.”

May combine two claims.

The study used retrospective data and did not include external validation across multiple sites or years, limiting its ability to predict future or out-of-distribution cases.

“the study used retrospective data and did not include external validation across multiple sites or years, limiting its ability to predict future or out-of-distribution cases.”

Probably not a substantial claim: may not pass S2 (checkable).

These findings suggest that decision support tools grounded in hybrid modeling can offer value in ethically sensitive, cross-disciplinary contexts such as involuntary psychiatric treatment.

“These findings suggest that decision support tools grounded in hybrid modeling can offer value in ethically sensitive, cross-disciplinary contexts such as involuntary psychiatric treatment.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

As health systems and courts increasingly turn to data-driven tools for support, this hybrid architecture offers a blueprint for responsible and context-sensitive AI deployment.

“As health systems and courts increasingly turn to data-driven tools for support, this hybrid architecture offers a blueprint for responsible and context-sensitive AI deployment.”

May combine two claims.

The recall score likely reflects model performance on this specific dataset and may not generalize to future or out-of-distribution cases.

“As such, the recall score likely reflects model performance on this specific dataset and may not generalize to future or out-of-distribution cases.”

May combine two claims.

By integrating structured demographic and legal variables with expert-informed semantic features and rule-based reasoning, the model achieved high predictive accuracy while preserving interpretability.

“By integrating structured demographic and legal variables with expert-informed semantic features and rule-based reasoning, the model achieved high predictive accuracy while preserving interpretability.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

The creation of the Burden_Score and Expanded_Score allowed for meaningful integration of clinical, behavioral, and procedural data, reinforcing the value of multidimensional modelling in such contexts.

“Furthermore, the creation of the Burden_Score and Expanded_Score allowed for meaningful integration of clinical, behavioral, and procedural data, reinforcing the value of multidimensional modelling in such contexts.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

The hybrid fuzzy-Random Forest model presented here offers a promising approach to capturing the intersection of evidence-based psychiatry and normative legal reasoning.

“The hybrid fuzzy–Random Forest model presented here offers a promising approach to capturing that intersection.”

• AssumptionRests on an assumption: The decision-making process in Quebec’s Superior Court is informed not only by medical evaluations but also by judicial discretion shaped by case law and social values.

May combine two claims.

Although the fuzzy logic system was based on expert-informed rules, the weighting of those rules was manually tuned rather than optimized through formal learning algorithms, potentially limiting the model's scalability and introducing overfitting.

“although the fuzzy logic system was based on expert-informed rules, the weighting of those rules was manually tuned rather than optimized through formal learning algorithms, potentially limiting the model’s scalability and introduce overfitting.”

May combine two claims.

The dataset was limited to 176 cases from Quebec's Superior Court in 2024, which may constrain the generalizability of the model to other jurisdictions or legal systems with differing criteria for involuntary treatment.

“The dataset was limited to 176 cases from Quebec’s Superior Court in 2024, which may constrain the generalizability of the model to other jurisdictions or legal systems with differing criteria for involuntary treatment.”