CoChat
Sign Up
Cited by 118Ethics and Social Impacts of AIMachine Learning and AlgorithmsHate Speech and Cyberbullying Detection

The DOI matches its Crossref record; 166 of the 173 references it cites were found in a registry or an index; none of the 23 sources its claims cite is checked for support yet.

Evolution and impact of bias in human and machine learning algorithm interaction

Open access
Sun, Wenlong, Nasraoui, Olfa, Shafto, Patrick·2020·PLOS ONE
DOI 10.1371/journal.pone.0235502PMC PMC7425868
Open this paper in CoChat — free accountOpen PDF

Abstract

Traditionally, machine learning algorithms relied on reliable labels from experts to build predictions. More recently however, algorithms have been receiving data from the general population in the form of labeling, annotations, etc. The result is that algorithms are subject to bias that is born from ingesting unchecked information, such as biased samples and biased labels. Furthermore, people and algorithms are increasingly engaged in interactive processes wherein neither the human nor the algorithms receive unbiased data. Algorithms can also make biased predictions, leading to what is now known as algorithmic bias. On the other hand, human's reaction to the output of machine learning methods with algorithmic bias worsen the situations by making decision based on biased information, which will probably be consumed by algorithms later. Some recent research has focused on the ethical and moral implication of machine learning algorithmic bias on society. However, most research has so far treated algorithmic bias as a static factor, which fails to capture the dynamic and iterative properties of bias. We argue that algorithmic bias interacts with humans in an iterative manner, which has a long-term effect on algorithms' performance. For this purpose, we present an iterated-learning framework that is inspired from human language evolution to study the interaction between machine learning algorithms and humans. Our goal is to study two sources of bias that interact: the process by which people select information to label (human action); and the process by which an algorithm selects the subset of information to present to people (iterated algorithmic bias mode). We investigate three forms of iterated algorithmic bias (personalization filter, active learning, and random) and how they affect the performance of machine learning algorithms by formulating research questions about the impact of each type of bias. Based on statistical analyses of the results of several controlled experiments, we found that the three different iterated bias modes, as well as initial training data class imbalance and human action, do affect the models learned by machine learning algorithms. We also found that iterated filter bias, which is prominent in personalized user interfaces, can lead to more inequality in estimated relevance and to a limited human ability to discover relevant data. Our findings indicate that the relevance blind spot (items from the testing set whose predicted relevance probability is less than 0.5 and who thus risk being hidden from humans) amounted to 4% of all relevant items when using a content-based filter that predicts relevant items. A similar simulation using a real-life rating data set found that the same filter resulted in a blind spot size of 75% of the relevant testing set.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

The paper proposes an iterated-learning framework to study how algorithmic and human biases interact and evolve in recommendation/filtering loops. Across synthetic and MovieLens simulations, iterated filter bias consistently shifted learned relevance boundaries, increased prediction inequality, and enlarged blind spots that hide relevant items, while random selection and active learning often did not.

Key findings

  • Iterated filter bias had significant effects in 2D synthetic data: boundary shift p<.001, effect size = 1.96; class-1 blind spot p=2.4e-10, effect size = -1.22; all-classes blind spot p=5.9e-12, effect size = -1.44. Active learning and random selection did not significantly affect boundary shift (p=.15 and .77, respectively).
  • On MovieLens real data, filter bias significantly decreased points predicted as class y=1 (p=7.1e-15, ES=1.88), increased class-1 blind spot (p=7.0e-15, ES=-1.89), and increased Gini inequality (p=7.7e-15, ES=-1.83); active learning also affected boundary shift and blind spot, while random selection did not significantly affect boundary or blind spot but decreased inequality.
  • The abstract reports a relevance blind spot of 4% of all relevant items for a content-based filter in one simulation, and a blind spot size of 75% of the relevant testing set in a similar simulation using a real-life rating data set.
  • Initial class imbalance and human action mattered: imbalanced initialization significantly affected boundary shift for all three bias modes (Kruskal-Wallis p=5.5e-19 filter, 3.27e-17 active learning, 1.34e-15 random); higher human action probability increased filter-bias boundary shift (p_action=1: p=0.4e-11, ES=3.087; p_action=0.5: p=1.8e-6, ES=0.87; p_action=0.01: p=.259). With class-dependent human action ratio 10:1, filter bias increased class-1 blind spot (p=1.1e-10, ES=-1.2) and random selection decreased it (p=.0008, ES=0.7).

Methods

  • The authors use an iterated-learning framework inspired by language evolution, simplifying recommendation into a two-class classification problem (relevant/non-relevant) with a Naive Bayes classifier. They compare three selection mechanisms: random, active-bias, and filter-bias; human action probabilities of 1, 0.5, and 0.01; and a class-dependent action ratio of 10:1, running 200 iterations and 40 independent simulations per condition.
  • Synthetic data: 2D Gaussian classes centered at {-2,0} and {2,0} with σ=1 and 1000 points per class, split into testing (200/class), validation (200/class), initialization ratios 10/100, 100/100, and 100/10, plus 3D, 4D, and 10D synthetic sets. Real data: MovieLens with 100,004 ratings on 9,125 movies from 671 users; user 547 (2,312 movies rated; 18 genres after filtering; PCA to 11 features) was studied first, then 7 more users; ratings were discretized as [1,3]=class 0 and (3,5]=class 1, with unbiased initialization only.
  • Main analyses measure boundary shift, Gini coefficient, and blind spot size (δ=0.5), using Shapiro-Wilk, Mann-Whitney U, t-test, ANOVA, Kruskal-Wallis, and effect sizes.

Limitations

  • The paper states future work will extend the framework to more users in a collaborative filtering setting, test additional parameters/configurations such as top-N recommendation list size and different active learning types beyond uncertainty-based active learning, and conduct human experiments similar to the simulated research questions.
  • Evident from the text: the findings rely on simulations rather than human experiments, use a simplified two-class content-based recommendation setup with Naive Bayes and independent-feature assumptions, and the real-data experiments focus on one user at a time and only the 8 most active MovieLens users, with genres as item features and PCA.
  • Evident from the text: for the real-life data set, the authors only explore unbiased initialization and assume action probabilities of 1, so human-action and class-imbalance effects are not tested on the real data.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0235502

Cites

166 of 166 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig 1 Language learning vs. machine learning. Language learning is analogous to machine learning in several aspects, such as ‘hypothesis’ to ‘model’ in sub-figure (a). Learning a language which gets transmitted throughout consecutive generations of human speakers is analogous to learning a model through consecutive iterations of online machine learning in sub-figure (b). In the iterative human-machine learning algorithm interaction, the output from ML affects human behavior and human also interact with the output which affects next iteration.
Fig 1 Language learning vs. machine learning. Language learning is analogous to machine learning in several aspects, such as ‘hypothesis’ to ‘model’ in sub-figure (a). Learning a language which gets transmitted throughout consecutive generations of human speakers is analogous to learning a model through consecutive iterations of online machine learning in sub-figure (b). In the iterative human-machine learning algorithm interaction, the output from ML affects human behavior and human also interact with the output which affects next iteration.CC BY · PubMed Central
Fig 2 Markov chains for iterated learning with (bottom) and without (top) dependency from previous iterations.
Fig 2 Markov chains for iterated learning with (bottom) and without (top) dependency from previous iterations.CC BY · PubMed Central
Fig 3 Evolution of bias between algorithm and human. In sub-figure (a), biased data from a human may lead to a biased algorithm, this is pre-algorithmic bias. In sub-figure (b), a biased algorithmic output might affect human behavior: For instance, by hiding certain items from humans, algorithms may affect human discovery, learning, and awareness in the long term. Sub-figure (c) indicates a continuous interaction between humans and algorithms that generates bias that we refer to as iterated bias, namely bias that results from repeated interaction between humans and algorithms.
Fig 3 Evolution of bias between algorithm and human. In sub-figure (a), biased data from a human may lead to a biased algorithm, this is pre-algorithmic bias. In sub-figure (b), a biased algorithmic output might affect human behavior: For instance, by hiding certain items from humans, algorithms may affect human discovery, learning, and awareness in the long term. Sub-figure (c) indicates a continuous interaction between humans and algorithms that generates bias that we refer to as iterated bias, namely bias that results from repeated interaction between humans and algorithms.CC BY · PubMed Central
Fig 4 Original data with two classes.
Fig 4 Original data with two classes.CC BY · PubMed Central
Fig 5 Boundary shift (Eq 18) based on the three iterated algorithmic bias forms. The y axis is the number of testing points which are predicted to be in class y = 1. This figure shows that random selection and active learning bias converge to the ground-truth boundary. Filter bias, on the other hand, results in decreasing numbers of points predicted in the target category class 1, consistent with an overly restrictive category boundary.
Fig 5 Boundary shift (Eq 18) based on the three iterated algorithmic bias forms. The y axis is the number of testing points which are predicted to be in class y = 1. This figure shows that random selection and active learning bias converge to the ground-truth boundary. Filter bias, on the other hand, results in decreasing numbers of points predicted in the target category class 1, consistent with an overly restrictive category boundary.CC BY · PubMed Central
Fig 6 Box-plot of the Gini coefficient resulting from three forms of iterated algorithmic bias. An ANOVA test across these three iterated algorithmic bias forms shows that the Gini index values are significantly different. The p-value from the ANOVA test is close to 0.000 (<0.05), which indicates that the three iterated algorithmic bias forms have different effects on the Gini coefficient. Here, FB, AL and RM are the abbreviations of filter bias, active learning bias and random selection, respectively. The ‘first’ indicates the beginning of iteration (i.e., t = 0), while the ‘last’ means the end of iteration (i.e., t = 200). Note that we use these abbreviations in the rest of our paper.
Fig 6 Box-plot of the Gini coefficient resulting from three forms of iterated algorithmic bias. An ANOVA test across these three iterated algorithmic bias forms shows that the Gini index values are significantly different. The p-value from the ANOVA test is close to 0.000 (<0.05), which indicates that the three iterated algorithmic bias forms have different effects on the Gini coefficient. Here, FB, AL and RM are the abbreviations of filter bias, active learning bias and random selection, respectively. The ‘first’ indicates the beginning of iteration (i.e., t = 0), while the ‘last’ means the end of iteration (i.e., t = 200). Note that we use these abbreviations in the rest of our paper.CC BY · PubMed Central
Fig 7 Box-plot of the size of the class-1-blind spot for all three iterated algorithmic bias forms. In this figure, the x-axis is the index of the three forms of iterated algorithms biases. As shown in this box-plot, the initial class-1-blind spot is centered at 7. This is because the 200 randomly selected initial points from both classes force the boundary to be similar regardless of the randomization.
Fig 7 Box-plot of the size of the class-1-blind spot for all three iterated algorithmic bias forms. In this figure, the x-axis is the index of the three forms of iterated algorithms biases. As shown in this box-plot, the initial class-1-blind spot is centered at 7. This is because the 200 randomly selected initial points from both classes force the boundary to be similar regardless of the randomization.CC BY · PubMed Central
Fig 8 Different initialization imbalance ratios affect the starting model boundary ax expected.
Fig 8 Different initialization imbalance ratios affect the starting model boundary ax expected.CC BY · PubMed Central
Fig 9 The box-plots showing the distribution of the number of points that moved across the boundary in the first and last iterations of the iterated learning for three iterated algorithmic biases with different class imbalanced initialization ratios. Because the number of points that move across the boundary indicates the intensity of the boundary shift, this shows that for all three iterated algorithmic bias modes, a high imbalanced initialization results in a higher boundary shift compared to a balanced initialization.
Fig 9 The box-plots showing the distribution of the number of points that moved across the boundary in the first and last iterations of the iterated learning for three iterated algorithmic biases with different class imbalanced initialization ratios. Because the number of points that move across the boundary indicates the intensity of the boundary shift, this shows that for all three iterated algorithmic bias modes, a high imbalanced initialization results in a higher boundary shift compared to a balanced initialization.CC BY · PubMed Central
Fig 10 Effect of Human action on the boundary learned during the iterations. The y-axis is the number of testing points which are predicted to be relevant. We can see that for both random selection and active learning bias, the number of points predicted as relevant converges to the ground truth boundary when there is a high probability of human action. On the other hand, filter bias tends to diverge from the ground truth boundary in the presence of high human action probability. With paction = 0.01, there are no obvious trends for all three algorithmic bias forms.
Fig 10 Effect of Human action on the boundary learned during the iterations. The y-axis is the number of testing points which are predicted to be relevant. We can see that for both random selection and active learning bias, the number of points predicted as relevant converges to the ground truth boundary when there is a high probability of human action. On the other hand, filter bias tends to diverge from the ground truth boundary in the presence of high human action probability. With paction = 0.01, there are no obvious trends for all three algorithmic bias forms.CC BY · PubMed Central
Fig 11 The box-plots showing the distribution of the number of points that are predicted to be in class y = 1 in the first and last iterations of the iterated learning for three iterated algorithmic biases with human action probability ratio 10:1. Filter bias significantly decreases the number of points predicted to be in class y = 1. On the other hand, random selection significantly increases the number of points predicted to be in class y = 1. Active learning has no such significant effect.
Fig 11 The box-plots showing the distribution of the number of points that are predicted to be in class y = 1 in the first and last iterations of the iterated learning for three iterated algorithmic biases with human action probability ratio 10:1. Filter bias significantly decreases the number of points predicted to be in class y = 1. On the other hand, random selection significantly increases the number of points predicted to be in class y = 1. Active learning has no such significant effect.CC BY · PubMed Central
Fig 12 The box-plots showing the size of the class-1 blind spot in the first and last iterations of the iterated learning for three iterated algorithmic biases with human action probability ratio 10:1. Filter bias significantly decreases the number of points predicted to be in class y = 1. On the other hand, random selection significantly increases the number of points predicted to be in class y = 1. Active learning has no such significant effect.
Fig 12 The box-plots showing the size of the class-1 blind spot in the first and last iterations of the iterated learning for three iterated algorithmic biases with human action probability ratio 10:1. Filter bias significantly decreases the number of points predicted to be in class y = 1. On the other hand, random selection significantly increases the number of points predicted to be in class y = 1. Active learning has no such significant effect.CC BY · PubMed Central
Fig 13 The box-plots showing the inequality score at the first and last iterations of the iterated learning for three iterated algorithmic biases with human action probability ratio 10:1. Filter bias significantly increases the Gini coefficient. On the other hand, random selection significantly decrease the Gini coefficient, as well as the active learning.
Fig 13 The box-plots showing the inequality score at the first and last iterations of the iterated learning for three iterated algorithmic biases with human action probability ratio 10:1. Filter bias significantly increases the Gini coefficient. On the other hand, random selection significantly decrease the Gini coefficient, as well as the active learning.CC BY · PubMed Central
Fig 14 The box-plots showing the distribution of the number of points that are predicted to be in class y = 1 in the first and last iterations of the iterated learning for three iterated algorithmic biases with the MovieLens data set. Filter bias significantly decreases the number of points predicted to be in class y = 1. On the other hand, random selection and Active learning have no such significant effect.
Fig 14 The box-plots showing the distribution of the number of points that are predicted to be in class y = 1 in the first and last iterations of the iterated learning for three iterated algorithmic biases with the MovieLens data set. Filter bias significantly decreases the number of points predicted to be in class y = 1. On the other hand, random selection and Active learning have no such significant effect.CC BY · PubMed Central
Fig 15 The box-plots showing the distribution of the size of the class-1 blind spot in the first and last iterations of the iterated learning for three iterated algorithmic biases with the MovieLens data set. Filter bias significantly increases the size of the class-1 blind spot. On the other hand, random selection and Active learning have no such significant effect.
Fig 15 The box-plots showing the distribution of the size of the class-1 blind spot in the first and last iterations of the iterated learning for three iterated algorithmic biases with the MovieLens data set. Filter bias significantly increases the size of the class-1 blind spot. On the other hand, random selection and Active learning have no such significant effect.CC BY · PubMed Central
Fig 16 The box-plots showing the distribution of the Gini coefficient of the prediction in the first and last iterations of the iterated learning for three iterated algorithmic biases on the MovieLens data set. Filter bias significantly increases the in equality of prediction. On the other hand, random selection and Active learning lead to a significantly decrease in the inequality of prediction.
Fig 16 The box-plots showing the distribution of the Gini coefficient of the prediction in the first and last iterations of the iterated learning for three iterated algorithmic biases on the MovieLens data set. Filter bias significantly increases the in equality of prediction. On the other hand, random selection and Active learning lead to a significantly decrease in the inequality of prediction.CC BY · PubMed Central

Verification

Claims the paper makes

71 Own result · 34 Own interpretation · 16 No basis given · 12 Not checked yet

Verify all 23 references — free account

Show all 89 claims

  • It may state two separate claims as one. 33
  • The quote only partly states this claim. 5
  • Borderline: one of the tests for a substantial claim is unclear. 1
  • Probably not a substantial claim. 11
44 of 44

  • The paper relies on this without support.

    People and algorithms are increasingly engaged in interactive processes wherein neither the human nor the algorithms receive unbiased data.

    “Furthermore, people and algorithms are increasingly engaged in interactive processes wherein neither the human nor the algorithms receive unbiased data.”

  • The authors state this as an assumption.

    A Bayesian model for prediction is assumed for the algorithm.

    “Following Griffiths’ model for human learners, we assume a Bayesian model for prediction.”

  • The authors state this as an assumption.

    People's response is always observed in the above analysis.

    “The above analysis assumes that people’s response is always observed.”

  • The authors state this as an assumption.

    People have some target hypothesis h*, which represents optimal performance for the algorithm.

    “Assume that people have some target hypothesis, h*, which represents optimal performance for the algorithm.”

  • The authors state this as an assumption.

    A simplified algorithm where only the very uncertain data are selected, and a mixture of random sampling and active learning, is assumed.

    “Assuming a simplified algorithm where only the very uncertain data are selected, we can investigate the limiting behavior of an algorithm with the active learning bias. Assuming a mixture of random sampling and active learning, we obtain:”

  • The authors state this as an assumption.

    The choice of whether to act can be formalized using Luce choice, a special case of softmax.

    “We can formalize this choice using Luce choice [56], a special case of softmax [137]”

  • The paper relies on this without support.

    The synthetic dataset represents a typical classification problem and is comparable to the real-life dataset after selecting one user and corresponding items.

    “The synthetic dataset represents a typical classification problem, it is the same for our real-life dataset after selecting one user and corresponding items.”

  • The paper relies on this without support.

    In the real world, some data can be invisible to some users because of bias either from users or from the algorithm itself.

    “In the real world, some data can be invisible to some users because of bias either from users or from the algorithm itself.”

  • The paper relies on this without support.

    Class imbalance can emerge from the algorithm intentionally asking users to rate only the most popular items, a common strategy used to collect initial ratings for new users.

    “Class imbalance can also emerge from the algorithm intentionally asking users to rate only the most popular items, a common strategy used to collect initial ratings for new users.”

  • The authors state this as an assumption.

    In RQ2, humans have a prior probability to act that is not dependent on the true label of the item presented.

    “In research question 2 (RQ 2), we assumed that humans have a prior probability to act which is not dependent on the true label of the item presented.”

  • The authors state this as an assumption.

    Humans have a higher probability to label or rate when the item presented is from relevant class y = 1.

    “we try to simulate this by assuming that humans have a higher probability to label or rate when the item presented is from relevant class y = 1 (see Eq 15).”

  • The paper relies on this without support.

    Filter bias already prefers points from class y = 1.

    “since filter bias already prefers points from class y = 1.”

  • The paper relies on this without support.

    Active learning bias already prefers points that are close to the boundary.

    “since this bias already prefers points that are close to the boundary.”

  • The paper relies on this without support.

    Both random selection and active learning are used to build an accurate model in machine learning.

    “both random selection and active learning are used to build an accurate model in machine learning.”

  • The paper relies on this without support.

    Random selection with class-dependent human action probability prefers points from class y = 1, however randomly.

    “Random selection with class-dependent human action probability prefer points from class y = 1, however randomly.”

  • The paper relies on this without support.

    Active learning aims to help learn the correct boundary.

    “Active learning aims to help learn correct boundary”

  • The paper relies on this without support.

    Humans and AI algorithms are both increasingly intertwined in feedback loops that end up feeding biased data to both humans and algorithms.

    “Humans and AI algorithms are both increasingly intertwined in feedback loops that end up feeding biased data to both humans and algorithms.”
17 of 17

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

7 of 7

Citations

Cites

Research works printed but not linked: 11

  1. Spark KJ. Artificial Intelligence: What can it offer to Information Retrieval. Proceedings of the Informatics 3, Aslib, ed, London. 1978;.
  2. McFadden DL. Quantal choice analaysis: A survey. In: Annals of Economic and Social Measurement, Volume 5, number 4. NBER; 1976. p. 363–390.
  3. TL Griffiths, 2005, Proceedings of the Cognitive Science Society
  4. Smith K. Iterated learning in populations of Bayesian agents. In: Proceedings of the 31st annual conference of the cognitive science society. Citeseer; 2009. p. 697–702.
  5. KJ Rothman, 2008, Modern epidemiology
  6. Dwork C, Immorlica N, Kalai AT, Leiserson MD. Decoupled classifiers for group-fair and efficient machine learning. In: Conference on Fairness, Accountability and Transparency; 2018. p. 119–133.
  7. Barocas S. Data mining and the discourse on discrimination. In: Data Ethics Workshop, Conference on Knowledge Discovery and Data Mining; 2014.
  8. Schnabel T, Swaminathan A, Singh A, Chandak N, Joachims T. Recommendations as treatments: Debiasing learning and evaluation. arXiv preprint arXiv:160205352. 2016;.
  9. A Sinha, 2016, Advances in neural information processing systems, 3243
  10. S Milano, 2020, AI & SOCIETY, 1
  11. A Stuart, 1994, Distribution theory

Cited by

Sorted by citations

See all 118

Related

See all

Metadata

Authors
Sun, W., Nasraoui, O., & Shafto, P.
Affiliations
  • University of Louisville (US)
  • Rutgers, The State University of New Jersey (US)
Corresponding author
Nasraoui, Olfa
ORCID
Venue
PLOS ONE
Publisher
Public Library of Science (PLoS)
Published
2020-08-13
Volume
15
Issue
8
Pages
e0235502
ISSN
1932-6203 (ISSN-L)
DOI
10.1371/journal.pone.0235502
PMC
PMC7425868
OpenAlex
W3048367007
Semantic Scholar
fba50a0f6e7f1bb47e80ffa8c523396aa747a6ae
Cited by
116
References
173
Citations by year
2026: 15 · 2025: 29 · 2024: 16 · 2023: 21 · 2022: 26 · 2021: 8 · 2020: 1
FWCI
19.26
Type
journal article
Language
en
Open access
Licence
Locations
  • PLoS ONE (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Division of Information and Intelligent Systems (1549981)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
  • Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
Keywords
algorithmic bias, human-machine interaction, iterated learning, filter bubbles, personalization, active learning (machine learning), class imbalance, personalized interfaces, information filtering, relevance prediction
MeSH terms
Machine Learning, Humans, Learning, Bias

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review