Quantifying and mitigating recorder-induced variability in ecological acoustic indices
Open accessPreprint — published as 10.1016/j.ecoinf.2024.102668
Abstract
Due to the complexity of soundscapes, Ecological Acoustic indices (EAI) are frequently used as metrics to summarize ecologically meaningful information from audio recordings. Recent technological advances have allowed the rapid development of many audio recording devices with significant hardware/firmware variations among brands, whose effects in calculating EAI have not yet be determined. In this work, we show how recordings of the same landscape with different devices effectively hinder reproducibility and produce contradictory results. To address these issues, we propose a preprocessing pipeline to reduce EAI variability resulting from different hardware without altering the target information in the audio. To this end, we tested eight EAI commonly used in soundscape analyses. We targeted three common cases of variability caused by recorder characteristics: sampling frequency, microphone gain variation, and frequency response. We quantified the difference in the probability density functions of each index among recorders according to the Kullback-Leibler divergence. As a result, our approach reduced up to 75% variations among recorders from different brands (AudioMoth and SongMeter) and identified the conditions in which these devices are comparable. In conclusion, we demonstrated that different devices effectively affect EAI and show how these variations can be mitigated. Graphical Abstract: Highlights: Addressing recorder-induced biases in acoustic indices for improved reproducibility. Proposing an effective method to mitigate recorder-related biases. Evaluating pipeline proposed performance via acoustic index distribution analysis.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Differences in microphones, recording settings, and sampling rates can affect spectral features and index values, complicating comparisons across studies.
“Differences in microphones, recording settings, and sampling rates can further affect spectral features and index values, complicating comparisons across studies”
From Listening forward: emerging roles of bioacoustics in ecology, evolution, and conservation (2026) · cites this paper as Luna-Naranjo et al. 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Measurement of Species Diversity
Peet, R. (1974) · Annual Review of Ecology and Systematics · doi:10.1146/annurev.es.05.110174.001441 · Cited by 2,399 · cited 1× · see citing works
“This difference results in a negative correlation between AEI and ADI indices, as demonstrated by (Peet, 1974).”
Bose, S. (2011) · Starting Digital Signal Processing in Telecommunication Engineering · doi:10.1007/978-981-13-7114-1_10 · Cited by 1,778 · cited 2×
“Many EAI are based on the Fast Fourier Transform (FFT) of the signal.”
Rényi Divergence and Kullback-Leibler Divergence
van Erven, T., & Peter Harremöes, D. (2014) · IEEE Transactions on Information Theory · doi:10.1109/tit.2014.2320500 · Cited by 1,553 · see citing works
Statistical Methods in Experimental Physics
James, F. (2006) · WORLD SCIENTIFIC eBooks · doi:10.1142/6096 · Cited by 971 · cited 1× · see citing works
“The advantage of using K-S distance is that it is sensitive to both the shape and position of the probability distribution, providing a clear comparison between distributions (James, 2006).”
Deep Learning for Audio Signal Processing
Purwins, H., Li, B., Virtanen, T. I., Schlüter, J., Chang, S.-Y., & Sainath, T. N. (2019) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2019.2908700 · Cited by 881 · cited 1× · see citing works
“Moreover, by making EAI comparable across studies , we can use them in new applications such as machine learning and deep learning, which require features in the same space, i.e., without considering the device with which they were acquired (Purwins et al., 2019).”
Emerging opportunities and challenges for passive acoustics in ecological assessment and monitoring
Gibb, R., Browning, E., Glover‐Kapfer, P., & Jones, K. E. (2018) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13101 · Cited by 753 · cited 3× · see citing works
“In this study, we address the challenge presented by this open issue, which mainly affects the growth of acoustic monitoring and is noticeably prominent in large-scale studies (Gibb et al., 2019).”
Rapid Acoustic Survey for Biodiversity Appraisal
Sueur, J., Pavoine, S., Hamerlynck, O., & Duvail, S. (2008) · PLoS ONE · doi:10.1371/journal.pone.0004065 · Cited by 689 · cited 2× · see citing works
“The Entropy Index (H) was proposed by Sueur et al. (2008) as an indicator of heterogeneity and is based on estimating the Shannon uniformity over time and frequency.”
Pieretti, N., Farina, A., & Morri, D. (2011) · Ecological Indicators · doi:10.1016/j.ecolind.2010.11.005 · Cited by 658 · see citing works
Acoustic Indices for Biodiversity Assessment and Landscape Investigation
Sueur, J., Farina, A., Gasc, A., Pieretti, N., & Pavoine, S. (2014) · Acta Acustica united with Acustica · doi:10.3813/aaa.918757 · Cited by 519 · cited 1× · see citing works
“EAI encode complex and biologically relevant information in a single numerical value, efficiently processing large volumes of data (Sueur et al., 2014; Eldridge et al., 2016).”
Ecoacoustics: the Ecological Investigation and Interpretation of Environmental Sound
Sueur, J., & Farina, A. (2015) · Biosemiotics · doi:10.1007/s12304-015-9248-x · Cited by 485 · cited 1× · see citing works
“Ecological acoustic indices (EAI) are one of the most common metrics to study soundscapes as they are easy to calculate and are assumed to have biological interpretability (Buxton et al., 2018; Sueur and Farina, 2015; Farina and Gage, 2017).”
Kasten, E. P., Gage, S. H., Fox, J., & Joo, W. (2012) · Ecological Informatics · doi:10.1016/j.ecoinf.2012.08.001 · Cited by 408 · see citing works
Monitoring animal diversity using acoustic indices: Implementation in a temperate woodland
Depraetere, M., Pavoine, S., Jiguet, F., Gasc, A., Duvail, S., & Sueur, J. (2011) · Ecological Indicators · doi:10.1016/j.ecolind.2011.05.006 · Cited by 404 · see citing works
On the Choice of Smoothing Parameters for Parzen Estimators of Probability Density Functions
Duin (1976) · IEEE transactions on computers · doi:10.1109/tc.1976.1674577 · Cited by 384 · cited 1× · see citing works
“To validate our approach, we compute the eight EAI’s probability density functions (PDF) at each site using the Parzen method Robert (1976).”
Towsey, M., Wimmer, J., Williamson, I., & Roe, P. (2014) · Ecological Informatics · doi:10.1016/j.ecoinf.2013.11.007 · Cited by 369 · cited 1× · see citing works
“Due to their flexibility, EAI have been used for different aims, such as acoustic niches, uniformity, soundscape degradation, and environmental studies (Towsey et al., 2014; Shaw et al., 2021; Benocci et al., 2020).”
A primer of acoustic analysis for landscape ecologists
Villanueva-Rivera, L. J., Pijanowski, B. C., Doucette, J. S., & Pekin, B. K. (2011) · Landscape Ecology · doi:10.1007/s10980-011-9636-9 · Cited by 357 · cited 2× · see citing works
“The Acoustic Diversity Index (ADI) was proposed by Villanueva-Rivera et al. (2011) to determine the occupancy degree of acoustic niches.”
Boelman, N. T., Asner, G. P., Hart, P. J., & Martin, R. E. (2007) · Ecological Applications · doi:10.1890/07-0004.1 · Cited by 354 · see citing works
Guidelines for the use of acoustic indices in environmental research
Bradfer‐Lawrence, T., Gardner, N., Bunnefeld, L., Bunnefeld, N., Willis, S. G., & Dent, D. H. (2019) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13254 · Cited by 319 · cited 4× · see citing works
“However, several studies using the same indices provide contradictory re-sults by correlating with biological variables, generating doubts about EAI effectiveness, despite the prevalence of their use Bradfer-Lawrence et al. (2019a).”
InfluentialNussbaumer, H. J. (1981) · Springer series in information sciences · doi:10.1007/978-3-662-00551-4_4 · Cited by 311 · see citing works
AudioMoth: A low-cost acoustic device for monitoring biodiversity and the environment
Hill, A. P., Prince, P., Snaddon, J. L., Doncaster, C. P., & Rogers, A. (2019) · HardwareX · doi:10.1016/j.ohx.2019.e00073 · Cited by 284 · cited 1× · see citing works
“This problem will increase since nowadays researchers commonly have in their stocks high and low cost devices (SongMeter and BAR and Au-dioMoth) (Hill et al., 2019).”
Acoustic indices as proxies for biodiversity: a meta‐analysis
Alcocer, I., Lima, H., Sugai, L. S. M., & Llusia, D. (2022) · Biological Reviews · doi:10.1111/brv.12890 · Cited by 273 · see citing works
Koizumi, S. (1959) · doi:10.14492/hokmj/1530756071 · Cited by 262
Efficacy of extracting indices from large‐scale acoustic recordings to monitor biodiversity
Buxton, R. T., McKenna, M. F., Clapp, M. K., Meyer, E. W., Stabenau, E., Angeloni, L. M., Crooks, K. R., & Wittemyer, G. (2018) · Conservation Biology · doi:10.1111/cobi.13119 · Cited by 230 · cited 15× · see citing works
“However, some species that make calls at low frequencies can generate values of NDSI 0 .”
InfluentialEldridge, A., Guyot, P., Moscoso, P., Johnston, A., Eyre-Walker, Y., & Peck, M. (2018) · Ecological Indicators · doi:10.1016/j.ecolind.2018.06.012 · Cited by 209 · cited 1× · see citing works
“However, since these indices may provide similar information, using both indices may sometimes be redundant (Eldridge et al., 2018).”
Do you hear what I hear? Implications of detector selection for acoustic monitoring of bats
Adams, A. M., Jantzen, M. K., Hamilton, R., & Fenton, M. B. (2012) · Methods in Ecology and Evolution · doi:10.1111/j.2041-210x.2012.00244.x · Cited by 198 · cited 2× · see citing works
“Step-1: Signal resampling Sampling frequency differences are often disregarded, Adams et al. (2012); Rempel et al. (2013).”
Biases of acoustic indices measuring biodiversity in urban areas
Fairbrass, A. J., Rennert, P., Williams, C., Titheridge, H., & Jones, K. E. (2017) · Ecological Indicators · doi:10.1016/j.ecolind.2017.07.064 · Cited by 198 · see citing works
Gasc, A., Pavoine, S., Lellouch, L., Grandcolas, P., & Sueur, J. (2015) · Biological Conservation · doi:10.1016/j.biocon.2015.06.018 · Cited by 151 · see citing works
Sound Analysis and Synthesis with R
Sueur, J. (2018) · Use R! · doi:10.1007/978-3-319-77647-7 · Cited by 147 · cited 1× · see citing works
“Subsequently, traditional peak detection algorithms (such as proposed by Dumpala et al. (1982) or Sueur (2018)) are used to find prominent peaks.”
Gasc, A., Sueur, J., Pavoine, S., Pellens, R., & Grandcolas, P. (2013) · PLoS ONE · doi:10.1371/journal.pone.0065311 · Cited by 140 · cited 1× · see citing works
“The Number of Peaks index (NP) was proposed by Gasc et al. (2013) as an indicator of acoustic complexity using the number of prominent peaks in frequency (see Figure B.10).”
Passive acoustic monitoring in ecology and conservation.
Browning, E., Gibb, R., Glover‐Kapfer, P., & Jones, K. E. (2017) · UCL Discovery (University College London) · doi:10.25607/obp-876 · Cited by 116 · see citing works
Dröge, S., Martin, D. A., Andriafanomezantsoa, R., Buřivalová, Z., Fulgence, T. R., Osen, K., Rakotomalala, E., Schwab, D., Wurz, A., Richter, T., et al. (2020) · Ecological Indicators · doi:10.1016/j.ecolind.2020.106929 · Cited by 112 · cited 1× · see citing works
“Additionally, the AEI index has been used to differentiate acoustic activity at different times of the day and in different habitats Dr¨oge et al. (2021).”
Acoustic indices perform better when applied at ecologically meaningful time and frequency scales
Metcalf, O., Barlow, J., Devenish, C., Marsden, S. J., Berenguer, É., & Lees, A. C. (2020) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13521 · Cited by 111 · cited 4× · see citing works
“The intra-study variability of EAI has been attributed to various factors, such as the sensitivity to changes in environmental conditions, noise, and the sampling protocol Bradfer-Lawrence et al. (2019a); Metcalf et al. (2021a); Lillis and Mooney (2022); Zwerts et al. (2022).”
InfluentialUsing acoustic indices in ecology: Guidance on study design, analyses and interpretation
Bradfer‐Lawrence, T., Desjonqueres, C., Eldridge, A., Johnston, A., & Metcalf, O. (2023) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.14194 · Cited by 106 · see citing works
Determining Temporal Sampling Schemes for Passive Acoustic Studies in Different Tropical Ecosystems
Pieretti, N., Duarte, M. H. L., de Sousa-Lima, R. S., Rodrigues, M., Young, R. J., & Farina, A. (2015) · Tropical Conservation Science · doi:10.1177/194008291500800117 · Cited by 105 · cited 1× · see citing works
“However, this strategy does not consider differences between devices for frequencies above 500 Hz, so the variability due as frequency response persists Pieretti et al. (2015).”
13C metabolomics reveals widespread change in carbon fate during coral bleaching
Hillyer, K. E., Dias, D. A., Lutz, A., et al. (2017) · Metabolomics · doi:10.1007/s11306-017-1306-8 · Cited by 105 · see citing works
scikit‐maad: An open‐source and modular toolbox for quantitative soundscape analysis in Python
Ulloa, J. S., Haupert, S., Latorre, J. F., Aubin, T., & Sueur, J. (2021) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13711 · Cited by 100 · cited 1× · see citing works
“In this study, all indices were calculated using the Python scikit-maad library proposed by Ulloa et al. (2021).”
Comparison of audio recording system performance for detecting and monitoring songbirds
Rempel, R. S., Francis, C. M., Robinson, J. N., & Campbell, M. (2013) · Journal of Field Ornithology · doi:10.1111/jofo.12008 · Cited by 93 · cited 2× · see citing works
“Step-1: Signal resampling Sampling frequency differences are often disregarded, Adams et al. (2012); Rempel et al. (2013).”
de Sousa-Lima, R. S., Ferreira, L. S., de Oliveira, E. G., Lopes, L. C., de Brito, M. R. M., Baumgarten, J., & Rodrigues, F. H. G. (2018) · Journal of Ecoacoustics · doi:10.22261/jea.pvh6yz · Cited by 83 · see citing works
Utility of acoustic indices for ecological monitoring in complex sonic environments
Ross, S. R. P.-J., Friedman, N. R., Yoshimura, M., Yoshida, T., Donohue, I., & Economo, E. P. (2021) · Ecological Indicators · doi:10.1016/j.ecolind.2020.107114 · Cited by 78 · cited 1× · see citing works
“…although different works aim to identify the possible causes of this variability and propose alternatives to measure the effectiveness of EAI for characterizing various habitats, a straightforward standard implementation is needed (Ross et al., 2021; Zhao et al., 2019a; Jorge et al., 2018).”
The effectiveness of acoustic indices for forest monitoring in Atlantic rainforest fragments
Jorge, F., Machado, C. G., Nogueira, S. S. D. C., & Nogueira-Filho, S. L. G. (2018) · Ecological Indicators · doi:10.1016/j.ecolind.2018.04.001 · Cited by 74 · cited 1× · see citing works
“…although different works aim to identify the possible causes of this variability and propose alternatives to measure the effectiveness of EAI for characterizing various habitats, a straightforward standard implementation is needed (Ross et al., 2021; Zhao et al., 2019a; Jorge et al., 2018).”
Zhao, Z., Xu, Z., Bellisario, K., Zeng, R.-W., Li, N., Zhou, W.-Y., & Pijanowski, B. C. (2019) · Ecological Indicators · doi:10.1016/j.ecolind.2019.105588 · Cited by 72 · cited 2× · see citing works
“These indices quantify acoustic variability at the sampling sites, provide different information about the soundscape, and their use spans different habitats and environmental scales Zhao et al. (2019b).”
Eldridge, A., Casey, M. A., Moscoso, P., & Peck, M. R. (2016) · PeerJ · doi:10.7717/peerj.2108 · Cited by 68 · see citing works
Pérez‐Granados, C., Bota, G., Giralt, D., Albarracín, J., & Traba, J. (2019) · Ardeola · doi:10.13157/arla.66.2.2019.ra4 · Cited by 48 · cited 1× · see citing works
“As gleaned from the literature, intra-brand evaluations predominantly concentrate on diagnosing the issue instead of proposing an effective remedy to counteract the bias stemming from recording devices (Adams et al., 2012; Rempel et al., 2013; P´erez-Granados et al., 2019).”
An algorithm for the detection of peaks in biological signals.
Dumpala, S. R., Reddy, S., & Sarna, S. K. (1982) · Computer Programs in Biomedicine · doi:10.1016/0010-468x(82)90030-7 · Cited by 40 · cited 1× · see citing works
“Subsequently, traditional peak detection algorithms (such as proposed by Dumpala et al. (1982) or Sueur (2018)) are used to find prominent peaks.”
Tuneu-Corral, C., Puig-Montserrat, X., Flaquer, C., et al. (2020) · Ecological Indicators · doi:10.1016/j.ecolind.2019.105849 · Cited by 39 · cited 1× · see citing works
“Despite the importance of this issue, many works focus on the behavioral pattern of the index without considering its magnitude (Tuneu-Corral et al., 2020), losing reproducibility among studies.”
Shaw, T., Hedes, R., Sandstrom, A., Ruete, A., Hiron, M., Hedblom, M., Eggers, S., & Mikusiński, G. (2021) · Environmental and Sustainability Indicators · doi:10.1016/j.indic.2021.100141 · Cited by 39 · cited 1× · see citing works
“Due to their flexibility, EAI have been used for different aims, such as acoustic niches, uniformity, soundscape degradation, and environmental studies (Towsey et al., 2014; Shaw et al., 2021; Benocci et al., 2020).”
Eco-Acoustic Indices to Evaluate Soundscape Degradation Due to Human Intrusion
Benocci, R., Brambilla, G., Bisceglie, A., & Zambon, G. (2020) · Sustainability · doi:10.3390/su122410455 · Cited by 36 · cited 1× · see citing works
“Due to their flexibility, EAI have been used for different aims, such as acoustic niches, uniformity, soundscape degradation, and environmental studies (Towsey et al., 2014; Shaw et al., 2021; Benocci et al., 2020).”
Quantifying the Soundscape: How filters change acoustic indices
Hyland, E. B., Schulz, A., & Quinn, J. E. (2023) · Ecological Indicators · doi:10.1016/j.ecolind.2023.110061 · Cited by 36 · cited 2× · see citing works
“The literature review (see Appendix A) highlights ecoacoustic data variability reduction strategies, yet it predominantly focuses on intra-brand evaluations using filtering methods and sampling adjustments for EAI Hyland et al. (2023a).”
Farina, A., & Gage, S. H. (2017) · Wiley eBooks · doi:10.1002/9781119230724.ch1 · Cited by 33 · cited 1× · see citing works
“Ecological acoustic indices (EAI) are one of the most common metrics to study soundscapes as they are easy to calculate and are assumed to have biological interpretability (Buxton et al., 2018; Sueur and Farina, 2015; Farina and Gage, 2017).”
Comparing normalization methods and the impact of noise
Vu, T., Riekeberg, E., Qiu, Y., & Powers, R. (2018) · Metabolomics · doi:10.1007/s11306-018-1400-6 · Cited by 28 · cited 1× · see citing works
“Furthermore, RMS normalization does not limit the audio to a specific range, thus avoiding using other signal normalization techniques that could suppress relevant information (Vu et al., 2018).”
Exploring spatio-temporal variation in soundscape saturation of an African tropical forest landscape
Zwerts, J. A., Wiegers, J., Sterck, E. H. M., & van Kuijk, M. (2022) · Ecological Indicators · doi:10.1016/j.ecolind.2022.108712 · Cited by 24 · cited 1× · see citing works
“The intra-study variability of EAI has been attributed to various factors, such as the sensitivity to changes in environmental conditions, noise, and the sampling protocol Bradfer-Lawrence et al. (2019a); Metcalf et al. (2021a); Lillis and Mooney (2022); Zwerts et al. (2022).”
Lillis, A., & Mooney, T. A. (2022) · Frontiers in Marine Science · doi:10.3389/fmars.2022.960881 · Cited by 24 · cited 1× · see citing works
“The intra-study variability of EAI has been attributed to various factors, such as the sensitivity to changes in environmental conditions, noise, and the sampling protocol Bradfer-Lawrence et al. (2019a); Metcalf et al. (2021a); Lillis and Mooney (2022); Zwerts et al. (2022).”
A Resampling Method Based on Filter Designed by Window Function Considering Frequency Aliasing
Liu, H., Lin, J., Xu, S., et al. (2020) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2020.3016736 · Cited by 19 · cited 2× · see citing works
“To solve this issue, we employed the Kaiser resampling method proposed in (Liu et al., 2020) to equalize the sampling rates of the devices and we found that effectively decreased the recorder bias for ACIft and M.”
Cifuentes, E., Vélez, J., & Butler, S. J. (2021) · Biota Colombiana · doi:10.21068/c2021.v22n01a02 · Cited by 18 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Bradfer-Lawrence, T., Gardner, N., Bunnefeld, L., Bunnefeld, N., Willis, S.G., Dent, D.H., 2019b. Guidelines for the use of acoustic indices in environmental research. Methods in Ecology and Evolution 10, 1796-1807. doi:10.1111/2041-210X.13254. · DOI · Paper page
- Johansson, M., 1999. The hilbert transform. Mathematics Master's Thesis. Växjö University, Suecia. Disponible en internet: http://w3. msi. vxu. se/exarb/mj ex. pdf, consultado el 19.
Cited by
Sorted by citations
Biases in Ecoacoustics Analysis: A Protocol to Equalize Audio Recorders
Potenza, A., Zaffaroni-Caorsi, V., Benocci, R., Guagliumi, G., Fouani, J. M., Bisceglie, A., & Zambon, G. (2024) · Sensors · doi:10.3390/s24144642 · Cited by 8 · see citing works
Listening forward: emerging roles of bioacoustics in ecology, evolution, and conservation
Zsebők, S. (2026) · Biologia Futura · doi:10.1007/s42977-026-00322-5
A workflow to optimize spatial sampling in ecoacoustic studies
Martínez-Arias, V. M., Paniagua-Villada, C., Guerrero, M. J., et al. (2026) · Landscape Ecology · doi:10.1007/s10980-026-02372-5
Self-citation
- Authors
- Luna-Naranjo, D., Martínez, J. D., Sánchez-Giraldo, C., Daza, J. M., & López, J. D.
- Affiliations
- Universidad de Antioquia (CO)
- Corresponding author
- López, José D.
- Venue
- Ecological Informatics
- Publisher
- openRxiv
- Published
- 2023-10-19
- Volume
- 82
- Pages
- 102668-102668
- ISSN
- 1574-9541 (ISSN-L)
- OpenAlex
- W4399555689
- Cited by
- 7
- References
- 47
- Citations by year
- 2026: 4 · 2025: 2 · 2024: 1
- FWCI
- 3.01
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc
- Locations
- Ecological Informatics (publishedVersion, cc-by-nc)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Developmental Biology › Animal Vocal Communication and Behavior
- Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
- Health Sciences › Health Professions › Speech and Hearing › Noise Effects and Management
- Keywords
- soundscape analysis, audio recording devices, reproducibility, sampling frequency, frequency response, Kullback-Leibler divergence, AudioMoth, audio preprocessing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review