Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 12

The DOI matches its Crossref record.

Use-Specific Considerations for Optimising Data Quality Trade-Offs in Citizen Science: Recommendations from a Targeted Literature Review to Improve the Usability and Utility for the Calibration and Validation of Remotely Sensed Products

Schacher, Alice, Roger, Erin, Williams, Kristen J., Stenson, Matthew P., Sparrow, Ben, Lacey, Justine·2023·Remote Sensing
DOI 10.3390/rs15051407
Open this paper in CoChat — free account

Abstract

The growth of citizen science presents a valuable potential source of calibration and validation data for environmental remote sensing at greater spatial and temporal scales, and with greater cost efficiency than is achievable by professional in situ reference-data collection alone. However, the frequent mismatch between in situ data-quality requirements for remote-sensing-product development and current data quality assurance in citizen science presents a significant challenge if widespread use of these complementary data sources is to be achieved. To evaluate the scope of this challenge, we conducted a targeted literature review into the nature of data-quality issues faced by citizen-science projects for routine incorporation into terrestrial environmental-monitoring systems. From the literature, we identify the challenges and trade-offs to inform best-practice implementation of data quality assurance in citizen-science projects. To assist practitioners in implementing our findings, we grouped these themes by stage of citizen-science project: (1) program planning and design; (2) participant engagement; (3) data collection; and (4) data processing. As a final step, we used our findings as the basis to formulate guiding questions that can be used to inform decision making when choosing optimal data-quality-improvement and assurance strategies for use of citizen science in remote-sensing calibration and/or validation. Our aim is to enhance future development of citizen-science projects for use with remote sensing in environmental monitoring.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/rs15051407

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 65

  1. Silvertown, 2009, A new dawn for citizen science, Trends Ecol. Evol., 24, 467, 10.1016/j.tree.2009.03.017 · DOI
  2. Fraisl, 2022, Citizen science in environmental and ecological sciences, Nat. Rev. Methods Prim., 2, 64, 10.1038/s43586-022-00144-4 · DOI
  3. Adler, 2020, Citizen science in ecology: A place for humans in nature, Ann. N. Y. Acad. Sci., 1469, 52, 10.1111/nyas.14340 · DOI
  4. Chandler, 2017, Contribution of citizen science towards international biodiversity monitoring, Biol. Conserv., 213, 280, 10.1016/j.biocon.2016.09.004 · DOI
  5. Roger, 2020, Maximising the potential for citizen science in New South Wales, Aust. Zool., 40, 449, 10.7882/AZ.2019.023 · DOI
  6. Fritz, S., Fonte, C.C., and See, L. (2017). The role of citizen science in earth observation. Remote Sens., 9. · DOI
  7. Kosmala, M., Crall, A., Cheng, R., Hufkens, K., Henderson, S., and Richardson, A.D. (2016). Season spotter: Using citizen science to validate and scale plant phenology from near-surface remote sensing. Remote Sens., 8. · DOI
  8. Kosmala, 2016, Assessing data quality in citizen science, Front. Ecol. Environ., 14, 551, 10.1002/fee.1436 · DOI · Paper page
  9. Comber, 2013, Using control data to determine the reliability of volunteered geographic information about land cover, Int. J. Appl. Earth Obs. Geoinf., 23, 37
  10. Dubovyk, 2017, The role of remote sensing in land degradation assessments: Opportunities and challenges, Eur. J. Remote Sens., 50, 601, 10.1080/22797254.2017.1378926 · DOI
  11. Mayr, S., Kuenzer, C., Gessner, U., Klein, I., and Rutzinger, M. (2019). Validation of earth observation time-series: A review for large-area and temporally dense land surface products. Remote Sens., 11. · DOI
  12. White, 2023, Towards a continent-wide ecological site-condition dataset using calibrated subjective evaluations, Ecol. Appl., 33, e2729, 10.1002/eap.2729 · DOI
  13. Roelfsema, 2010, Integrating field data with high spatial resolution multispectral satellite imagery for calibration and validation of coral reef benthic community maps, J. Appl. Remote Sens., 4, 043527, 10.1117/1.3430107 · DOI
  14. Sparrow, 2020, A vegetation and soil survey method for surveillance monitoring of rangeland environments, Front. Ecol. Evol., 8, 157, 10.3389/fevo.2020.00157 · DOI
  15. Sparrow, 2020, Effective ecosystem monitoring requires a multi-scaled approach, Biol. Rev., 95, 1706, 10.1111/brv.12636 · DOI
  16. Fischer, 2021, Going beyond hooked participants: The nibble-and-drop framework for classifying citizen science participation, Citiz. Sci. Theory Pract., 6, 10, 10.5334/cstp.350 · DOI
  17. Kohl, 2021, GLOBE observer and the GO on a trail data challenge: A citizen science approach to generating a global land cover land use reference dataset, Front. Clim., 3, 620497, 10.3389/fclim.2021.620497 · DOI
  18. Fritz, 2009, Geo-wiki.org: The use of crowdsourcing to improve global land cover, Remote Sens., 1, 345, 10.3390/rs1030345 · DOI · Paper page
  19. Clark, 2011, Virtual interpretation of earth web-interface tool (VIEW-IT) for collecting land-use/land-cover reference data, Remote Sens., 3, 601, 10.3390/rs3030601 · DOI
  20. Conrad, 2011, A review of citizen science and community-based environmental monitoring: Issues and opportunities, Environ. Monit. Assess., 176, 273, 10.1007/s10661-010-1582-5 · DOI · Paper page
  21. Vohland, K., Land-Zandstra, A., Ceccaroni, L., Lemmens, R., Perelló, J., Ponti, M., Samson, R., and Wagenknecht, K. (2021). The Science of Citizen Science, Springer International Publishing. · DOI
  22. Hunter, 2013, Assessing the quality and trustworthiness of citizen science data, Concurr. Comput. Pract. Exp., 25, 454, 10.1002/cpe.2923 · DOI
  23. Baker, 2021, The verification of ecological citizen science data: Current approaches and future possibilities, Citiz. Sci. Theory Pract., 6, 1
  24. Corbari, 2022, Citizen science supporting agricultural monitoring with hundreds of low-cost sensors in comparison to remote sensing data, Eur. J. Remote Sens., 55, 388, 10.1080/22797254.2022.2084643 · DOI
  25. Mengersen, 2021, Understanding the reliability of citizen science observational data using item response models, Methods Ecol. Evol., 12, 1533, 10.1111/2041-210X.13623 · DOI
  26. Ceccaroni, 2019, Opportunities and Risks for Citizen Science in the Age of Artificial Intelligence, Citiz. Sci. Theory Pract., 4, 29, 10.5334/cstp.241 · DOI
  27. Vasiliades, M.A., Hadjichambis, A.C., Paraskeva-Hadjichambi, D., Adamou, A., and Georgiou, Y. (2021). A Systematic Literature Review on the Participation Aspects of Environmental and Nature-Based Citizen Science Initiatives. Sustainability, 13. · DOI
  28. Goodchild, 2007, Citizens as sensors: The world of volunteered geography, GeoJournal, 69, 211, 10.1007/s10708-007-9111-y · DOI
  29. Gilfedder, 2019, Brokering trust in citizen science, Soc. Nat. Resour., 32, 292, 10.1080/08941920.2018.1518507 · DOI
  30. Tarko, 2021, Producing consistent visually interpreted land cover reference data: Learning from feedback, Int. J. Digit. Earth, 14, 52, 10.1080/17538947.2020.1729878 · DOI
  31. Gengler, S., and Bogaert, P. (2016). Integrating crowdsourced data with a land cover product: A bayesian data fusion approach. Remote Sens., 8. · DOI
  32. Grainger, A. (2017). Citizen observatories and the new earth observation science. Remote Sens., 9. · DOI
  33. Cruickshank, 2019, Quantifying data quality in a citizen science monitoring program: False negatives, false positives and occupancy trends, Conserv. Sci. Pract., 1, e54, 10.1111/csp2.54 · DOI
  34. Vermeiren, 2016, Hierarchical toolbox: Ensuring scientific accuracy of citizen science for tropical coastal ecosystems, Ecol. Indic., 66, 242, 10.1016/j.ecolind.2016.01.031 · DOI
  35. Stenglein, 2019, Tradeoffs and tools for data quality, privacy, transparency, and trust in citizen science, Biol. Conserv., 238, 108195, 10.1016/j.biocon.2019.108195 · DOI
  36. Rasmussen, 2019, Citizen science ethics, Citiz. Sci. Theory Pract., 4, 5, 10.5334/cstp.235 · DOI
  37. Welvaert, 2016, Citizen surveillance for environmental monitoring: Combining the efforts of citizen science and crowdsourcing in a quantitative data framework, SpringerPlus, 5, 1890, 10.1186/s40064-016-3583-5 · DOI
  38. Lewandowski, 2015, Influence of volunteer and project characteristics on data quality of biological surveys, Conserv. Biol., 29, 713, 10.1111/cobi.12481 · DOI
  39. Clare, 2019, Making inference with messy (citizen science) data: When are data accurate enough and how can they be improved?, Ecol. Appl., 29, e01849, 10.1002/eap.1849 · DOI
  40. Foody, 2015, An assessment of citizen contributed ground reference data for land cover map accuracy assessment, ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci., II-3/W5, 219, 10.5194/isprsannals-II-3-W5-219-2015 · DOI
  41. Lukyanenko, 2020, Citizen science: An information quality research frontier, Inf. Syst. Front., 22, 961, 10.1007/s10796-019-09915-z · DOI
  42. Adeleye, 2017, The accuracy of citizen science data: A quantitative review, Bull. Ecol. Soc. Am., 98, 278, 10.1002/bes2.1336 · DOI
  43. Mesaglio, 2021, An overview of the history, current contributions and future outlook of iNaturalist in Australia, Wildl. Res., 48, 289, 10.1071/WR20154 · DOI
  44. Tredick, 2017, A rubric to evaluate citizen-science programs for long-term ecological monitoring, BioScience, 67, 834, 10.1093/biosci/bix090 · DOI
  45. Ogunseye, S., and Parsons, J. (2018, January 11–15). Designing for information quality in the era of repurposable crowdsourced user-generated content. Proceedings of the Lecture Notes in Business Information Processing, Tallinn, Estonia. · DOI
  46. Rotman, D., Preece, J., Hammock, J., Procita, K., Hansen, D., Parr, C., Lewis, D., and Jacobs, D. (2012, January 11–15). Dynamic changes in motivation in collaborative citizen-science projects. Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work, Seattle, WA, USA. · DOI
  47. Lukyanenko, 2016, Emerging problems of data quality in citizen science, Conserv. Biol., 30, 447, 10.1111/cobi.12706 · DOI
  48. Dickinson, 2012, The current state of citizen science as a tool for ecological research and public engagement, Front. Ecol. Environ., 10, 291, 10.1890/110236 · DOI · Paper page
  49. Crall, 2011, Assessing citizen science data quality: An invasive species case study, Conserv. Lett., 4, 433, 10.1111/j.1755-263X.2011.00196.x · DOI
  50. Sun, 2021, Citizen science data collection for integrated wildlife population analyses, Front. Ecol. Evol., 9, 682124, 10.3389/fevo.2021.682124 · DOI
  51. Kennedy, 1987, Some considerations for the use of visual estimates of plant cover in biomonitoring, J. Ecol., 75, 151, 10.2307/2260541 · DOI
  52. Iwao, 2006, Validating land cover maps with degree confluence project information, Geophys. Res. Lett., 33, L23404, 10.1029/2006GL027768 · DOI
  53. Fink, 2014, Crowdsourcing meets ecology: Hemispherewide spatiotemporal species distribution models, AI Mag., 35, 19
  54. Kelling, 2015, Taking a ‘big data’ approach to data quality in a citizen science project, Ambio, 44, 601, 10.1007/s13280-015-0710-4 · DOI
  55. Geldmann, 2016, What determines spatial bias in citizen science? Exploring four recording schemes with different proficiency requirements, Divers. Distrib., 22, 1139, 10.1111/ddi.12477 · DOI · Paper page
  56. Tokmakoff, 2016, AusPlots Rangelands field data collection and publication: Infrastructure for ecological monitoring, Future Gener. Comput. Syst., 56, 537, 10.1016/j.future.2015.08.016 · DOI
  57. Hochachka, 2012, Data-intensive science applied to broad-scale citizen science, Trends Ecol. Evol., 27, 130, 10.1016/j.tree.2011.11.006 · DOI
  58. Kelling, S., Yu, J., Gerbracht, J., and Wong, W.K. (2011, January 5–8). Emergent filters: Automated data verification in a large-scale citizen science project. Proceedings of the 7th IEEE International Conference on e-Science Workshops, eScienceW, Stockholm, Sweden. · DOI
  59. Robinson, 2020, Integrating citizen science data with expert surveys increases accuracy and spatial extent of species distribution models, Divers. Distrib., 26, 976, 10.1111/ddi.13068 · DOI
  60. Swanson, 2016, A generalized approach for producing, quantifying, and validating citizen science data from wildlife images, Conserv. Biol., 30, 520, 10.1111/cobi.12695 · DOI · Paper page
  61. See, 2022, Lessons learned in developing reference data sets with the contribution of citizens: The Geo-Wiki experience, Environ. Res. Lett., 17, 065003, 10.1088/1748-9326/ac6ad7 · DOI
  62. Vahidi, 2018, Trust as a proxy indicator for intrinsic quality of volunteered geographic information in biodiversity monitoring programs, GIScience Remote Sens., 55, 502, 10.1080/15481603.2017.1413794 · DOI
  63. Fonte, 2015, Usability of VGI for validation of land cover maps, Int. J. Geogr. Inf. Sci., 29, 1269, 10.1080/13658816.2015.1018266 · DOI
  64. Elmore, A.J., Stylinski, C.D., and Pradhan, K. (2016). Synergistic use of citizen science and remote sensing for continental-scale measurements of forest tree phenology. Remote Sens., 8. · DOI
  65. Bayraktarov, 2019, Do big unstructured biodiversity data mean more knowledge?, Front. Ecol. Evol., 6, 239, 10.3389/fevo.2018.00239 · DOI

Cited by

See all 12

Metadata

Authors
Schacher, A., Roger, E., Williams, K. J., Stenson, M. P., Sparrow, B., & Lacey, J.
Affiliations
  • Commonwealth Scientific & Industrial Research Organisation (CSIRO), Canberra, ACT 2601, Australia
  • The Terrestrial Ecosystem Research Network, Adelaide, SA 5005, Australia
ORCID
Venue
Remote Sensing
Publisher
MDPI AG
Published
2023-03-02
Volume
15
Issue
5
Pages
1407
ISSN
2072-4292
DOI
10.3390/rs15051407
Cited by
12
References
0
Language
en
Licence
creativecommons.org/licenses/by/4.0/ (vor)
Funders
CSIRO’s Responsible Innovation Future Science Platform

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Use-Specific Considerations for Optimising Data Quality Trade-Offs in Citizen Science: Recommendations from a Targeted Literature Review to Improve the Usability and Utility for the Calibration and Validation of Remotely Sensed Products. https://app.cochat.ai/papers/10.3390/rs15051407
The paper
Schacher, A., Roger, E., Williams, K. J., Stenson, M. P., Sparrow, B., & Lacey, J. (2023). Use-Specific Considerations for Optimising Data Quality Trade-Offs in Citizen Science: Recommendations from a Targeted Literature Review to Improve the Usability and Utility for the Calibration and Validation of Remotely Sensed Products. Remote Sensing. https://doi.org/10.3390/rs15051407

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.