CoChat
Sign Up
Cited by 129Mobile Crowdsensing and CrowdsourcingOpen Source Software InnovationsDigital Mental Health Interventions

The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.

Crowdsourcing for Cognitive Science – The Utility of Smartphones

Open access
Brown, Harriet R., Zeidman, Peter, Smittenaar, Peter, Adams, Rick A., McNab, Fiona, Rutledge, Robb B., Dolan, Raymond J.·2014·PLoS ONE
DOI 10.1371/journal.pone.0100662PMC PMC4099129
Open this paper in CoChat — free accountOpen PDF

Abstract

By 2015, there will be an estimated two billion smartphone users worldwide. This technology presents exciting opportunities for cognitive science as a medium for rapid, large-scale experimentation and data collection. At present, cost and logistics limit most study populations to small samples, restricting the experimental questions that can be addressed. In this study we investigated whether the mass collection of experimental data using smartphone technology is valid, given the variability of data collection outside of a laboratory setting. We presented four classic experimental paradigms as short games, available as a free app and over the first month 20,800 users submitted data. We found that the large sample size vastly outweighed the noise inherent in collecting data outside a controlled laboratory setting, and show that for all four games canonical results were reproduced. For the first time, we provide experimental validation for the use of smartphones for data collection in cognitive science, which can lead to the collection of richer data sets and a significant cost reduction as well as provide an opportunity for efficient phenotypic screening of large populations.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0100662

Cites

32 of 32 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Screenshot of the app, while playing the stop-signal reaction time game. Fruit fell from the tree and participants were asked to tap simultaneously on both sides of the screen as the fruit passed through the circles. If a piece of fruit turned brown during its fall, participants had to inhibit their response on that side.
Figure 1 Screenshot of the app, while playing the stop-signal reaction time game. Fruit fell from the tree and participants were asked to tap simultaneously on both sides of the screen as the fruit passed through the circles. If a piece of fruit turned brown during its fall, participants had to inhibit their response on that side.CC BY · PubMed Central
Figure 2 Demographic characteristics of app users. (a) Gender and age breakdown. Young women were the primary app users. (b) Location. Most users originated from outside the UK, where at app was developed, and users from the UK were not concentrated in any single region. (c) The app reached participants with higher education degrees as well as those without. Only participants over 25 years of age were included in this analysis as those younger than 25 may not have completed their education. (d) Life satisfaction rated on a scale from 0–10. This information was recorded for follow-up analyses in relation to the decision-making task and is not further analysed here.
Figure 2 Demographic characteristics of app users. (a) Gender and age breakdown. Young women were the primary app users. (b) Location. Most users originated from outside the UK, where at app was developed, and users from the UK were not concentrated in any single region. (c) The app reached participants with higher education degrees as well as those without. Only participants over 25 years of age were included in this analysis as those younger than 25 may not have completed their education. (d) Life satisfaction rated on a scale from 0–10. This information was recorded for follow-up analyses in relation to the decision-making task and is not further analysed here.CC BY · PubMed Central
Figure 3 Data from the games in this app. (a) Working memory performance in ‘no distraction’ (remember red circles) and ‘distraction’ (remember red circles and ignore yellow circles) conditions, for younger and older participants. Performance deteriorated with age and distraction, and distraction had a more detrimental effect for older compared to younger adults. (b) Stop-signal reaction time, which measures inhibitory ability, could be estimated from participant's data. (c) Probability of successful identification of T2 in the attentional blink task, for each ISI and lag. T2 recognition was significantly impaired 150–500 ms after T1 presentation, but preserved T2 recognition at lag 1 (‘Lag-1 sparing’) was noted for shorter ISIs.
Figure 3 Data from the games in this app. (a) Working memory performance in ‘no distraction’ (remember red circles) and ‘distraction’ (remember red circles and ignore yellow circles) conditions, for younger and older participants. Performance deteriorated with age and distraction, and distraction had a more detrimental effect for older compared to younger adults. (b) Stop-signal reaction time, which measures inhibitory ability, could be estimated from participant's data. (c) Probability of successful identification of T2 in the attentional blink task, for each ISI and lag. T2 recognition was significantly impaired 150–500 ms after T1 presentation, but preserved T2 recognition at lag 1 (‘Lag-1 sparing’) was noted for shorter ISIs.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 31 references

Cited by

Sorted by citations

Related

See all

Metadata

Authors
Brown, H. R., Zeidman, P., Smittenaar, P., Adams, R. A., McNab, F., Rutledge, R. B., & Dolan, R. J.
Affiliations
University of Oxford (GB)
Corresponding author
Brown, Harriet R.
ORCID
Venue
PLoS ONE
Publisher
Public Library of Science (PLoS)
Published
2014-07-15
Volume
9
Issue
7
Pages
e100662
ISSN
1932-6203 (ISSN-L)
DOI
10.1371/journal.pone.0100662
PMC
PMC4099129
OpenAlex
W2044432453
Cited by
129
References
31
Citations by year
2026: 5 · 2025: 4 · 2024: 7 · 2023: 10 · 2022: 8 · 2021: 16 · 2020: 14 · 2019: 19 · 2018: 13 · 2017: 6 · 2016: 15 · 2015: 11 · 2014: 1
FWCI
16.39
Type
journal article
Language
en
Update notices
correction (2014-10-14)
Open access
Licence
Locations
  • PLoS ONE (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • UCL Discovery (University College London) (submittedVersion)
  • University of Birmingham Research Portal (University of Birmingham) (submittedVersion, cc-by)
  • University of Birmingham Research Portal (University of Birmingham) (submittedVersion, other-oa)
  • MPG.PuRe (Max Planck Society) (submittedVersion, cc-by)
Funders
Wellcome Trust (091593, 091593/Z/10/, 098362, 098362/Z/12/Z, 091593/Z/10/Z, 101252/Z/13/Z)
Topics
  • Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
  • Physical Sciences › Computer Science › Computer Science Applications › Open Source Software Innovations
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
Keywords
crowdsourcing, smartphones, mobile data collection, cognitive science, large-scale experimentation, data validity, reproducibility, experimental paradigms, mobile application, sample size, phenotypic screening, cost reduction
MeSH terms
Adolescent, Adult, Age Factors, Aged, Decision Making, Female, Humans, Male, Memory, Short-Term, Middle Aged, Cognitive Science, Cell Phone, Attentional Blink, Young Adult, Wireless Technology, Crowdsourcing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review