The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Presentation and response timing accuracy in Adobe Flash and HTML5/JavaScript Web experiments
Open accessAbstract
Web-based research is becoming ubiquitous in the behavioral sciences, facilitated by convenient, readily available participant pools and relatively straightforward ways of running experiments: most recently, through the development of the HTML5 standard. Although in most studies participants give untimed responses, there is a growing interest in being able to record response times online. Existing data on the accuracy and cross-machine variability of online timing measures are limited, and generally they have compared behavioral data gathered on the Web with similar data gathered in the lab. For this article, we took a more direct approach, examining two ways of running experiments online-Adobe Flash and HTML5 with CSS3 and JavaScript-across 19 different computer systems. We used specialist hardware to measure stimulus display durations and to generate precise response times to visual stimuli in order to assess measurement accuracy, examining effects of duration, browser, and system-to-system variability (such as across different Windows versions), as well as effects of processing power and graphics capability. We found that (a) Flash and JavaScript's presentation and response time measurement accuracy are similar; (b) within-system variability is generally small, even in low-powered machines under high load; (c) the variability of measured response times across systems is somewhat larger; and (d) browser type and system hardware appear to have relatively small effects on measured response times. Modeling of the effects of this technical variability suggests that for most within- and between-subjects experiments, Flash and JavaScript can both be used to accurately detect differences in response times across conditions. Concerns are, however, noted about using some correlational or longitudinal designs online.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Conducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172
Evaluating amazon’s mechanical turk as a tool for experimental behavioral research
Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M. (2013) · PLoS ONE · doi:10.1371/journal.pone.0057410 · Cited by 1,617
Task Switching Across the Life Span: Effects of Age on General and Specific Switch Costs.
Reimers, S., & Maylor, E. A. (2005) · Developmental Psychology · doi:10.1037/0012-1649.41.4.661 · Cited by 273 · cited 2×
“In one of the earliest examples, Reimers and Maylor (2005) used Adobe Flash to examine the effects of age on task-switching performance, finding largely similar results to those from laboratory studies, where data existed.”
ResultSelf-citationPlant, R. R., & Quinlan, P. T. (2013) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-013-0166-6 · Cited by 61
Response time accuracy in Apple Macintosh computers
Neath, I., Earle, A., Hallett, D., & Surprenant, A. M. (2011) · Behavior Research Methods · doi:10.3758/s13428-011-0069-9 · Cited by 57
Cited by
Gorilla in our midst: An online behavioral experiment builder
Anwyl-Irvine, A. L., Massonnié, J., Flitton, A., Kirkham, N., & Evershed, J. K. (2020) · Behavior Research Methods · doi:10.3758/s13428-019-01237-x · Cited by 1,862
The timing mega-study: comparing a range of experiment generators, both lab-based and online
Bridges, D., Pitiot, A., MacAskill, M. R., & Peirce, J. W. (2020) · PeerJ · doi:10.7717/peerj.9414 · Cited by 546
Realistic precision and accuracy of online experiment platforms, web browsers, and devices
Anwyl-Irvine, A., Dalmaijer, E. S., Hodges, N., & Evershed, J. K. (2021) · Behavior Research Methods · doi:10.3758/s13428-020-01501-5 · Cited by 341
Survey-software implicit association tests: A methodological and empirical analysis
Carpenter, T. P., Pogacar, R., Pullig, C., Kouril, M., Aguilar, S., LaBouff, J., Isenberg, N., & Chakroff, A. (2019) · Behavior Research Methods · doi:10.3758/s13428-019-01293-3 · Cited by 310
Conducting perception research over the internet: A tutorial review
Woods, A. T., Velasco, C., Levitan, C. A., Wan, X., & Spence, C. (2015) · PeerJ · doi:10.7717/peerj.1058 · Cited by 301
Metadata
- Authors
- Reimers, S., & Stewart, N.
- Venue
- Behavior Research Methods
- Published
- 2014-06-06
- Volume
- 47
- Issue
- 2
- Pages
- 309-327
- ISSN
- 1554-351X (ISSN-L)
- 1554-3528
- OpenAlex
- W2141185752
- Cited by
- 194
- References
- 5
- FWCI
- 16.45
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Presentation and response timing accuracy in Adobe Flash and HTML5/JavaScript Web experiments. https://app.cochat.ai/papers/10.3758/s13428-014-0471-1
- The paper
- Reimers, S., & Stewart, N. (2015). Presentation and response timing accuracy in Adobe Flash and HTML5/JavaScript Web experiments. Behavior Research Methods. https://doi.org/10.3758/s13428-014-0471-1
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.