CoChat

Encourage Playing with Data and Discourage Questionable Reporting Practices

Open access
Wigboldus, Daniel H. J., Dotsch, Ron · 2016 · Psychometrika
Cited by 63

The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.

Abstract

Not so long ago, in the early days of experimental psychology, researchers performed their data analyses by hand (or paid people to do these calculations for them).No wonder, they took ample time beforehand to think through which analyses to perform (and especially which not).Exploring data was not really an option.The t value or F value that resulted from the calculations provided a more or less definitive answer.H0 could be rejected or not.During the past decades, we have quickly moved from punch cards, via expensive mainframes, to relatively cheap but powerful personal computers that are fulltime available to individual researchers and students.Nowadays, SPSS and other user-friendly statistical packages make it easy to run all types of analyses, which encourage analyzing your data in every possible way.It is stunningly easy to (re-)run analyses with or without a specific group of participants, with or without an additional between-subject condition, with or without an extra variable in a scale.Indeed, it may be a waste of resources to leave a dataset partly unexplored when SPSS offers all the tools to do so.In many research labs, analyzing results has shifted from spending days to do confirmatory analyses by hand, to spending days engaging in all kinds of data analyses to see "what's in the data."Is this a bad thing?We do not think so, necessarily.Given that you know what you are doing in SPSS, you can learn a lot from exploring a dataset.However, there clearly are risks (see Simmons, Nelson & Simonsohn, 2011), which we will revisit later.This contribution is a commentary on "Playing with Data-Or How to Discourage Questionable Research Practices and Stimulate Researchers to Do Things Right" by Sijtsma (2015).In his paper, Klaas Sijtsma makes the important point that prevention of questionable research practices may be more important than detection.He suggests two policy measures.First, make research data and research materials publicly available.Second, encourage researchers to consult methodologists or statisticians for help and a second opinion.In this commentary, we will not reiterate the points made by Klaas Sijtsma.We support his analysis and his policy recommendations.Making research data and research materials publicly available and consulting methodologists and statisticians will help in increasing the transparency of the data analysis phase of the empirical research cycle within psychology.Currently, in too many cases, this phase is the exclusive domain of (small groups of) individual researchers and their personal computers.However, making the data analysis phase transparent to others will not prevent scientific fraud.A person who willingly aims to fool others always will be able to do so, also when sharing (well-faked) data.Nevertheless, it will help in reducing the errors researchers

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s11336-015-9445-1

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 5 references

Cited by

See all 63

Metadata

Authors
Wigboldus, D. H. J., & Dotsch, R.
Venue
Psychometrika
Published
2016-03
Volume
81
Issue
1
Pages
27-32
ISSN
  • 0033-3123 (ISSN-L)
  • 1860-0980
OpenAlex
W2065437013
Cited by
63
References
5
FWCI
29.57
Type
journal article
Language
en
Open access
hybrid

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Encourage Playing with Data and Discourage Questionable Reporting Practices. https://app.cochat.ai/papers/10.1007/s11336-015-9445-1
The paper
Wigboldus, D. H. J., & Dotsch, R. (2016). Encourage Playing with Data and Discourage Questionable Reporting Practices. Psychometrika. https://doi.org/10.1007/s11336-015-9445-1

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.