Data-Intensive Science and Research Integrity

May combine two claims.

Before 2000, fewer than five ORI cases per year involved image manipulation, whereas after 2000 the ORI handled about twenty-five such cases per year.

“Prior to 2000, less than 5 cases per year that went to ORI involved image manipulation; after 2000, the ORI has dealt with approximately 25 such cases per year.”

May combine two claims.

Between 1994 and 2000, the ORI issued four misconduct findings related to image manipulation, and between 2001 and 2007 it issued fifteen such findings.

“From 1994 to 2000, the ORI made only 4 findings of misconduct related to image manipulation; from 2001 to 2007, it made 15 of these findings (Parrish and Noonan 2009).”

• cites Parrish and Noonan 2009Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Data-intensive research is important in many disciplines, including physics, astronomy, cancer therapy, drug discovery, radiology, ecology, climatology, geology, molecular biology, systems biology, economics, and social science.

“Data-intensive research now plays an important role in many disciplines, including particle physics, astronomy, cancer therapy, drug discovery, radiology, ecology, climatology, geology, molecular biology, molecular biology, systems biology, economics, and social science (Elliott et al 2016, Sugimoto et al 2016).”

• cites Elliott et al 2016Not checked yet

  Not checked in the free run.

• cites Sugimoto et al 2016 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In data-intensive science, researchers commonly use computer-encoded statistical methods to search for patterns and associations in large datasets.

“In data-intensive science, investigators often use statistical methods encoded in computer algorithms to search for patterns and associations inone or many largedatasets, which may contain billions of terabytes of information, millions of data-points, and hundreds of variables (Ekbia et al 2015).”

• cites Ekbia et al 2015Not checked yet

  Not checked in the free run.

May combine two claims.

Contemporary data-intensive science differs from earlier work by using new computational methods and technologies and by often involving large, interdisciplinary teams.

“Nevertheless, contemporary data-intensive science is distinctive because it incorporates new computational methods and technologies and tends to involve large, interdisciplinary scientific teams (Leonelli 2014; Strasser 2012).”

• cites Leonelli 2014Not checked yet

  Not checked in the free run.

• cites Strasser 2012Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Data-intensive science is among the most important current research trends.

“The growing importance of data-intensive science (a.k.a. “big data”) is one of the most significant trends in contemporary research (Bell et al 2009, Miller 2010, Elliott et al 2016, Sigumoto et al 2016).”

• cites Bell et al 2009Not checked yet

  Not checked in the free run.

• cites Miller 2010Not checked yet

  Not checked in the free run.

• cites Elliott et al 2016Not checked yet

  Not checked in the free run.

May combine two claims.

In 1981, Al Gore Jr. held hearings on fraud in federally funded biomedical research and found that government agencies and universities lacked effective policies.

“In 1981, Representative Al Gore Jr. held hearings on fraud in federally-funded biomedical research and discovered that neither government agencies nor universities had effective policies for dealing with such problems (Steneck 1999).”

• cites Steneck 1999Not checked yet

  Not checked in the free run.

May combine two claims.

Babbage's ideas were not adopted as official government policies but influenced thinking about honesty in science.

“While Babbage’s ideas were never adopted as official government policies, they haveinfluencedhow scientists and policymakers have thought about honesty in science (Broad and Wade 1993).”

• cites Broad and Wade 1993Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In 2000, the Office of Science and Technology Policy adopted a research misconduct policy applying to all federal agencies, and institutions receiving federal funds must follow it.

“In 2000, the Office of Science and Technology Policy (OSTP) adopted a research misconduct policy that applies to all federal agencies. Institutions that receive federal funds for research are required to follow this policy (Shamoo and Resnik 2015).”

• cites Shamoo and Resnik 2015Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Duke officials told Perez that his criticism of Potti's work was a difference of scientific opinion and did not meet the definition of research misconduct.

“He brought these concerns to Duke University officials, who suggested thathis criticism of Potti’s work amounted to a difference of scientific opinion and did not meet the definition of research misconduct (Goldberg 2015).”

• cites Goldberg 2015Not checked yet

  Not checked in the free run.

May combine two claims.

Most misconduct definitions adopted by countries, research institutions, and journal associations include FFP, and some include additional categories such as serious deviations, inappropriate authorship, peer-review confidentiality violations, or undisclosed conflicts of interest.

“While most of these include FFP, some encompass other categories, such asserious deviations, inappropriate authorship, violating confidentiality of peer review, or failing to disclose conflicts of interest (Resnik et al 2015b).”

• cites Resnik et al 2015bNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The U.K.'s misconduct definition includes FFP and misrepresentation, defined as suppression or deliberate or negligent misrepresentation of findings or data.

“The U.K.’s definition includes FFP as well as misrepresentation, which is defined as “suppression or deliberate or negligent misrepresentation of findings and/or data (United Kingdom Research Council 2012).”

• cites United Kingdom Research Council 2012Not checked yet

  Not checked in the free run.

May combine two claims.

Deceptive data analysis could occur if researchers deliberately use an inappropriate statistical test or data-mining technique to boost support for a preferred conclusion, overstate statistical significance, or omit evidence conflicting with the presented analysis.

“Deceptive data-analysis could occur if researchers deliberately use an inappropriate statistical test or data-mining technique to boost support for a preferred conclusion, if they overstate the statistical significance of their results (Resnik 2000, Fan et al 2014), or if they leave out evidence or results that conflict with parts of the presented analysis.”

• cites Resnik 2000Not checked yet

  Not checked in the free run.

• cites Fan et al 2014Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Government agencies, academic institutions, and scientific journals have adopted their own misconduct definitions for compliance and oversight.

“Government agencies, academic institutions, and scientific journals have adopted their own definitions of misconduct for compliance and oversight purposes.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

This type of deception might go unnoticed because most studies have valid data processing criteria and procedures that are difficult to decipher.

“This type of deception might go unnoticed, since most studies have valid data processing criteria and procedures which can be difficult to decipher.”

• AssumptionRests on an assumption: Most studies have valid data processing criteria and procedures that are difficult to decipher.

May combine two claims.

Data-intensive science existed when U.S. policies were developed but was smaller in scale, and the technological capacity and computational expertise for processing and analyzing large amounts of information were not yet developed.

“Though data-intensive science was occurring (for example, the Human Genome Project began in 1990), it was on asmaller scale than it is today, and the technological capacity and computational expertise required to process and analyze large amounts of information had not yet been developed.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

OSTP officials worried that including the other serious deviations category could encourage allegations unrelated to truthfulness and deter innovative or unorthodox research.

“They were concerned that including this category in the definition of misconductmight 1) encourageallegations which have little to do withtruthfulness in research, such as accusations of sexual harassment or financial mismanagement, and 2) deter scientists from conducting important research that uses innovative or unorthodox methods or concepts because they would fear that they could be accused of”

• cites National Academy of Sciences 1992Not checked yet

  Not checked in the free run.

• cites Dooley and Kerch 2000Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Deceptive practices related to data processing, analysis, or interpretation are not unique to data-intensive science.

“While these forms of deception are not unique to data-intensive science, they become more difficult to accurately detect when working with large volumes of data.”

• AssumptionRests on an assumption: Deceptive practices related to data processing, analysis, or interpretation are not unique to data-intensive science.

May combine two claims.

Deception could occur if researchers overstate the scientific or practical significance of findings or claim causal connections when they have only shown statistical associations.

“Deception could occur if researchers overstate the scientific or practical significance of their findings or claim to have demonstrated causal connections when they have only produced evidence of statistical associations (Resnik 2000, Master and Resnik 2013).”

• cites Resnik 2000Not checked yet

  Not checked in the free run.

• cites Master and Resnik 2013Not checked yet

  Not checked in the free run.

May combine two claims.

The federal policy focused on fabrication, falsification, and plagiarism and removed the category of other serious deviations from accepted research practices.

“The federal policy focused on three unethical behaviors—fabrication, falsification, and plagiarism (FFP)—and eliminated a category previously used by the PHS and NSF, “other serious deviations from accepted research practices”(Dooley and Kerch 2000).”

• cites Dooley and Kerch 2000Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Australia's misconduct definition includes deception in proposing, carrying out, or reporting research.

“For example, Australia defines misconduct as “fabrication, falsification, plagiarism, or deception in proposing, carrying out or reporting the results of research (Australian Government 2007).”

• cites Australian Government 2007Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

Research integrity is important for advancing science, promoting trust among scientists, fostering public support and accountability, and ensuring science conforms to moral standards.

“Research integrity is important to 1) advance the goals of science, 2) promote trust among scientists, 3) foster the public’s support for research and hold science accountable to the public; and 4) ensure that science conforms to moral standards (Shamoo and Resnik 2015).”

• cites Shamoo and Resnik 2015Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Misconduct has a legal meaning in the United States and many other countries.

“However, misconduct also has alegal meaning in the United States and many other countries.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

A scientist analyzing a small dataset could commit misconduct by fabricating or falsifying data, but scientists could also engage in other deceptive practices involving data.

“Clearly, a scientist analyzing a small dataset could commit misconduct by fabricating or falsifying data, but scientists could engage in other deceptive practices involving data.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Many U.S. research institutions define misconduct to include categories beyond FFP, such as serious deviations and significant violations of human or animal research regulations.

“Many U.S. research institutions also have definitions of misconduct that include categories of behavior that go beyond FFP, such as serious deviations and significant violations of human or animal research regulations(Resnik et al 2015a).”

• cites Resnik et al 2015aNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Perez found that the method had not been independently validated and the computer program was unstable.

“Third year medical student Bradford Perez, who was working with Potti, discovered that the method had not been independently validated and the computer program was unstable.”

• cites Goldberg 2015Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The American Statistical Association has produced ethical guidelines for statistical practice that could inform policies on data manipulation.

“The American Statistical Association (2016) has developed guidelines for ethical statistical practice that could also inform policies concerning data manipulation.”

• cites American Statistical Association 2016Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

In data-intensive science, methods and techniques are continually evolving in response to growing dataset size, heterogeneity, and complexity.

“This concern may be especially important in data-intensive science, where methods and techniques continue to evolve in response to increases in the size, heterogeneity, and complexity of datasets (Fan et al 2014).”

• cites Fan et al 2014Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Researchers could avoid developing or adopting innovative data-processing, analysis, or interpretation methods in order to protect themselves from misconduct allegations.

“Second, researchers might refrain from developing or implementing innovative and useful methods for processing, analyzing, or interpreting data to protect themselves againstmisconduct allegations.”

• AssumptionThe authors state this as an assumption: Promoting progress and innovation in data-intensive science is important.

May combine two claims.

Failure to properly describe and disclose steps in data processing, analysis, or interpretation could entail deception and lead to irreproducibility, bias, or erroneous inference.

“Additionally, as with any form of scientific approach, failure to properly describe and disclose any of the steps related to data processing, analysis, or interpretation could entail deception and lead to irreproducibility, bias, or erroneous inference (Fan et al 2014).”

• cites Fan et al 2014Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable).

Adding deceptive statistical practices to the definition of misconduct could divert the attention and resources of institutions, funding agencies, and journals away from more important research integrity concerns, particularly fabrication, falsification, and plagiarism.

“First, institutional officials, funding agencies, and journals could be inundated with misconduct allegations related to deceptive uses of statistics and might not have enough time or resources to deal with important cases related to FFP.”

• Also stated in: § Deception in Data-Intensive Science · hedged

  “Expanding the definition of misconduct to include deceptive uses of statistics could distract institutions, agencies, and journals from focusing on more important research integrity concerns.”

• Also stated in: § Integrity in Data-Intensive Science · hedged

  “Because we are concerned that a separate category of deceptive data practices could distract institutions, funding agencies, and journals from focusing on FFP cases”

• AssumptionRests on an assumption: Fabrication, falsification, and plagiarism are more important research integrity concerns than deceptive data processing, analysis, or interpretation.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Classifying deceptive statistical practices that do not involve fabrication, falsification, or plagiarism as misconduct could lead to scientists facing allegations from critics or competitors who disagree with their methods, with potentially damaging consequences for research.

“If deceptive uses of statistics which do not involve FFP were classified as misconduct, scientists could face misconduct allegations from critics or competitors who disagree with their methods, a situation which could have damaging consequences for scientific research.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The Center for Open Science guidelines incorporate many of these data-policy recommendations.

“Many of these recommendations are included in guidelines developed by the Center for Open Science (2016).”

• cites Center for Open Science 2016Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Discussions of best practices for transparency about methods, databases, computational algorithms, and supplementary material are ongoing.

“Discussions of best practices pertaining to transparency concerning methods, databases, computational algorithms, and supplementary material are ongoing (Fan et al 2014).”

• cites Fan et al 2014Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Software such as Photoshop allows digital images of proteins, cells, and other structures to be manipulated to improve their clarity and quality.

“Computer programs, such as Photoshop, have made it possible to manipulate digital images of proteins, cells, and other structures to enhance the clarity and quality of the image (Rossner and Yamada 2000).”

• cites Rossner and Yamada 2000Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Distinguishing appropriate from inappropriate data management is likely to be difficult, but explicit policies could improve clarity.

“As with the manipulation of images, it is likely to be difficult to distinguish appropriate and inappropriate forms of data management, but the formulation of explicit policies could provide greater clarity.”