Randomly auditing research labs could be an affordable way to improve research quality: A simulation study.

08:00 EDT 12th April 2018 | BioPortfolio

Summary of "Randomly auditing research labs could be an affordable way to improve research quality: A simulation study."

The "publish or perish" incentive drives many researchers to increase the quantity of their papers at the cost of quality. Lowering quality increases the number of false positive errors which is a key cause of the reproducibility crisis. We adapted a previously published simulation of the research world where labs that produce many papers are more likely to have "child" labs that inherit their characteristics. This selection creates a competitive spiral that favours quantity over quality. To try to halt the competitive spiral we added random audits that could detect and remove labs with a high proportion of false positives, and also improved the behaviour of "child" and "parent" labs who increased their effort and so lowered their probability of making a false positive error. Without auditing, only 0.2% of simulations did not experience the competitive spiral, defined by a convergence to the highest possible false positive probability. Auditing 1.35% of papers avoided the competitive spiral in 71% of simulations, and auditing 1.94% of papers in 95% of simulations. Audits worked best when they were only applied to established labs with 50 or more papers compared with labs with 25 or more papers. Adding a ±20% random error to the number of false positives to simulate peer reviewer error did not reduce the audits' efficacy. The main benefit of the audits was via the increase in effort in "child" and "parent" labs. Audits improved the literature by reducing the number of false positives from 30.2 per 100 papers to 12.3 per 100 papers. Auditing 1.94% of papers would cost an estimated $15.9 million per year if applied to papers produced by National Institutes of Health funding. Our simulation greatly simplifies the research world and there are many unanswered questions about if and how audits would work that can only be addressed by a trial of an audit.


Journal Details

This article was published in the following journal.

Name: PloS one
ISSN: 1932-6203
Pages: e0195613


DeepDyve research library

PubMed Articles [29605 Associated PubMed Articles listed on BioPortfolio]

Assessing data quality and the variability of source data verification auditing methods in clinical research settings.

Data audits within clinical settings are extensively used as a major strategy to identify errors, monitor study operations and ensure high-quality data. However, clinical trial guidelines are non-spec...

Development and validation of the Person-Centeredness of Research Scale.

Person-centeredness shifts the focus of healthcare and research to the needs and priorities of patients and communities, and may improve health outcomes. There are no instruments available, however, w...

Increasing Residency Research Output While Cultivating Community Research Collaborations.

Having a research curriculum in addition to hosting a resident research day stimulates research activity in residency programs. Research collaborations outside an individual residency program may also...

The Effects of the Patient Protection and Affordable Care Act on Children's Health Coverage.

Prior research of the impacts of the Patient Protection and Affordable Care Act (PPACA) on children's health coverage has been largely descriptive and focused on the Medicaid expansions.

Improving research and policy interactions requires a better understanding of what works in different contexts.

There is keen interest in many jurisdictions in finding ways to improve the way that research evidence informs policy. One possible mechanism for this is to embed academics within government agencies ...

Clinical Trials [13535 Associated Clinical Trials listed on BioPortfolio]

Transdisciplinary Oral/Oropharyngeal Cancer Research & Care in Head and Neck Cancer (TORCH)

The purpose of this research study is to see if it is possible to collect tissue, saliva and blood samples from patients who are having surgery and send those samples to different labs acr...

Design of an Affordable Gym for Post Stroke Rehabilitation

Rehab in a Crate is a therapy gym designed to be affordable, compact and easily transportable. The purpose of this particular study is to gain feedback on the initial design of the Rehab i...

Long-Term Effects of Bariatric Surgery

The primary objective of LABS-2 is to use standardized techniques and measures to assess the longer-term safety and efficacy of bariatric surgery by: 1. comparing post-surgical outcomes...

Post Affordable Care Act Evaluation

This innovative study will measure the impact of Affordable Care Act-sponsored Medicaid expansions on access to and utilization of community health center (CHC) services. Building on our p...

Investigation of a Novel Approach to Improve Treatment Success Rates for Tuberculosis Patients in Senegal

Tuberculosis (TB) patients who receive inadequate treatment or do not complete therapy are more likely to remain infectious, thus contributing to the continuous spread of TB infection in c...

Medical and Biotech [MESH] Definitions

Research carried out by nurses in the clinical setting and designed to provide information that will help improve patient care. Other professional staff may also participate in the research.

Component of the NATIONAL INSTITUTES OF HEALTH. It seeks to improve oral, dental and craniofacial health through research, research training, and the dissemination of health information by conducting and supporting basic and clinical research. It was established in 1948 as the National Institute of Dental Research and re-named in 1998 as the National Institute of Dental and Craniofacial Research.

A designation for publications of research resulting from extramural research funded by the National Institutes of Health.

The moral obligations governing the conduct of research. Used for discussions of research ethics as a general topic.

A designation for publications of research resulting from intramural research at the National Institutes of Health.

Quick Search


DeepDyve research library

Relevant Topic

Pediatrics is the general medicine of childhood. Because of the developmental processes (psychological and physical) of childhood, the involvement of parents, and the social management of conditions at home and at school, pediatrics is a specialty. With ...

Searches Linking to this Article