Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences

Alec P. Christie; David Abecasis; Mehdi Adjeroud; Juan C. Alonso; Tatsuya Amano; Alvaro Anton; Barry P. Baldigo; Rafael Barrientos; Jake E. Bicknell; Deborah A. Buhl; Just Cebrian; Ricardo S. Ceia; Luciana Cibils-Martina; Sarah Clarke; Joachim Claudet; Michael D. Craig; Dominique Davoult; Annelies De Backer; Mary K. Donovan; Tyler D. Eddy; Filipe M. França; Jonathan P. A. Gardner; Bradley P. Harris; Ari Huusko; Ian L. Jones; Brendan P. Kelaher; Janne S. Kotiaho; Adrià López-Baucells; Heather L. Major; Aki Mäki-Petäys; Beatriz Martín; Carlos A. Martín; Philip A. Martin; Daniel Mateos-Molina; Robert A. McConnaughey; Michele Meroni; Christoph F. J. Meyer; Kade Mills; Monica Montefalcone; Norbertas Noreika; Carlos Palacín; Anjali Pande; C. Roland Pitcher; Carlos Ponce; Matt Rinella; Ricardo Rocha; María C. Ruiz-Delgado; Juan J. Schmitter-Soto; Jill A. Shaffer; Shailesh Sharma; Anna A. Sher; Doriane Stagnol; Thomas R. Stanley; Kevin D. E. Stokesbury; Aurora Torres; Oliver Tully; Teppo Vehanen; Corinne Watts; Qingyuan Zhao; William J. Sutherland; Sveriges lantbruksuniversitet

doi:10.1038/s41467-020-20142-y

Back

Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences

Journal article

Open access

Peer reviewed

Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences

Alec P. Christie, David Abecasis, Mehdi Adjeroud, Juan C. Alonso, Tatsuya Amano, Alvaro Anton, Barry P. Baldigo, Rafael Barrientos, Jake E. Bicknell, Deborah A. Buhl, …

Nature communications, Vol.11(1), pp.6377-6377

12/11/2020

DOI: https://doi.org/10.1038/s41467-020-20142-y

PMID: 33311448

Abstract

631/158

704/172/4081

706/648

706/689

Article

Humanities and Social Sciences

multidisciplinary

Science

Science (multidisciplinary)

Building trust in science and evidence-based decision-making depends heavily on the credibility of studies and their findings. Researchers employ many different study designs that vary in their risk of bias to evaluate the true effect of interventions or impacts. Here, we empirically quantify, on a large scale, the prevalence of different study designs and the magnitude of bias in their estimates. Randomised designs and controlled observational designs with pre-intervention sampling were used by just 23% of intervention studies in biodiversity conservation, and 36% of intervention studies in social science. We demonstrate, through pairwise within-study comparisons across 49 environmental datasets, that these types of designs usually give less biased estimates than simpler observational designs. We propose a model-based approach to combine study estimates that may suffer from different levels of study design bias, discuss the implications for evidence synthesis, and how to facilitate the use of more credible study designs. Randomised controlled experiments are the gold standard for scientific inference, but environmental and social scientists often rely on different study designs. Here the authors analyse the use of six common study designs in the fields of biodiversity conservation and social intervention, and quantify the biases in their estimates.

Files and links (1)

url

https://doi.org/10.1038/s41467-020-20142-yView

Published (Version of record) Open

Metrics

1 Record Views

See more details

Details

Title: Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences
Creators: Alec P. Christie - Conservation Leadership Programme
David Abecasis - University of Algarve
Mehdi Adjeroud - Labex Corail
Juan C. Alonso - Museo Nacional de Ciencias Naturales
Tatsuya Amano - University of Queensland
Alvaro Anton - University of the Basque Country
Barry P. Baldigo - United States Geological Survey
Rafael Barrientos - Universidad Complutense de Madrid
Jake E. Bicknell - University of Kent
Deborah A. Buhl - United States Geological Survey
Just Cebrian - Northern Gulf Institute
Ricardo S. Ceia - University of Coimbra
Luciana Cibils-Martina - National University of Río Cuarto
Sarah Clarke - Marine Institute
Joachim Claudet - École Pratique des Hautes Études
Michael D. Craig - University of Western Australia
Dominique Davoult - Sorbonne Université
Annelies De Backer - Vlaams Instituut voor Biotechnologie
Mary K. Donovan - University of California, Santa Barbara
Tyler D. Eddy - University of South Carolina
Filipe M. França - Lancaster University
Jonathan P. A. Gardner - Victoria University of Wellington
Bradley P. Harris - Alaska Pacific University
Ari Huusko - Natural Resources Institute Finland
Ian L. Jones - Memorial University of Newfoundland
Brendan P. Kelaher - Southern Cross University
Janne S. Kotiaho - University of Jyväskylä
Adrià López-Baucells - University of Lisbon
Heather L. Major - University of New Brunswick
Aki Mäki-Petäys - Teollisuuden Voima
Beatriz Martín - Intergovernmental Oceanographic Commission of UNESCO
Carlos A. Martín - Universidad Complutense de Madrid
Philip A. Martin - St Mary's University College
Daniel Mateos-Molina - Universidad de Murcia
Robert A. McConnaughey - NOAA National Marine Fisheries Service
Michele Meroni - Joint Research Centre
Christoph F. J. Meyer - University of Lisbon
Kade Mills - Parks Victoria
Monica Montefalcone - University of Genoa
Norbertas Noreika - Swedish University of Agricultural Sciences
Carlos Palacín - Museo Nacional de Ciencias Naturales
Anjali Pande - Victoria University of Wellington
C. Roland Pitcher - CSIRO Oceans and Atmosphere
Carlos Ponce - Museo Nacional de Ciencias Naturales
Matt Rinella - Fort Keogh Livestock and Range Research Laboratory
Ricardo Rocha - University of Lisbon
María C. Ruiz-Delgado - Universidad Pablo de Olavide
Juan J. Schmitter-Soto - El Colegio de la Frontera Sur
Jill A. Shaffer - United States Geological Survey
Shailesh Sharma - New York State Department of Environmental Conservation
Anna A. Sher - University of Denver
Doriane Stagnol - Station Biologique de Roscoff
Thomas R. Stanley - Fort Collins Science Center
Kevin D. E. Stokesbury - University of Massachusetts Dartmouth
Aurora Torres - UCLouvain
Oliver Tully - Marine Institute
Teppo Vehanen - Natural Resources Institute Finland
Corinne Watts - Manaaki Whenua – Landcare Research
Qingyuan Zhao - University of Cambridge
William J. Sutherland - St Mary's University College
Sveriges lantbruksuniversitet
Publication Details: Nature communications, Vol.11(1), pp.6377-6377
Publisher: Nature Publishing Group UK
Language: English
Academic Unit: Department of Fisheries Oceanography; School for Marine Science and Technology
Resource Type: Journal article
DOI: https://doi.org/10.1038/s41467-020-20142-y
Record Identifier: 9914436821601301