Scientific research, often portrayed as the acme of human rigorous thought, is actually an activity that’s grown rather organically over the centuries from Aristotle, via Bacon, to an endeavour that now involves millions of people across the globe. Recently, however, it’s been recognised that there are grave problems inherent in this organic structure, and that these problems may be obstructing the progress of science, wasting precious and limited resources, and producing misleading conclusions. This realisation has led to the emergence of the field of meta-research, which aims to study how research is done, and ultimately to inform measures that might mitigate these problems and realise the full potential of scientific research.
A collateral thrust of meta-research is the recognition of the key role that scientific journals have played in perpetuating research practices, and the role that they could and should play in improving the standards of consistency in performance, analysis, reporting and interpretation of research. This is part of the reasoning behind the new broadening of PLOS Biology’s scope to include meta-research papers, and ongoing initiatives to make reporting of research more uniform and rigorous across scientific journals.
An example of how inconsistent reporting can get in the way of science is the non-uniform reporting of clinical rating scales, which can interfere with the subsequent inclusion of studies in meta-analyses; a recent PLOS ONE paper assesses the scale of the problem and proposes a practical solution that could improve the usefulness of individual studies and maximise the overall benefit to society.
Meta-analysis, often thought of as a tool for combining clinical and preclinical trial data (as in the example just mentioned), can be just as powerful in basic science, as seen in this PLOS Genetics meta-analysis of behavioural studies in fruit fly mutants, wresting statistically significant results from studies that are themselves individually marginal.
Crucial to all of these endeavours is the availability of the data that underlie studies, which a study in PLOS ONE shows to vary massively between scientific disciplines, being best in biology, but still falling well short of an ideal situation. And within biology itself, field-to-field variation is colossal, with some woeful behaviour reported among phylogeneticists, where some unique data may be lost for ever. Even in genetics, which has a 33-year culture of data-sharing inaugurated by GenBank/ENA/DDBJ, there’s room for significant improvement, as explored by this PLOS Genetics editorial.
Between them these articles – and those included with the launch of the PLOS Meta-Research Collection – exemplify the level of scrutiny, rigour, and practical recommendation and enforcement that will be needed to get the most out of scientific research. This research is after all conducted by a few million scientists on behalf of all humans, and largely funded by their billions of dollars of taxes and charitable donations. Making the most of that investment is a duty, not a nicety.
Featured image credit: Flickr user 118118485@N05
Cumming, T., Churilov, L., & Sena, E. (2015). The Missing Medians: Exclusion of Ordinal Data from Meta-Analyses PLOS ONE, 10 (12) DOI: 10.1371/journal.pone.0145580