Social and psychological experiments 'must report methods'

Researchers from Oxford University and UCL (University College London) reviewed over 200 experiments across 40 of the leading journals in social and behavioural sciences (covering clinical psychology, criminology, education, and social work). They conclude that public money is being wasted because the reports are not carrying full details about how the interventions were implemented. Without the ‘missing details’, it is difficult for policymakers and practitioners to put the interventions into practice despite the amounts of money being invested in high-quality research, says the study published in the current issue of the journal PLOS ONE.

The study, which was supported by the Economic and Social Research Council (ESRC), suggests that reporting guidelines have improved the quality of trial reports in medicine. Yet those existing guidelines may not be suitable for social and psychological intervention trials. The study argues that a higher quality of reporting would give policymakers and practitioners the guidance they need on what groups might benefit from interventions. Details about the study methods would also indicate whether the results of an experiment are biased, says the research.

Paul Montgomery, Professor of Psycho-Social Interventions at the University of Oxford, said: ‘In this era of austerity, policymakers increasingly look for evidence of “what works” to ensure that revenue is well-spent on programmes that address issues such as poverty, mental health, crime, and drug use.  Evaluations of these programmes can be expensive.  When they are reported fully and transparently, they can help policymakers choose the most effective way to spend public funds; however, readers rely on reports of these studies in academic journals to effectively understand and use the research. Reporting guidelines are a critical step in improving this area of research for policy decision-making.’

In response, the study authors are leading a global effort to develop reporting standards for social and psychological experiments, building on existing guidelines used in medicine and other disciplines. The goal is for scientists who are involved in experiments of complex interventions follow a common set of guidelines that identifies the essential information required to understand all experiments, including disciplines outside of medicine. 

The research team hopes scientists, journal editors, policymakers, and other interested groups will contribute to the discussion. Participants can give their views by using an online form on the Social Policy and Intervention website under the CONSORT project (Consolidated Standards of Reporting Trials). 

Dr Evan Mayo-Wilson, Senior Researcher in Psychology at UCL, said: ‘There are many excellent experiments that identify ways to improve social and psychological care.  Too often, these studies cannot be used because published reports are missing critical details. Social and psychological research should help us change people’s lives.  To do that, the scientific record needs to be both accurate and complete. We are thrilled that so many researchers and journal editors have joined this project.  Through collective effort, we hope this problem can be solved in a few years.’