Fraudulent psychologists like Diederik Stapel and Lorenza Colzato and the revelation that many study results can’t be replicated: the behavioral sciences have been haunted by questionable research practices for years. It is no coincidence, then, that it was a group of young psychologists who initiated the Rotterdam branch of the R.I.O.T. Science Club. This community was founded at King’s College London, and provides training and education in how to do better science and ultimately achieve a better research culture.
Their name reflects their ideals. Research must be R.I.O.T, they say, which means reproducible, interpretable, open, and transparent. The four concepts interact, argues Lorenza Dall’Aglio. She is a Ph.D. student at the Department of Child and Adolescent Psychiatry at Erasmus MC and started the Rotterdam R.I.O.T. Science Club together with fellow Ph.D. student Elisabet Blok. ‘Transparency is achieved with openness and allows reproducibility. Together they will lead to better, unambiguous, and reproducible science.’
Little sense of urgency
In psychology, the replication crisis has been a topic of conversation for years. At the heart of the crisis is the finding that many study results can’t be replicated when the research is repeated. But in hospitals and biomedical sciences, there is little sense of urgency yet, noticed Dall’Aglio and co-RIOT’er Bing Xu. ‘In the medical world, many scientists are not aware this problem applies to their field too.’
While it has been shown this is likely the case. A Nature survey of more than 1,500 researchers found that 70 percent of them have tried and failed to reproduce another scientist’s experiments. Nearly 1,000 of the respondents worked in biology and medicine. In several medical fields, including cancer research, multiple reports of low reproducibility rates have also been published.
‘In the medical world, many scientists are not aware this problem applies to their field too’
Scientists have the best intentions and are almost never deliberately cheating, Dall’Aglio and Xu stress. ‘Science usually is a vocation. People are doing it for the public good. But scientists work in a system with pervasive incentives. They are judged by how much funding they bring in, the number of articles they publish, and their H-index. Their next career move depends largely on these incentives, but such incentives don’t necessarily correspond to what constitutes good science.’
On top of that, according to Dall’Aglio, there is still too little attention to the principles of Open Science during training. ‘It’s important not to blame scientists. It is also about the way in which they were taught to do research.’
The Rotterdam R.I.O.T. Science Club is trying to address that lack of knowledge. They organize free seminars and training sessions and present awards to researchers who lead by example. Dall’Aglio: ‘We create the space to talk about Open Science and reproducibility. By teaching people how to do it, we create change. Our idea is: now that we know better, it’s important to do better.’
Magic wand
The community is not a magic wand that will solve all problems, R.I.O.T.’ers argue. Dall’Aglio: ‘We don’t want to preach Open Science as if it were the new H-index. Criteria like the H-index and the number of publications were introduced with the best of intentions. But they turned out to have negative consequences. It is important to consider the usefulness of Open Science to each situation. If we are doing it for the sake of doing it, we might cause harm: if we share our data in an irresponsible way, that’s worse than not sharing at all. We do support Open Science, but done so in a mindful way.’
R.I.O.T. Rotterdam was deliberately started by early career researchers. But ultimately they need the support of senior colleagues. Dall’Aglio: ‘Changes in research culture usually start top-down, with funding agencies and journals changing their policies.’ Xu adds: ‘There is progress. For example, the biggest Dutch science funding body NWO has introduced the narrative curriculum vitae. The Netherlands is one of the leading countries in Open Science.’
Hear our voices
Dall’Aglio: ‘Communities like R.I.O.T. are trying to say: hey, this is important to us. But the people making the decisions need to hear our voices. Being in a senior position can be a valuable opportunity to push these initiatives forward, for example, by changing how you hire and promote. Bottom-up change is great, but it’s even better when it is complemented by top-down change.’
Join
Want to join? Rotterdam R.I.O.T. Science Club welcomes new members from all disciplines and stages in their careers. For more information, including the activity calendar, please visit the website.
