What Makes Science Transparent? A Consensus-Based Checklist

This post is a synopsis of Aczel et al. (2019). A consensus-based transparency checklist. Nature Human Behaviour. Open Access: https://www.nature.com/articles/s41562-019-0772-6.
The associated Shiny app is at http://www.shinyapps.org/apps/
TransparencyChecklist/.

 
How can social scientists make their work more transparent? Sixty-three editors and open science advocates reached consensus on this topic and created a checklist to help authors document various transparency-related aspects of their work.

“The final version of the Transparency Checklist 1.0 contains 36 items that cover four components of a study: Preregistration; Methods; Results and Discussion; Data, Code, and Materials Availability. For each item, authors select the appropriate answer from pre-specified options. It is important to emphasize that none of the responses on the checklist is a priori good or bad, and the transparency report provides researchers the opportunity to explain their choices at the end of each section.”

 

Fig 1: The Shortened Transparency Checklist 1.0
Long version: http://www.shinyapps.org/apps/TransparencyChecklist/

Main Benefits of the Checklist

1. The checklist can help authors improve the transparency of their work before submitting it to a journal.
2. Disclosed checklist responses can help editors, reviewers, and readers gain insight into the transparency of the submitted studies.
3. Guidelines built on the checklist can be used for educational purposes and to raise the standards of social and behavioural sciences, as well as other scientific disciplines, regarding transparency and credibility.
4. Funding agencies can use a version of this checklist to improve the research culture and accelerate scientific progress.

A Shiny Transparency Report

In order to facilitate the use of the checklist we created a Shiny app that you can take for a test drive here: http://www.shinyapps.org/apps/TransparencyChecklist/

The app allows researchers to choose between a shortened 12-item version and the full 36-item version, and at the end produces a report in pdf format. With the Shiny app in place, researchers can produce a comprehensive transparency report in minutes.

References

Aczel, B., Szaszi, B., Sarafoglou, A., Kekecs, Z., Kucharsky, S., Benjamin, D., Chambers, C. D., Fisher, A., Gelman, A., Gernsbacher, M. A., Ioannidis, J. P., Johnson, E., Jonas, K., Kousta, S., Lilienfeld, S. O., Lindsay, D. S., Morey, C. C., Munafo, M., Newell, B. R., Pashler, H., Shanks, D. R., Simons, D. J., Wicherts, J. M., Albarracin, D., Anderson, N. D., Antonakis, J., Arkes, H., Back, M. D., Banks, G. C., Beevers, C., Bennett, A. A., Bleidorn, W., Boyer, T. W., Cacciari, C., Carter, A. S., Cesario, J., Clifton, C., Conroy, R. M., Cortese, M., Cosci, F., Cowan, N., Crawford, J., Crone, E. A., Curtin, J., Engle, R., Farrell, S., Fearon, P., Fichman, M., Frankenhuis, W., Freund, A. M., Gaskell, M. G., Giner-Sorolla, R., Green, D. P., Greene, R. L., Harlow, L. L., Hoces de la Guardia, F., Isaacowitz, D., Kolodner, J., Lieberman, D., Logan, G. D., Mendes, W. B., Moersdorf, L., Nyhan, B., Pollack, J., Sullivan, C., Vazire, S., & Wagenmakers, E.-J. (in press). A consensus-based transparency checklist. Nature Human Behaviour. Open Access: https://www.nature.com/articles/s41562-019-0772-6.

About The Authors

Balazs Aczel

Balazs Aczel is associate professor at ELTE University in Budapest, Hungary. For more information about Balazs, you can visit his Website or his Twitter.

Alexandra Sarafoglou

Alexandra Sarafoglou is a PhD candidate at the Psychological Methods Group at the University of Amsterdam.

Eric-Jan Wagenmakers

Eric-Jan (EJ) Wagenmakers is professor at the Psychological Methods Group at the University of Amsterdam.