This post is a synopsis of Bartoš, F., Maier, M., Quintana D. S., & Wagenmakers, E. (2021). Adjusting for Publication Bias in JASP & R — Selection Models, PET-PEESE, and Robust Bayesian Meta-Analysis. Preprint available at https://doi.org/10.31234/osf.io/kvsp7
Meta-analyses are essential for cumulative science, but their validity can be compromised by publication bias. In order to mitigate the impact of publication bias, one may apply publication bias adjustment techniques such as PET-PEESE and selection models. Implemented in JASP & R, these methods allow researchers without programming experience to conduct state-of-the-art publication bias adjusted meta-analysis. In this tutorial, we demonstrate how to conduct a publication bias adjusted meta-analysis in JASP & R and interpret the results. First, we explain two frequentist bias correction methods: PET-PEESE and selection models. Second, we introduce Robust Bayesian Meta-Analysis (RoBMA), a Bayesian approach that simultaneously considers both PET-PEESE and selection models. We illustrate the methodology on an example data set, provide an instructional video, a R-markdown script, and discuss the interpretation of the results. Finally, we include concrete guidance on reporting the meta-analytic results in an academic article.
In the updated preprint we provide an accessible introduction to performing publication bias adjusted meta-analyses with both JASP and R. The JASP portion of the tutorial is recorded by Daniel Quintana and published on the JASP YouTube channel,
while the R portion of the tutorial is accompanied by a commented R-markdown script.
We hope that these resources will facilitate the adoption of state-of-the-art publication bias adjustment methods in the research community.
Bartoš, F., Maier, M., Wagenmakers, E. J., Doucouliagos, H., & Stanley, T. D. (2021). No need to choose: Robust Bayesian meta-analysis with competing publication bias adjustment methods. Preprint. https://psyarxiv.com/kvsp7/
Bartoš, F., Maier, M., Quintana D. S., & Wagenmakers, E. (2021). Adjusting for publication bias in JASP & R — Selection models, PET-PEESE, and robust Bayesian meta-analysis. Preprint. https://doi.org/10.31234/osf.io/75bqn
Maier, M., Bartoš, F., & Wagenmakers, E. J. (in press). Robust Bayesian meta-analysis: Addressing publication bias with model-averaging. Psychological Methods. (Preprint available at https://psyarxiv.com/u4cns/)
Stanley, T. D., & Doucouliagos, H. (2014). Meta‐regression approximations to reduce publication selection bias. Research Synthesis Methods, 5(1), 60-78. https://doi.org/10.1002/jrsm.1095
Vevea, J. L., & Hedges, L. V. (1995). A general linear model for estimating effect size in the presence of publication bias. Psychometrika, 60(3), 419–435. https://doi.org/10.1007/BF02294384
About The Authors
František Bartoš is a PhD candidate at the Psychological Methods Group of the University of Amsterdam.
Maximilian Maier is a PhD candidate at University College London.
Daniel Quintana is a senior researcher in psychoneuroendocrinology at the University of Oslo.
Eric-Jan (EJ) Wagenmakers is professor at the Psychological Methods Group at the University of Amsterdam.