From P-values to Bayes Factors with eJAB

This post is a teaser for Velidi, P., Wei, Z., Kalaria, S. N., Liu, Y., Laumont, C. M., Nelson, B. H., & Nathoo, F. S. (2025). Generalized Jeffreys’s approximate objective Bayes factor: Model-selection consistency, finite-sample accuracy, and statistical evidence in 71,126 clinical trial findings. ArXiv preprint:2510.10358.

Abstract

“Concerns about the misuse and misinterpretation of p-values and statistical significance have motivated alternatives for quantifying evidence. We define a generalized form of Jeffreys’s approximate objective Bayes factor (eJAB), a one-line calculation that is a function of the p-value, sample size, and parameter dimension. We establish conditions under which eJAB is model-selection consistent and verify them for ten common statistical tests. We assess finite-sample accuracy by comparing eJAB with Markov chain Monte Carlo computed Bayes factors in 12 simulation studies. We then apply eJAB to 71,126 results from ClinicalTrials.gov (CTG) and find that the proportion of findings with p-value ⩽ α yet eJAB01 > 1 closely tracks the significance level α, suggesting that such contradictions are pointing to the type I errors. We catalog 4,088 such candidate type I errors and provide details for 131 with reported p-value ⩽ .01. We also identify 487 instances of the Jeffreys-Lindley paradox. Finally, we estimate that 75% (6%) of clinical trial plans from CTG set α ⩾ .05 as the target evidence threshold, and that 35.5% (0.22%) of results significant at α = .05 correspond to evidence that is no stronger than anecdotal under eJAB.”

Teaser Figure

Evaluation

I thoroughly enjoyed reading this paper. It presents theoretical progress that is of immediate practical relevance.

References

Velidi, P., Wei, Z., Kalaria, S. N., Liu, Y., Laumont, C. M., Nelson, B. H., & Nathoo, F. S. (2025). Generalized Jeffreys’s approximate objective Bayes factor: Model-selection consistency, finite-sample accuracy, and statistical evidence in 71,126 clinical trial findings. ArXiv preprint:2510.10358.

Wagenmakers, E.-J. (2022). Approximate objective Bayes factors from p-values and sample size: The 3p√n rule. Preprint available on PsyArXiv: https://psyarxiv.com/egydq

About The Author

Eric-Jan Wagenmakers

Eric-Jan (EJ) Wagenmakers is professor at the Psychological Methods Group at the University of Amsterdam.