The purpose of this blog post is to call attention to the paper “Prior Distributions for Objective Bayesian Analysis”, authored by Guido Consonni, Dimitris Fouskakis, Brunero Liseo, and Ioannis Ntzoufras (NB: Ioannis is a member of the JASP advisory board!). The paper –published in the journal “Bayesian Analysis”— provides a comprehensive overview of objective Bayesian analysis, with an emphasis on model selection and linear regression.
This is one of the best papers on objective Bayesian analysis that I’ve read, and indeed one of the best papers on Bayesian analysis in general. I predict this will become the go-to reference for objective Bayesian inference. At the start, the authors define the need for an objective procedure:
“In many situations a researcher is not able to express his/her prior opinion into a prior distribution. This may happen, for example, in complex applications, where the parameter space has large dimension and a genuine elicitation of the prior dependence structure among the parameters can be out of reach. In other cases, a very limited knowledge of the problem at hand is available, and one would like to encapsulate prior ignorance into a probability distribution.”
I would add to this that objective priors often serve a useful role as a reference analysis; for instance, in the t-test we might use a default choice that fulfills certain desiderata (discussed in the Consonni et al. paper) such as a Cauchy with location 0 and scale 0.707. This choice then provides an anchor, both conceptually and numerically, that can be contrasted with the outcome of a more risky informed hypothesis (e.g., Gronau et al. in press). At any rate, the authors are careful to point out that a “subjective” analysis has its place as well:
“In the end, our view of what constitutes an OB [Objective Bayes — EJ] analysis is unavoidably pragmatic. First of all, we firmly believe that OB and subjective Bayesian analysis should complement each other, the former being helpful in particular scenarios (prior elicitation is too hard, or time consuming, or for reference analysis in scientific reporting). Subjective analysis is still a great resource, especially in applications where information about context is available and can be meaningfully incorporated.”
This 50+ page article is an invaluable resource for anybody who wishes to learn more about objective Bayesian procedures. The authors’ explanations are remarkably clear and insightful, and their treatment is so complete that even experts can profit from references to recent material.
Abstract
“We provide a review of prior distributions for objective Bayesian analysis. We start by examining some foundational issues and then organize our exposition into priors for: i) estimation or prediction; ii) model selection; iii) high-dimensional models. With regard to i), we present some basic notions, and then move to more recent contributions on discrete parameter space, hierarchical models, nonparametric models, and penalizing complexity priors. Point ii) is the focus of this paper: it discusses principles for objective Bayesian model comparison, and singles out some major concepts for building priors, which are subsequently illustrated in some detail for the classic problem of variable selection in normal linear models. We also present some recent contributions in the area of objective priors on model space. With regard to point iii) we only provide a short summary of some default priors for high-dimensional models, a rapidly growing area of research.”
References
Consonni, G., Fouskakis, D., Liseo, B., & Ntzoufras, I. (2018). Prior distributions for objective Bayesian analysis. Bayesian Analysis, 13, 627-679.
Gronau, Q. F., Ly, A., & Wagenmakers, E.-J. (in press). Informed Bayesian t-tests. Journal of the American Statistical Association.
About The Author
Eric-Jan Wagenmakers
Eric-Jan (EJ) Wagenmakers is professor at the Psychological Methods Group at the University of Amsterdam.
