Currently Browsing: Misconceptions

# The Butler, The Maid, And The Bayes Factor

This post is based on the example discussed in Wagenmakers et al. (in press).

### The Misconception

Bayes factors are a measure of absolute goodness-of-fit or absolute pre-
dictive performance.

### The Correction

Bayes factors are a measure of relative goodness-of-fit or relative predictive performance. Model A may outpredict model B by a large margin, but this does not imply that model A is good, appropriate, or useful in absolute terms. In fact, model A may be absolutely terrible, just less abysmal than model B.

### The Explanation

Statistical inference rarely deals in absolutes. This is widely recognized: many feel the key objective of statistical modeling is to quantify the uncertainty about parameters of interest through confidence or credible intervals. What is easily forgotten is that there is additional uncertainty, namely that which concerns the choice of the statistical model.

# Bayes Factors for Those Who Hate Bayes Factors

This post is inspired by Morey et al. (2016), Rouder and Morey (in press), and Wagenmakers et al. (2016a).

## The Misconception

Bayes factors may be relevant for model selection, but are irrelevant for
parameter estimation.

## The Correction

For a continuous parameter, Bayesian estimation involves the computation of an infinite number of Bayes factors against a continuous range of different point-null hypotheses.

## The Explanation

Let H0 specify a general law, such that, for instance, the parameter θ has a fixed value θ0. Let H1 relax the general law and assign θ a prior distribution p(θ | H1). After acquiring new data one may update the plausibility for H1 versus H0 by applying Bayes’ rule (Wrinch and Jeffreys 1921, p. 387):

# Popular Misconceptions About Bayesian Inference: Introduction to a Series of Blog Posts

“By seeking and blundering we learn.”

– Johann Wolfgang von Goethe

Bayesian methods have never been more popular than they are today. In the field of statistics, Bayesian procedures are mainstream, and have been so for at least two decades. Applied fields such as psychology, medicine, economy, and biology are slow to catch up, but in general researchers now view Bayesian methods with sympathy rather than with suspicion (e.g., McGrayne 2011).

The ebb and flow of appreciation for Bayesian procedures can be explained by a single dominant factor: pragmatism. In the early days of statistics, the only Bayesian models that could be applied to data were necessarily simple – the more complex, more interesting, and more appropriate models escaped the mathematically demanding derivations that Bayes’ rule required. This meant that unwary researchers who accepted the Bayesian theoretical outlook effectively painted themselves into a corner as far as practical application was concerned. How convenient then that the Bayesian paradigm was “absolutely disproved” (Peirce 1901, as reprinted in Eisele 1985, p. 748); how reassuring that it would “break down at every point” (Venn 1888, p. 121); and how comforting that it was deemed “utterly unacceptable” (Popper 1959, p. 150).