# Addressing Elizabeth Loftus’ Lament: When Peeking at Data is Guilt-Free

Elizabeth Loftus is one of the world’s most influential psychologists and I have the greatest respect for her and her work. Several years ago we attended the same party and I still recall her charisma and good sense of humor. Also, Elizabeth Loftus studied mathematical psychology in Stanford, and that basically makes us academic family.

But…just as Stanford math psych graduate Rich Shiffrin, Elizabeth Loftus appears less than enthusiastic about the recent turning of the methodological screws. Below is an excerpt from a recent interview for the Dutch journal De Psycholoog (the entire interview can be accessed, in Dutch, here. I back-translated the relevant fragment from Dutch to English:

Vittorio Busato, the interviewer: “What is your opinion on the replication crisis in psychology?”
(more…)

# The Man Who Rewrote Conditional Probability

The universal notation for “the probability of A given B” is p(A | B). We were surprised to learn that the vertical stroke was first introduced by none other than… Sir Harold Jeffreys! At least Jeffreys himself seems to think so, and the sentiment is echoed on the website “Earliest Uses of Symbols in Probability and Statistics” Specifically, on page 15 of his brilliant book “Scientific Inference” (1931), Jeffreys introduces the vertical stroke notation:

What is commonly referred to as “Lindley’s paradox” exposed a deep philosophical divide between frequentist and Bayesian testing, namely that, regardless of the prior distribution used, high-N data that show a significant p-value may at the same time indicate strong evidence in favor of the null hypothesis (Lindley, 1957). This “paradox” is due to Dennis Lindley, one of the most brilliant and influential scholars in statistics.1

Lindley was thoroughly and irrevocably a Bayesian, never passing on the opportunity of being polemic. For example, he argued that “the only good statistics is Bayesian statistics” (Lindley, 1975) or that Bradley Efron, who just received a big price, may have been “falling over all those bootstraps lying around” (Lindley, 1986). He also trashed Taleb’s Black Swan in great style. Somewhat surprisingly, he also took issues with the Bayes factor.2

(more…)

# A Short Writing Checklist for Students

A number of years ago I compiled a writing checklist for students. Its primary purpose was to make my life easier, but the list was meant to be helpful for the students as well. The checklist is here.

My pet peeves: (1) abrupt changes of topic; (2) poorly designed figures; (3) tables and figures that are not described properly in the main text; and (4) ambiguous referents (“this”, “it”).

For more detailed advice I highly recommend the guidelines from Dan Simons. Also, you may enjoy the article I wrote for the APS Observer a decade ago (Wagenmakers, 2009).

# Prediction is Easy, Especially About the Past: A Critique of Posterior Bayes Factors

### The Misconception

Posterior Bayes factors are a good idea: they provide a measure of evidence but are relatively unaffected by the shape of the prior distribution.

### The Correction

Posterior Bayes factors use the data twice, effectively biasing the outcome in favor of the more complex model.

### The Explanation

The standard Bayes factor is the ratio of predictive performance between two rival models. For each model $M_i$, its predictive performance $p(y | M_i)$ is computed as the likelihood for the observed data, averaged over the prior distribution for the model parameters $\theta | M_i$. Suppressing the dependence on the model, this yields $p(y) = \int p(y | \theta) p(\theta) \, \text{d}\theta$. Note that, as the words imply, “predictions” are generated from the “prior”. The consequence, of course, is that the shape of the prior distribution influences the predictions, and thereby the Bayes factor. Some consider this prior dependence to be a severe limitation; indeed, it would be more convenient if the observed data could be used to assist the models in making predictions — after all, it is easier to make “predictions” about the past than about the future.

# Is Polya’s Fundamental Principle Fundamentally Flawed?

One of the famous fallacies in deductive logic is known as “affirming the consequent”. Here is an example of a syllogism gone wrong:

General statement
When Socrates rises early in the morning,     he always has a foul mood.
Specific statement
Socrates has a foul mood.
Deduction (invalid)
Socrates has risen early in the morning.

The deduction is invalid because Socrates may also be in a foul mood at other times of the day as well. What the fallacy does is take the general statement “A -> B” (A implies B), and interpret it as “B -> A” (B implies A).
(more…)