Powered by JASP

“Don’t Interfere with my Art”: On the Disputed Role of Preregistration in Exploratory Model Building

Recently the 59th annual meeting of the Psychonomic Society in New Orleans played host to an interesting series of talks on how statistical methods should interact with the practice of science. Some speakers discussed exploratory model building, suggesting that this activity may not benefit much, if any at all, from preregistration. On the Twitterverse, reports of these talks provoked an interesting discussion between supporters and detractors of preregistration for the purpose of model building. Below we describe the most relevant presentations, point to some interesting threads on Twitter, and then provide our own perspective.

The debate started when Twitter got wind of the fact that my [EJ] mentor and intellectual giant Rich Shiffrin was arguing against preregistration (his slides have also been made available, thanks to both Rich and Trish Van Zandt). Here is the abstract of his talk “Science Should Govern the Practice of Statistics”:

“Although there are two sides to these complex issues, this talk will make the case for the scientific judgment side of the ledger. I will I argue that statistics should serve science and should be consistent with scientific judgment that historically has produced progress. I argue against one-size-fits-all statistical criteria, against the view that a fundamental scientific goal should be reproducibility, and against the suppression of irreproducible results. I note that replications should on average produce smaller sized effects than initial reports, even when science is done as well as possible. I make a case that science is post hoc and that most progress occurs when unexpected results are found (and hence against the case for general use of pre-registration). I argue that much scientific progress is often due to production of causal accounts of processes underlying observed data, often instantiated as quantitative models, but aimed at explaining qualitative patterns across many conditions, in contrast to well defined descriptive statistical models.”

(more…)


Transparency and The Need for Short Sentences

Recently I came across an article by Morton Ann Gernsbacher, entitled “Writing empirical articles: Transparency, reproducibility, clarity, and memorability” (preprint). The author covers a lot of ground and makes a series of good points. Also, as one would hope and expect, the article itself is a joy to read. Here is a fragment from the section “Recommendations for Clarity” — subsection “Write short sentences”:

 
 
 
 
 
 
 
 

“Every writing guide, from Strunk and White’s (1959) venerable Elements of Style to the prestigious journal Nature’s (2014) guide, admonishes writers to use shorter, rather than longer, sentences. Shorter sentences are not only easier to understand, but also better at conveying complex information (Flesch, 1948).

The trick to writing short sentences is to restrict each sentence to one and only one idea. Resist the temptation to embed multiple clauses or parentheticals, which challenge comprehension. Instead, break long, rambling sentences into crisp, more concise ones. For example, write the previous three short sentences rather than the following long sentence: The trick to writing short sentences is to restrict each sentence to one and only one idea by breaking long, rambling sentences into crisp, more concise ones while resisting the temptation to embed multiple clauses or parentheticals, which challenge comprehension.

(more…)


Powered by WordPress | Designed by Elegant Themes