Powered by JASP

A Cartoon to Explain How Blinding Works

A Cartoon to Explain How Blinding Works

The cartoon presented below is available from the artwork library of BayesianSpectacles.org under a CC-BY license. The cartoon was conceptualized by Alexandra Sarafoglou and was drawn by Viktor Beekman. It is included as an appendix in Dutilh, G., Sarafoglou, A., & Wagenmakers, E.-J. (in press). Flexible yet fair: Blinding analyses in experimental psychology. Synthese. PsyArXiv Preprint: https://psyarxiv.com/d79r8

Because the procedure of analysis blinding is relatively rare and (in our experience) easily misunderstood, we clarified the key concepts using a cartoon. Here it is, courtesy of Viktor Beekman:
(more…)


Flexible Yet Fair: Blinding Analyses in Experimental Psychology

This post is an extended synopsis of Dutilh, G., Sarafoglou, A., & Wagenmakers, E.-J. (in press). Flexible yet fair: Blinding analyses in experimental psychology. Synthese. Preprint available on PsyArXiv: https://psyarxiv.com/h39jt

 

Abstract

The replicability of findings in experimental psychology can be improved by distinguishing sharply between hypothesis-generating research and hypothesis-testing research. This distinction can be achieved by preregistration, a method that has recently attracted widespread attention. Although preregistration is fair in the sense that it inoculates researchers against hindsight bias and confirmation bias, preregistration does not allow researchers to analyze the data flexibly without the analysis being demoted to exploratory. To alleviate this concern we discuss how researchers may conduct blinded analyses (MacCoun & Perlmutter, 2015). As with preregistration, blinded analyses break the feedback loop between the analysis plan and analysis outcome, thereby preventing cherry-picking and significance seeking. However, blinded analyses retain the flexibility to account for unexpected peculiarities in the data. We discuss different methods of blinding, offer recommendations for blinding of popular experimental designs, and introduce the design for an online blinding protocol.
(more…)


The Liberating Feeling of Relinquishing Control: Advice for Advisors

Disclaimer: advice based purely on the life and lab of the author. May not generalize to other people and other contexts. No literature whatsoever was consulted. Take advice at your own risk.

For most of my life I have had the idea that the key to happiness is control, or at least the illusion of control. What person would delight in having to follow orders and obeying the wishes of some random overlord? This is one of the perks of academia (at least in my Dutch bubble): the complete and utter freedom to do what you want, when you want, and where you want. Of course there can be lack of control in academia as well, even in the Dutch bubble. Once I felt a burn-out looming, and it was when I imagined myself standing at the bottom of a tall mountain, with an avalanche of work hurdling its way toward me, intent on crushing and suffocating — in other words, I experienced a profound lack of control.
(more…)


Preprint: “Because it is the Right Thing to Do”: Taking Stock of the Peer Reviewers’ Openness Initiative

This post is an extended synopsis of Dahrendorf, M., Hoffmann, T., Mittenbühler, M., Wiechert, S., Sarafoglou, A., Matzke, D., & Wagenmakers, E.-J. (2019). “Because it is the Right Thing to Do”: Taking Stock of the Peer Reviewers’ Openness Initiative. Preprint available on PsyArXiv: https://psyarxiv.com/h39jt

 

Abstract

In recent years, multiple initiatives have sought to improve the transparency and reproducibility of psychological research. One example is the Peer Reviewers’ Openness Initiative (PRO), which aims to promote the sharing of data and code. PRO signatories pledge to provide a full review only for manuscripts that publicly share data and code, or include a justification why sharing is not possible. The punitive element of this approach attracted criticism: PRO may be perceived as coercive and could lead to negative repercussions for its signatories. Therefore we conducted a survey to assess signatories’ experiences with PRO. Contrary to the criticism, the reported experiences were predominantly positive, and 92% (117/127) of the signatories who completed the survey indicated that they would sign the initiative again today. Only 15% (19/127) of the respondents experienced negative reactions related to their commitment to PRO. Almost 50 respondents suggested ways in which PRO could be improved. We conclude that, from the signatories’ perspective, the benefits of the PRO initiative outweigh its drawbacks.
(more…)


The Invigorating Pleasure of Witnessing a Well-Contested Rat-Fight

Throughout his books, Bayesian godfather Sir Harold Jeffreys was in the habit of starting each chapter with an epigraph. Usually these epigraphs came from different sources, but not so for his 1935 geophysics book “Earthquakes and mountains”. The book has a total of seven chapters; here are the seven associated epigraphs:

 

For chapter 1, “Solids and Liquids”:

…but who attempts to eat an orange without first disposing of the peel, or what manner of a dwelling could be erected unless an adequate foundation be first provided?– Kai Lung’s Golden Hours, 144.

Although the purpose of the epigraph has eluded me, I do like it. The part about the orange is an apt response to those statisticians who wish to estimate parameters before assessing whether there is anything to estimate at all (see the earlier post here).
(more…)


Powered by WordPress | Designed by Elegant Themes