I'm not a fan of multiple comparisons corrections (e.g., Bonferroni, Benjamini-Hochberg, etc.). Just preregister everything and report all results (or else do some kind of multiverse analysis or specification curve), and let the reader decide. One of the biggest issues for me is the following paradox. Say I run an RCT testing a drug as to LDL levels, and I'm trying to decide what outcomes to measure: reduction in LDL over 1 year, cardiac outcomes over 3 years, or mortality over 5 years. Suppose I measure all three, and the trial ultimately shows a reduction in all of those outcomes (p=.04 for each). With Bonferroni etc., the trial would report a "null" effect across the board, even though these outcomes are all consistent and mutually reinforcing, and the evidence for the treatment is way stronger than if I had only collected evidence on cholesterol levels at 1 year.  It makes no sense to me that any single outcome by itself would have been statistically significant, but just because, in the past, I decided to collect more and better evidence, now the treatment has "no effect"?

Wow…Roland Fryer and @BillAckman have created a $1 million prize for academic truth-telling, the Carob Trust Prize for Academic Courage. “The prize awards $1 million each to as many as five social scientists a year who have demonstrated intellectual independence, published findings that were attacked rather than answered, and been validated by the evidence—despite the professional cost. The selection criteria are designed to distinguish courage from contrarianism: Nominees must show a sustained commitment to following logic and evidence regardless of pressure, a willingness to ask questions others avoid, and work that has shifted academic debate, public discourse or policy—often despite being misread, mischaracterized or vilified at the time of publication.”

Beyond “non-significant” results: Why and how to test for practical equivalence | PNAS pnas.org/doi/10.1073/pnas.26…

A Spanish translation of my textbook Improving Your Statistical Inferences, made by Juan G. Fernández Fernández, is now available here: juangff85.github.io/inferenc… I am very grateful Juan worked so hard to made the contents available to Spanish speakers!