Manipulating the Alpha Level Cannot Cure Significance Testing

Keywords

Null hypothesis, Statistical significance, Significance testing, Statistical hypothesis testing, p-value

References

• Beyond statistical significance: clinical interpretation of rehabilitation research literature.Phil Page — 2014
• Lash Responds to "Is Reproducibility Thwarted by Hypothesis Testing?" and "The Need for Cognitive Science in Methodology".Roger Boshier — 2017
• The Precautionary Principle: Scientific Uncertainty and Type I and Type II ErrorsJohn Lemons, Kristin Shrader‐Frechette, Carl F. Cranor — 1997
• Deconstructing arguments from the case against hypothesis testingShlomo S. Sawilowsky — 2003
• Estimating the reproducibility of psychological scienceAlexander A. Aarts — 2015
• Robust Statistical EstimationDavid M. Erceg‐Hurn, Rand R. Wilcox, H. J. Keselman — 2013
• Understanding the New Statistics: Effect Sizes, Confidence Intervals, and Meta-AnalysisGeoff Cumming — 2011
• Theory-Testing in Psychology and Physics: A Methodological ParadoxPaul E. Meehl — 1967
• How many people have to die over a type II error?Stephen Rice, David Trafimow — 2010
• Clinical Versus Statistical Significance: Interpreting P Values and Confidence Intervals Related to Measures of Association to Guide Decision MakingMary J. Ferrill, Dana Brown, Jeffrey A. Kyle — 2010
• Setting an Optimal α That Minimizes Errors in Null Hypothesis Significance TestsJoseph F. Mudge, Leanne F. Baker, Christopher B. Edge — 2012
• The Controversy over Null Hypothesis Significance Testing RevisitedNekane Balluerka, Juana Gómez, Dolores Hidalgo — 2005
• On the Persistence of Low Power in Psychological ScienceIvan Vankov, Jeffrey S. Bowers, Marcus R. Munafò — 2014
• Statistical and clinical significance, and how to use confidence intervals to help interpret both.Judith Fethney — 2010
• The earth is round (p < .05)Jacob Cohen — 1994
• “On the Relative Motion of the Earth and the Luminiferous Ether”Jeanette G. Grasselli — 1987
• Tutorial to robust statisticsPeter J. Rousseeuw — 1991
• A Robust Journey in the New MillenniumStephen Portnoy, Xuming He — 2000
• The fickle P value generates irreproducible resultsLewis G. Halsey, Douglas Curran‐Everett, Sarah L. Vowler — 2015
• A Precautionary Approach to Genetically Modified Organisms: Challenges and Implications for Policy and ScienceAnne Ingeborg Myhr — 2010

Cited by

• Why "Redefining Statistical Significance" Will Not Improve Reproducibility and Could Make the Replication Crisis WorseHarry Crane — 2017
• Retract p < 0.005 and propose using JASP, insteadJ Perezgonzalez, María Dolores Frías Navarro — 2017
• Twenty Steps Towards an Adequate Inferential Interpretation of p-Values in EconometricsNorbert Hirschauer, Oliver Mußhoff, Claudia Becker — 2017
• The replication crisis in epidemiology: snowball, snow job, or winter solstice?Timothy L. Lash, Lindsay J. Collin, Miriam E. Van Dyke — 2018
• MicroRNA‐184 and its long noncoding RNA sponge urothelial carcinoma associated 1 are induced in wounded keratinocytes in a store‐operated calcium entry‐dependent mannerAdam Richardson, DJ Owens, Kehinde Ross — 2018
• The Impact of P-hacking on “Redefine Statistical Significance”Harry Crane — 2018
• Reconnecting p-Value and Posterior Probability Under One- and Two-Sided TestsHaolun Shi, Guosheng Yin — 2018
• Pitfalls of significance testing and p-value variability: An econometrics perspectiveNorbert Hirschauer, Sven Grüner, Oliver Mußhoff — 2018
• Paving the Way to Eureka—Introducing “Dira” as an Experimental Paradigm to Observe the Process of Creative Problem SolvingFrank Loesche, Jeremy Goslin, Guido Bugmann — 2018
• Investigating the moment when solutions emerge in problem solvingFrank Lösche — 2018
• Knowing how effective an intervention, treatment, or manipulation is and increasing replication rates: accuracy in parameter estimation as a partial solution to the replication crisisGjalt-Jorn Peters, Rik Crutzen — 2020
• Effect of fine and gross motor training or motor imagery, delivered via novel or routine modes, on cognitive functionNatalia Brandwayn, David Restrepo, Adriana Marcela Martínez Martínez — 2020
• Inferential Statistics as Descriptive Statistics: There Is No Replication Crisis if We Don’t Expect ReplicationValentin Amrhein, David Trafimow, Sander Greenland — 2018
• Beyond p values: utilizing multiple methods to evaluate evidenceK. D. Valentine, Erin Michelle Buchanan, John E. Scofield — 2019
• A Proposed Hybrid Effect Size Plus p-Value Criterion: Empirical Evidence Supporting its UseWilliam M. Goodman, Susan E. Spruill, Eugene Komaroff — 2019
• 心理学の社会貢献に関する私見：教育・発達領域の論文を読んで友永 雅己 — 2017
• Brain event-related potentials predict individual differences in inhibitory control.Laura M. Rueda‐Delgado, Laura O’Halloran, Nadja Enz — 2019
• Interpretable hypothesis testsVictor Coscrato, Luís Gustavo Esteves, Rafael Izbicki — 2019
• Sparse Sampling of Silence Type I Errors With an Emphasis on Primary Auditory CortexFrancis A. M. Manno, Juan Fernández-Ruíz, Sinai H. C. Manno — 2019
• False confidence, non-additive beliefs, and valid statistical inferenceRyan Martin — 2016

Related papers

• Issues in biomedical statistics: statistical inference.John Ludbrook, Hugh Dudley — 1994
• Are Only p-Values Less Than 0.05 Significant? A p-Value Greater Than 0.05 Is Also Significant!Sang Gyu Kwak — 2023
• Review on Problems with Null Hypothesis Significance Testing in Dental Research and Its AlternativesKwang-Hee Lee — 2013
• Significance Testing in Accounting Research: A Critical Evaluation Based on EvidenceJae H. Kim, Kamran Ahmed, Philip Inyeob Ji — 2018
• Problems and alternatives of testing significance using null hypothesis and P-value in food researchWon-Seok Choi — 2023