Making Null Results Credible: An Overview of Design and Analytical Tools

Abstract

Despite widespread acknowledgement of the importance of disseminating null results, researchers often struggle to successfully publish null findings. One common criticism leveled against such findings is that null results could be driven by design factors like inadequate sample size or measurement error. In this essay, we provide an overview of several tools and practices that researchers can implement, both during the design stage and during post-hoc analysis, to make null results more credible. Specifically, as researchers design their studies, they can make use of power analysis and preregistration, while taking care to follow best practices for variable measurement and—in the case of experimental studies—manipulation checks. During the analysis stage, researchers can move beyond “failing to reject the null hypothesis” by using confidence intervals, equivalence tests (such as the two one-sided tests (TOST) procedure), or Bayesian statistical approaches such as the Bayes Factor.