How does the effect size inform the design (or analysis) of an NHST?

Cross Validated Asked on January 3, 2022

Consider this answer on how to design an NHST. I don’t quite understand what exact process one is supposed to follow to determine the minimum sample size once we have:

  • A null hypothesis that is falsifiable. The choice of this null hypothesis will inform the assumed distribution of the data under the null.
  • A minimum effect size that we want to be able to observe.
  • A maximum false positive rate, i.e. the significant level $alpha$ against which we’ll compare a computed $p$ value.
  • The maximum false negative rate = 1-$beta$, i.e. where $beta$ is called the power or sensitivity of the experiment.

If simpler, let’s assume a 1D Normal distribution for the Null.

My recollection is that one is supposed to use the CDF of the Null somehow for the test, but how? E.g. how does the chosen value for the minimum effect size inform the sample size?

Add your own answers!

Related Questions

Number of samples and R^2

1  Asked on November 16, 2021 by ss_19


How to estimate the leafsize of the kd-tree?

1  Asked on November 12, 2021 by curiosus


Ask a Question

Get help from others!

© 2023 All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP