Issues with the orthodox statistical test

The basic logic of the orthodox statistical test involves defining a null hypothesis H0, deriving a p-value - which is the probability of obtaining data as extreme as that actually observed, given H0 - and rejecting H0 if the p-value is small.

There are a number of problems with this approach:

A solution to most of these problems is to actually calculate P(H0|data) using Bayes theorem, i.e. P(H0|data) = P(data|H0)P(H0)/P(data). See the references linked from the main page for details of how this can be done in practice, and for expanded discussion of each of the points above.