Methodism and Frequentism

Some philosophers of science focus on questions like "What are the necessary and sufficient conditions for data D to count as evidence for hypothesis H?"  A few think that questions of this kind are misguided and instead focus on questions like "How do the methods used in science help us get closer to the truth?"  I will call the first group evidentialists and the second group methodists.  (With apologies to mainstream epistemologists, who use these terms in slightly different ways, as well as to the relevant group of mainline Protestants.  And also to my readers for the fact that this distinction may not be entirely clear and my terminology across posts may not be entirely uniform.  This research is a work in progress!)

Frequentist methods tend to appeal to methodists because they are typically taken to derive their warrant not primarily from intuitions about evidential relations but instead from their long-run operating characteristics.  For instance, a uniformly most powerful level α null hypothesis significance test is typically taken to be a good procedure (when it is so taken) because it has the highest probability of rejecting the null hypothesis when it is false among all tests that have probability α or less of rejecting the null hypothesis when it is true.  By contrast, Bayesian methods are typically taken to derive their warrant from considerations of coherence or consistency with plausible axioms about rational inference or behavior.


Although the typical way of motivating frequentist methods has a methodist behavior while the typical way of motivating Bayesian methods does not, a methodist need not be a frequentist.  The frequentist methods in common use are appealing because they can be proven to have certain desirable operating characteristics when their assumptions are satisfied.  However, their desirable operating characteristics are typically much weaker than they seem at first glance and are compatible with terrible performance in important respects.  And Bayesian methods often perform better; they just can't be proven to have any particular truth-related virtues (except from the standpoint of the degrees of belief of a particular agent) because their performance depends on the relationship between their priors and the true state of affairs.  Methodists are not committed to preferring methods with weak but provable performance characteristics over methods that may or may not perform better depending on the circumstances.