Howson's Argument Against the Severity Requirement

To my knowledge, Colin Howson was the first person to raise in print the objection that Deborah Mayo’s error-statistical theory of evidence is subject to the base-rate fallacy.  (He is far from the first to have raised the point with regard to frequentist hypothesis testing in general.)  Howson presented his case in a PSA symposium paper that was published in 1997.  Mayo and Ronald Giere also presented papers at this symposium.  I’ll start my attempts to make sense of this debate by examining Howson’s paper.

Howson presents what he takes to be a counterexample to Mayo’s account: a test in which he takes a particular result to satisfy Mayo’s requirements to count as good evidence for H, such that if you infer the correctness of H from that result then you will draw the wrong conclusion nearly every time that result occurs.  Howson’s example involves a diagnostic test for a given disease.  Let H be the hypothesis that the disease is present in a particular, randomly chosen test subject.  Suppose that the test yields a dichotomous outcome: positive (+) or negative (-).  In addition, suppose that the test yields 0% false negatives, and that we have adequate data to estimate that it has 0% false negatives (to within a finite margin of error that one can take to be as small as one likes).

A Bayesian would generally express the stipulation that the test yields 0% false negatives with a conditional probability: P(-|H)=0 (which entails P(+|H)=1, because + and - outcomes are mutually exclusive and exhaustive outcomes).  However, Mayo objects to conditioning on hypotheses because in the most standard axiomatizations of the probability calculus, a conditional probability is defined as the probability of the conjunction of the item in question and the item conditioned on divided by the probability of the item conditioned on: for instance, P(+|H) =_df P(+&H)/P(H).  Mayo denies that it is appropriate to apply speak of probability for anything other than physical chances.  The probability of H in this case is not a physical chance.  The physical chance that the patient in question has the disease in question is either 0 or 1: she either has it or she doesn’t.  Intermediate values for P(H) represent not a physical chance, but a degree of belief. 

Howson anticipates this objection and avoids it by using an alternative notation to signify that he is not speaking of conditional probabilities, but of probabilities (which one could think of as physical chances) that would obtain in a hypothetical situation.  For instance, he uses P_H(+) (rather than P(+|H)) to signify the probability of a positive result for a patient who does in fact have the disease.

Thus, Howson expresses the fact that the test yields 0% false negatives using the notation P_H(-)=0 (which is equivalent to P_H(+)=1).  He supposes also that the test has a false positive rate of 5%, i.e. P_~H(+)=.05, and that again we have sufficient data to estimate this value with as much precision as one likes.  In addition, Howson supposes that the disease has a small incidence—say 1 in 1000—in the population from which the patient in question has been drawn.

In this case, Howson claims, Mayo’s error-statistical theory yields the conclusion that a positive test result provides good evidence that the patient in question has the disease in question.  On the other hand, a Bayesian calculation using the incidence rate of the disease as the prior probability shows that P(H|+) is quite small—less than 2%.  Howson regards Bayesianism as providing a normatively correct logic of probability judgments, so he concludes that the fact that Mayo’s theory endorses a conclusion that a Bayesian calculation assigns a posterior probability of only 2% indicates that Mayo’s theory provides an unsound inference rule.

In summary, Howson’s argument has the following structure:

1. In the case of a positive test result, the severity requirement yields the conclusion that there is good evidence that the patient has the disease.
2.  In that same case, Bayes’ theorem yields the conclusion that the probability that the patient has the disease is low.
3. The severity requirement and Bayes’ theorem are inference rules.
4. The claim that there is good evidence that the patient has the disease is incompatible with the claim that the probability that the patient has the disease is low.
5. If two inference rules applied to the same case yield incompatible conclusions, and one of those inference rules is sound in its application to that case, then the other inference rule is unsound.
6. Bayes’ theorem is sound in its application to the case of a positive test result.
7. Therefore, the severity requirement is unsound.

(7) follows from (1)-(6), so a defender of the severity requirement as a sound inference rule faces a challenge to find a flaw in one or more of (1)-(7).  If what I wrote in a previous post is correct, then Spanos argues against (1), while Mayo argues against (4).  I’ll start working through the premises of Howson's argument in my next post.