The Premises of Howson's Argument

In my last post about the philosophy comp, I summarized Howson’s argument against the severity requirement as follows:

1. In the case of a positive test result, the severity requirement yields the conclusion that there is good evidence that the patient has the disease.
2. In that same case, Bayes’ theorem yields the conclusion that the probability that the patient has the disease is low.
3. The severity requirement and Bayes’ theorem are inference rules.
4. The claim that there is good evidence that the patient has the disease is incompatible with the claim that the probability that the patient has the disease is low.
5. If two inference rules applied to the same case yield incompatible conclusions, and one of those inference rules is sound in its application to that case, then the other inference rule is unsound.
6. Bayes’ theorem is sound in its application to the case of a positive test result.
7. Therefore, the severity requirement is unsound.

The argument is deductive, so evaluating it requires only examining the premises.

Let’s start with (1). Here, again, is Mayo’s severity requirement:

• Data x in test T provide good evidence for inferring H (just) to the extent that hypothesis H has passed a severe test T with x.

And here is her characterization of “severe test:”

• Hypothesis H passes a severe test T with x if (and only if):
• (i) x agrees with or “fits” H (for a suitable notion of fit), and
• (ii) test T would (with very high probability) have produced a result that fits H less well than x does, if H were false or incorrect.

Are these conditions satisfied in the case of a positive test result? Here again is the example. It involves a diagnostic test for a given disease. The incidence of the disease in the relevant population is 1 in 1000. The test yields a dichotomous outcome: positive (+) or negative (-). It gives 0% false negatives and 5% false positives, and we are able to estimate these numbers to a high degree of precision from available data.

A particular patient (chosen at random from the relevant population) gets a positive test result. According to Howson, the hypothesis that that patient has the disease passes a severe test with that result. The positive result does seem to satisfy (i): it agrees or “fit” the hypothesis H that the patient has the disease. Mayo is deliberately vague about the notion of fit, but it’s hard to see how she could deny that the positive test result fits the hypothesis that the patient has the disease without invoking the prior probabilities that her account is deliberately designed to avoid. After all, PH(+), the probability of a positive result in a hypothetical situation in which H is true, is 1.

The positive result also seems to satisfy (ii): the test would have produced a negative result, which fits H less well than the positive result does, with probability .95, if H were false. (If .95 isn’t high enough, let it be higher. Just lower the prior probability appropriately and the counterexample will still work.)

Given that the positive test result satisfies (i) and (ii), Mayo’s account says that it provides good evidence for inferring H.

One way in which Mayo could defend herself would be to claim that she only means “good evidence” in an incremental sense. The fact that a hypothesis passes a severe test does not mean that the hypothesis is belief-worthy; it only means that the hypothesis is more belief-worthy than it was without the test. That claims would be consistent with the Bayesian result that P(H|E)=.02, which is twenty times P(H)=.001. However, Mayo does not take that approach.

I believe that Spanos would deny premise (1) of Howson’s argument, but I have not worked through his argument carefully enough to understand what he is saying. For the moment, Howson appears to be on strong ground with (1).

(2)-(5) all seem to me innocuous. I’m sure one could quibble about the phrase “inference rule,” but I don’t think that doing so would get one anywhere. To deny (4) seems absurd, but I believe that Mayo does so in her paper “Evidence as Passing Severe Tests: Highly Probably versus Highly Probed Hypotheses.” I’ll consider her reasoning in that paper in a later post.

I worded (6) carefully so as not to make Howson’s argument depend on strong claims about the validity of subjective Bayesianism. All he needs to make his case is that it is appropriate to use Bayes’ theorem in this case to calculate the probability that the patient has the disease given that he got a positive result. It seems to me that frequentist scruples have gone too far when they lead one to deny this claim. We have even supposed that the patient in question was randomly chosen, and we may suppose that we know nothing further about him except that he received a positive test result. Mayo denies (6) by claiming that one commits a “fallacy of instantiating probabilities” in moving from the claim that there is a 2% chance that a randomly chosen person with a positive test result has the disease to the claim that this person, who was randomly chosen and has a positive test result, has a probability of 2% of having the disease. After all, this person either has the disease or doesn’t. I see the response, but it seems to me ill-motivated. Why not instantiate the probability and regard it as epistemic? You would seem to do better that way than by using Mayo’s severity requirement.

I am starting to wonder whether the frequentist responses to this kind of argument are worth the trouble to examine, when the argument itself seems quite clear and quite devastating. That’s what the Bayesians who raised this objection have been thinking, it seems. There are four motivations, however, that make me want to continue with this project. First, I hope that by scrutinizing frequentist responses to the base-rate objection I will come to appreciate better the overall error-statistical framework, which I think has some real insights into scientific practice even if its theory of evidence is fatally flawed. Second, if the frequentist responses are fatally flawed, then there should be a paper in the published literature explaining how they are flawed. Third, I am open to the possibility that there is a strong response to Howson’s argument that I have overlooked or not taken sufficiently seriously. Finally, I need a philosophy comp!