popper, falsification, statistics and public health (whew...)

Mike Friedman mikedf at amnh.org
Tue Nov 12 20:27:46 MST 2002

Sabri -

This is kind of off topic -- it is and it isn't, for a reason I'll
get to in a moment -- but while what you say is true -- a type II
error is the possibility of accepting a false hypothesis as true (a
failure to reject it) --  we are really talking about two different
things. The term falsifier refers to the data you are subjecting your
null hypothesis to. Only to that, not to whether the hypothesis is
true or not. It is the Type I error that is decisive, here, because
with a Type I error, your falsifier is wrong. You can never totally
eliminate this possibility, of course, but we limit it to a 5%
chance. On the other hand, in a Type II error, it is the hypothesis
that is incorrect, not the falsifier. In any case, by "easily," I
mean that evidence falsifying hypotheses is decisive, whereas
supporting evidence is just that.

By the way, courts theoretically recognize the falsification
criterion by requiring that conviction be decided by proving guilt (a
hypothesis) "beyond a reasonable doubt." The "reasonable doubt"
refers to evidence falsifying the claim. In normative science,
"reasonable doubt" is the significance level of .05. We can be 95%
sure that a falsifier is valid (and the case will be thrown out of


P.S. there are valid social criteria for critiquing the 0.05
significance in epidemiology and public health: it sets the bar too
high, as an epidemiologist friend whom I consulted points out:

>"I'd agree with your correction. Rothman talks about refutation --
>the importance
>of investigating the things that would prove your alt. hypothesis
>false. But he
>also emphasizes that the significance of .05 is overdone in epidemiology,
>recommending instead an evaluation of the p function (graph of CIs
>at all levels)
>because this gives you a better sense of the possible strength of an
>(As an aside: He reminds us that data showing a relationship where the 95% CI
>crosses 1, with an OR of 2 may equally well represent an OR of say
>10. We may be
>irresponsable as epidemiologists to say that the relationship does
>not exist.)"

In other words, a polluting company that is interested in avoiding
being connected to a cancer cluster will prefer to set the bar higher
for proof of association. This is essentially what is done in the
field of "risk assessment." But if your children are getting
leukemia, you're not going to want to hear that the association "is
not significant at the .05 level." The "beyond a reasonable doubt" is
too extreme.

In fact, it is arguable whether "innocent until proven guilty" should
even apply to toxics. Rather, community health activists argue that
any putative toxic substance should be seen as guilty until proven
innocent. And this is an anathema for statisticians (although not,
apparently, for the justice system), for reasons that go back to the
nature of falsification: you can show association by falsifying a
null hypothesis of no association (or rather, random association),
but statistically you can't assume a null hypothesis of association
and then "prove" no association. Statisticians and their bosses would
have a fit: how could they ever "prove" that PCBs are harmless?

>>Date: Mon, 11 Nov 2002 16:50:12 -0800
>>From: Sabri Oncu <soncu at pacbell.net>
>>Subject: Re: Jurriaan on Popper
>>Mike Friedman wrote:
>>>  Actually, science is about falsifying null hypotheses,
>>>  while supporting your alternative hypothsis. You can't
>>>  logically (or statistically) "prove" a hypothesis true,
>>>  but you can easily falsify it, prove it false.
>>Here is one objection and it is to the word "easily". Even when a
>>hypothesis is false, we might not be able to falsify it. To put
>>it differently, that none of us is able to falsify a hypothesis
>>doesn't mean that it is true. They call it Type II error, if I
>>remember correctly. It is also possible to falsify a true
>>hypothesis, which is what they call Type I error, as far as I
>>Just a minor point,

PLEASE clip all extraneous text before replying to a message.

More information about the Marxism mailing list