What's Wrong With Single Hypotheses

The classic description of the scientific method begins with devising a hypothesis. The problem with starting with a hypothesis, however, is that bias and self-delusion can arise owing to an emotional attachment to the hypothesis, as Chamberlin honestly described in 1897:The moment one has offered an original explanation for a phenomenon which seems satisfactory, that moment affection for [one's] intellectual child springs into existence, and as the explanation grows into a definite theory [one's] parental affections cluster about [the] offspring and it grows more and more dear ... There springs up also unwittingly a pressing of the theory to make it fit the facts and a pressing of the facts to make them fit the theory ...1The temptation to misinterpret results that contradict the desired hypothesis is probably irresistible. This mistake occurs repeatedly in the history of science. Some examples were collected by Langmuir,2 who correctly called them "pathological science."Peer-review of research can help avoid these mistakes, so long as the reviewers are not in the thrall of the same hypothesis. But if there is a shared enthrallment among the reviewers in a commonly believed hypothesis, then innovation becomes difficult because alternative hypotheses are not seriously considered, and sometimes not even permitted.To speed scientific progress, Platt suggested the use of "strong-inference,"3 a thumbnail sketch of which includes:1. Identify a scientifically interesting observation.2. Enumerate all alternative hypotheses that can account for the observation, based on present knowledge.3. Reject hypotheses by experimental observations until a single hypothesis remains that has survived an experimental test by which it could have been rejected. The remaining hypothesis is the currently held view of the cause of the observation.To the laity, the remaining hypothesis is truth, but the scientist knows that this currently held view can change if new hypotheses arise from new knowledge. The presence of strong-inference in publications and grant proposals can identify those areas of science that are rapidly evolving, and if funding were to be shifted towards these areas, progress in science would be more rapid and efficient.Yet, despite its logical structure, strong-inference does not deal with the following question: What method can efficiently generate the observations needed to start new cycles? Strong-inference is neither useful nor needed for creating the critical seed observation upon which one can base alternative hypotheses. Instead, we need strong-inference-plus, which consists of the exploratory, pilot, and hypotheses-testing phases.The exploratory phase has different goals (and uses different methods) than the other two phases. During the exploratory phase the factors that influence success are unclear, so the research must proceed by way of hunches, serendipitous observations, and selections based on inadequate information. Pet theories are certainly acceptable here, if they work. Such nonscientific procedures are necessary whenever the parameter space is too large for an exhaustive search. It is here that scientific creativity can play a crucial role, as years of experience may enhance or hinder discovery. And here is where new scientific instruments or methods might demonstrate their worth.When the exploratory phase provides reasonably reliable observations, the research enters the pilot phase, during which a small number of experiments, usually six to eight, are replicated under identical experimental conditions. The pilot phase is the experiential test of the exploratory phase. A statistical power calculation of the pilot data will predict the size of study necessary in order for the hypotheses-testing phase to have a reasonable probability of success.If the analysis leads to further funding, the project moves into the hypotheses-testing phase. None of the experimental data from the exploratory and pilot phases can be reused in the hypotheses-testing phase, in order to avoid statistical bias. The hypotheses-testing phase will be most efficient when the multiple hypotheses of strong-inference are utilized to guide the experiments. Quality in strong-inference is judged on the efficiency of ruling out alternative hypotheses. Quality in the exploratory phase of strong-inference-plus should be judged on how interesting the observations are, and especially whether the observations cannot be explained by the "standard view." The observations that are not explainable by current scientific theories are the most valuable, for they may propel the field forward in the next cycle of innovation, possibly to a paradigm shift.Strong-inference-plus is a paradigm shift for some scientists. For example, it is common knowledge that R21 exploratory research grants at NIH are evaluated not in terms of the exploratory phase, but instead, are critiqued as if the proposal were for R01 hypothesis-testing research. Explicitly labeling the phases of strong-inference-plus makes it easier for students and researchers to identify changes in the approach to experiment and to data as new knowledge is first encountered, then elaborated on, and finally codified.Don L. Jewett (djewett@itsa.ucsf.edu) is an emeritus professor at the University of California, San Francisco, and research director at Abratech, a small biotech R&D company.

What's Wrong With Single Hypotheses

The classic description of the scientific method begins with devising a hypothesis.

Interested in reading more?

Become a Member of