Advertisement
Salesforce
Salesforce

The Problem of Perception

Your interpretation of results depends on more than just the results.

By | March 1, 2009

There is a common perception among young students that the surest path to resolving scientific controversies is to design a clever experiment, one that will definitively resolve conflicting hypotheses. However, I have found that most scientific controversies do not revolve around specific experimental data, but instead are disputes over data interpretation.

Data interpretations depend on a scientist's underlying assumptions and worldview. For example, a molecular biologist might think of protein expression as an outcome of mRNA levels, whereas a biochemist might think in terms of synthetic and degradation rates. Both are right, of course, but each might expect different reasons for a change in the amount of a protein. Our perspective and assumptions regarding how living systems work defines us as biologists, which is why arguments over interpretations can get so nasty. If another scientist disputes the validity of your viewpoint, it can impact your reputation as well as your ego.

I experienced a dispute over data interpretation in the late 1980's that turned somewhat contentious. My laboratory was investigating the role of tyrosine kinase activity in the endocytosis of the EGF receptor in collaboration with Gordon Gill at the University of California, San Diego. By using a series of kinetic measurements, we concluded that kinase activity was necessary for ligand-induced endocytosis and down-regulation, whereas a competing group thought that it was not. What was interesting about the dispute was that the primary data gathered by both groups were essentially identical. The main difference was in how the data were interpreted.

We were calculating receptor endocytosis rates on a per receptor basis whereas the other group used a per cell basis. In the case of receptors with kinase activity, EGF binding accelerated endocytosis 10-fold, which caused cell-surface receptors to drop 10-fold. Thus, the net rate of endocytosis per-cell (rate x receptors) did not change in response to kinase activity, making it appear to have no effect on receptor internalization.

It was hard to convince the other group that differences in a simple rate calculation would make a big difference in what they concluded from the data. However, we were working from a computational model of endocytosis that allowed us to try out different sets of assumptions and see how they would affect the system's behavior. The other group felt that our computer model was a poor substitute for their own scientific intuition regarding what was happening. As they stated at the time, "(Our) results are in agreement with the results of Chen et al. (Cell, 59:33–43,1989), but in conflict with their interpretation. By applying a simple mathematical model (J Cell Biol, 107:801–10, 1988), the lack of down-regulation of kinase-negative mutants was misinterpreted as an indication for reduced endocytosis."1

Over time, our view prevailed and it is now generally accepted that the kinase activity of the EGF receptor is required for ligand-induced endocytosis.2 Interestingly, our view was vindicated not because people came to accept our use of computational modeling, but because our hypothesis was more successful in predicting subsequent experimental results. Scientists don't generally care about who is right or who is wrong in a dispute. They want a conclusion that can help predict their own experimental outcomes. Science is built brick by brick from ideas and concepts that can lead to the next successful series of experiments and concepts. If an idea doesn't support the next brick, it is discarded. It's natural selection in science.

Scientific disputes seem inevitable in any career, but mine gave me a keen appreciation of the need for caution in accepting simple interpretations of the behavior of complex systems. In science, we do not gather facts. We make observations. Our interpretation of observations is only as good as our assumptions and conceptual frameworks.

The ability of a simple computer model to correctly interpret a seemingly non-intuitive result was also quite revealing. It convinced me that as biology becomes even more complex, we will need computational models even more to help us out. They can not only let us see past our inherent interpretational biases, but can also be used to design experiments to test our concepts—a foundation for building a truly predictive biology.

Steven Wiley is a Pacific Northwest National Laboratory Fellow and director of PNNL's Biomolecular Systems Initiative.

References

1. A. Ullrich, and J. Schlessinger, "Signal transduction by receptors with tyrosine kinase activity," Cell, 61:203–12, 1990. 2. A. Sorkin, and L.K. Goh, "Endocytosis and intracellular trafficking of ErbBs," Experimental Cell Res, 314:3093–106, 2008.

Comments

Avatar of: anonymous poster

anonymous poster

Posts: 1

March 3, 2009

In a paragraph you explain: Science is built brick by brick from ideas and concepts that can lead to the next successful series of experiments and concepts. If an idea doesn't support the next brick, it is discarded. It's natural selection in science. \nThere is a serious conceptual error in this sentence. The natural selection, at least the formally understood, does not operate like that. This way or mode of operation, would be an intelligent design. \nAs the investigator, behaves knowing towards where it leads, ie, has an ultimate goal (predicting successful experimental results). This is not natural selection. The error of concept is for me basic and seriously.
Avatar of: Ruth Rosin

Ruth Rosin

Posts: 117

March 3, 2009

Of course!\n\nExperimental results do not speak for themselves. In fact, they do not speak at all. Instead, it is the scientists who speak for the results by interpreting them.\n\nAnd since the results do not speak at all, they can never give scientists any clue as to how to interpret them!
Avatar of: Gary Huber

Gary Huber

Posts: 23

March 10, 2009

One of the books on my to-read list is Thomas Kuhn's "The Structure of Scientific Revolutions". It is my understanding that he discusses this observation at great length.
Avatar of: Michael Holloway

Michael Holloway

Posts: 55

March 10, 2009

Yes, thank you. Us poor molecular biologists are vaguely aware of protein level regulation, even if it is inferior and not as interesting.\n\nThe term "worldview" is something you see alot of at the creationism museum outside of Cincinnatti, and makes me cringe. For the AIG crowd, a different "worldview" can legitimate any "scientific" conclusion. \n\nI think your experience is a better anecdote for the necessity of peer review in science. Science is done in a community.
Avatar of: Nils Jansma

Nils Jansma

Posts: 19

March 10, 2009

To Anonymous poster:\n\nI appreciate the observations of the Anonymous poster. However, it just could be that Intelligent Design (Theistic Evolution TE) is involved and thus a viable theory. If you assume that TE is an invalid theory, you have just limited the use of your findings to supporting your assumptions just as the article cautions. An analogy would be an investigation of a roulette wheel. If you could construct an algorithm that would consistently predict the outcome of a roulette spin, how would you interpret the data? If you assume that the wheel is not fixed, that would greatly limit the successful use of your findings. It just may be that the wheel is fixed which would validate the findings. So, instead of assuming TE is incorrect and altering our data interpretations accordingly, why don?t we just let the chips fall where they will and forget, if possible, our worldview biases.\n
Avatar of: David Ross

David Ross

Posts: 1

March 10, 2009

"The structure of scientific revolutions" contains references to a number of analogous situations where a priori assumptions ran rampant over data. Still makes interesting reading.
Avatar of: Tim Edwards

Tim Edwards

Posts: 1

March 10, 2009

One thing's for sure: Wiley's example is fascinating.\n\nUnlike other commenters, I'm not sure if Kuhn applies here. His work, "The Structure of Scientific Revolutions" (1962), concerns disputes between rival scientific "paradigms." These are large-scale conflicts between such wholesale theories as Ptolemy vs. Copernicus or Classical Atomic Theory vs. Quantum Mechanics. \n\nOn the other hand, the sticking point between Wiley and rivals seems pretty specific: whether "the kinase activity of the EGF receptor is required for ligand-induced endocytosis" (paragraph 6).\n \nSo I'm not sure whether the dispute Wiley mentions would count, for Kuhn, as a dispute between "paradigms"--or, rather, as a dispute *within* a common "paradigm." \n\n(Of course, it would have helped if Kuhn had precisely defined "paradigm." But, beyond a few generalities, he really didn't. Perhaps that partially accounts for the success of Kuhn's book: "paradigm" becomes an empty vessel in which anyone can drop all sorts speculations.) \n\nThe sentence of Wiley's I'd focus on is this one: "our view was vindicated...because our hypothesis was more successful in predicting experimental results." To me, that seems more amenable to a philosophy like Karl Popper's (when he's carefully read, not caricatured) or Norwood Russell Hanson's, than Kuhn's.
Avatar of: Brent Neumann

Brent Neumann

Posts: 4

March 10, 2009

The majority or biomedical research is hypothesis-driven, as the funding system requires it to be this way.\nProblems can occur when when experiments are designed to fit the hypothesis such that no other outcomes are deemed plausible and so if they occur, experimental designs are changed or results are ignored. We as scientists need to be more open to unexpected results and even those that oppose our line of thought.\n\nThere is a need for more negative results to be published and journals need to be more willing to accept these types of papers. Publication of negative or conflicting data allows the field to re-examine previous data and their perceived outcomes.
Avatar of: Ruth Rosin

Ruth Rosin

Posts: 117

March 10, 2009

Kuhn provided a long partial list of criteria which scientists use to decide between competing paradigms.\n\nHe stressed that his list is incomplete, and that subjective criteria still become involved, because different scientist may subjectively attach a different weight to one and the same criterion on the list. He did not say so, but when you carefully examine his list, you realise that some items on the list are inter-dependent. (For instance, you can reduce the number of internal contradictions by increasing the number of auxiliary hypotheses included in a paradigm.)\n\nHowever, contrary to your judgement, his list certainly includes the ability to predict future results!
Avatar of: Ruth Rosin

Ruth Rosin

Posts: 117

March 10, 2009

I cannot claim this to be a general rule. A controversy of small dimensions can, however, constitute a reflection of another controversy of far wider dimensions and importance.\n\nFor instance, the honeybee "dance language" (DL) controversy, ongoing now for over 40 years, (even tough many scientists are not even aware of it, because DL opponents, who would never relinquish their opposition, are continually denied the right to be heard), had long become the most important reflection of a far wider, older, and much more basic controversy over the very foundations of the whole field of Behavioral Science, between European Ethology (co-founded by Lorenz & Tinbergen in 1935) and Schneirla's School in Behavior (not to be confused with Behaviorism).\n\nK. von Frisch was awarded a 1973 Nobel Prize for the "discovery" of the "instinctual" DL of honeybees. The Prize was, not by accident, shared by Lorenz & Tinbergen, the co-founder of a general approach to Behavior that is based on the belief in the existence of genetically predetermined traits, known as "instincts' in Behavior. Schneirla's School is based, instead, on Morgan's Canon, and all the ideas which led Lloyd C. Morgan (a young contemporary of Ch. Darwin), to formulate his well-known Canon, and on the conclusion that all individual traits (including behavioral traits), of all living organisms, develop ontogenetically (in the individual organism), under inseparable (!) effects of both (!) genes & environment; which means that "instincts' do not exist at all!\n\nMorgan, like Darwin, and all other biologists of that day, believed in the existence of "instincts". Fortunately, Morgan's Canon, and all the ideas underlying it, are totally independent of his belief in "instincts", and remained intact when Schneirla kicked "instincts" out.\n\nYou don't hear much about the general controversy nowadays, perhaps because disciples of Schneirla's School have completely lost patience with European Ethologists. But European Ethologists seem to gradually realise there is something seriously wrong in their approach to Behavior, and to Biology in general, and have begun relatively recently, to speak of epigenesis. They are still very far from Schneirla's School, but they seem heading in that direction.\n\nThe honeybee DL controversy, which, on the surface of it, seems to concern only honeybee behavior, had, however, long become the most important reflection of the general controversy between European Ethology and Schneirla's School.
Avatar of: Melissa McCoy

Melissa McCoy

Posts: 4

March 10, 2009

In this article it is written:\n\n'We were calculating receptor endocytosis rates on a per receptor basis whereas the other group used a per cell basis. In the case of receptors with kinase activity, EGF binding accelerated endocytosis 10-fold, which caused cell-surface receptors to drop 10-fold. Thus, the net rate of endocytosis per-cell (rate x receptors) did not change in response to kinase activity, making it appear to have no effect on receptor internalization.'\n\nThe use of the word "Thus" is misplaced (since the rest of the sentence seems in disagreement with the previous sentence), and so makes understanding of the ideas presented much more difficult. "However", on the other hand, would make sense. If scientists could learn to write and use grammar correctly, and so communicate more effectively, perhaps our differences in interpretation would dwindle.

March 11, 2009

It is not merely interpretation of Data that poses problems.The act of perception itself is difficult to understand.Perception, at the rudimentary level,is the process by which the stimulii are received and interpreted by the brain.We are aware of the brain/brain's existence only by its receipt and interpretation of stimulii.So, we need brain to perceive and the things perceived to prove the existence of brain.\nThe interplay of senses and objects to be perceived,interplay of senses and brain determine Perception.We are aware of the senses because of brain and brain because of senses and objects perceived.\nAt the external level, what constitutes a whole object is difficult to fathom.Is it because of a particular trait we perceive a thing as we see it or an integtrated traits /qualities that makes us aware of an object as a whole?\nHow do we recognise objects as such?\nIs it only through senses? In that case , how did we recognise it as a particular object at the first instance?If it is because we have been told so, how did the first individual recognise it as such?\nFurther, things change continuously and same thing or event presents a different picture from a different angle in case of vision(ex:see different angles from the camera in a game, say cricket or foot ball) and different interpretation when we perceive at different times the same thing /event.. \nThe answer is in Indian Philosophical treatises.

March 11, 2009

Interesting article as usual with this author. Regarding Steven Wiley´s commentary ?Scientists don't generally care about who is right or who is wrong in a dispute. They want a conclusion that can help predict their own experimental outcomes. Science is built brick by brick from ideas and concepts that can lead to the next successful series of experiments and concepts. If an idea doesn't support the next brick, it is discarded. It's natural selection in science.? \nI absolutely agree, with this statement. We are not aware of the fact that when we approach a scientific problem, our brain generally applies "Ockham's Razor", an epistemological rule that implies that when we select among different hypothesis that explain experimental results (i.e the hypothesis that best explains the ?reality? of what we observe), we generally choose the simplest hypothesis, which surprisingly, is the one that has the higher ability to predict future result. Sometimes we forget that the use of the Razor, often not even aware of it, has had an enormous impact in science (see Rodríguez-Fernández, JL. (1999). "Ockham's Razor". Endeavour 23: 121?125).\n
Avatar of: Michael Holloway

Michael Holloway

Posts: 55

March 11, 2009

"However, it just could be that Intelligent Design (Theistic Evolution TE) is involved and thus a viable theory."\n\nTwo errors in one sentence. Theistic Evolution certainly is not Intelligent Design. Intelligent Design is a faction of scientific creationism that grew out of the 1987 United States Supreme Court Edwards v. Aguillard case that smacked down the injection of scientific creationism into public school science classes. Intelligent Design evolved as a means of moving the anti-science education campaign away from the preceived legal vulnerabilities of the failed scientific creationism movement. Exactly like scientific creationism though, ID is just a collection of propaganda arguments that seek to convince the general naive public that evolution and natural selection are false assertions. The collection is large and fluid, changing in whatever way the speaker thinks is necessary to make their audience uncomfortable with science education. ID is essentially the trademark of The Discovery Institute, a far right political think tank. One of their standard propaganda points is that their detractors never define ID correctly, and misrepresent it. They then decline to define it. They falsely assert that they have publications in peer review research journals, and have done no research to support a scientific theory.\n\nTheistic evolution is not a term asserting a "scientific" theory, and doesn't argue against science education. It is simply the religous belief that God is principle cause of creation, but does not have any objection to scientific knowledge. It is creationism, in the classical and simple sense of the word, that acknowledges current science. The term can refer to a range of different theological concepts and understandings. It is ridiculed by both of the warring factions that insist the religion must be opposed to science (anti-science religous fundamentalists, and prostletizing atheists), but essentially defines the philosophical stance of the majority of the population that has no problem with teaching science in public schools.
Avatar of: Michael Holloway

Michael Holloway

Posts: 55

March 11, 2009

I should add to my previous post below that ID has been selected against in the courts now, same as scientific creationism before it. The anti-science education campaign strategy has again decided that its the label that's the problem. This time they're making a greater effort to obscure terms, selecting code phrases that they believe aren't as easy to recognize as religious viewpoints. "Academic freedom", and "teach the controversy" are the two most often used at the moment, and the basis for legislation being introduced across the US. \n\nIt should be pointed out that the courts are not blocking the teaching of pseudoscience in US public school science classes. Laws such as those in Louisiana and Kentucky, and legislation being introduced in other states, are a symptom, not the cause, of the anti-science education campaign. We (that is pro-science education) are losing. The courts can't change the attitudes of teachers, school administrators, and politicians. So long as the debate over good biology instruction is rigidly framed as a false dichotomy between religion and science we will lose. If your work depends on publicly funded grants, and you aren't gravely concerned about this, there is something wrong with your thinking.
Avatar of: Matt Cuthbert

Matt Cuthbert

Posts: 1

March 12, 2009

Great article with an insightful example of what it means to interpret data. Of course this has little bearing on the ID/evolution argument since neither of them are science; both are history.
Avatar of: anonymous poster

anonymous poster

Posts: 7

March 12, 2009

Brent Neuman's post is insightful and practical. Thanks!
Avatar of: Pekka Kohonen

Pekka Kohonen

Posts: 2

March 14, 2009

Why were evolution, intelligent design and such again brought forward when the article itself is internal to the scientific discourse. As far as I understand, it deals with the interpretation of scientific observations and has nothing to do with faith-based knowledge systems. Maybe these anti-science considerations themselves could be partially holding back the sciences themselves, particularly biosciences. The need to keep science "simple" and reductionist could end up hampering discussion on the basis of scientific knowledge - for instance regarding systems biology and the usefulness of computer models. We should not let the fear of appearing weak hamper any useful developments. \n\nScientific and faith-based knowledge systems are entirely separate strands of human endeavor. Since they start from entirely different basic premises,faith vs. evidence, they can never be combined - even in the very unlikely event that the faith-based knowledge of any particular religion would end up being correct. Science would still need to arrive at that conclusion on its own and could not accept any faith-based assurances. It is regrettable that so many people are confused and cannot see the underlying differences beyond terms and sound bites. Science should abandon the idea of absolute knowledge and faith should stop seeking objective confirmation of its truths and return to its own roots: "Blessed are those who do not see, but yet believe."

Follow The Scientist

icon-facebook icon-linkedin icon-twitter icon-vimeo icon-youtube
Advertisement
Ingenuity
Ingenuity

Stay Connected with The Scientist

  • icon-facebook The Scientist Magazine
  • icon-facebook The Scientist Careers
  • icon-facebook Neuroscience Research Techniques
  • icon-facebook Genetic Research Techniques
  • icon-facebook Cell Culture Techniques
  • icon-facebook Microbiology and Immunology
  • icon-facebook Cancer Research and Technology
  • icon-facebook Stem Cell and Regenerative Science
Advertisement
Anova
Anova
Advertisement
NeuroScientistNews
NeuroScientistNews
Life Technologies