Advertisement

Opinion: A Wolf in Sheep’s Clothing

Nutrition research must overcome pseudoscientific measures and self-interest to make progress in the fight against obesity.

By | October 22, 2013

WIKIMEDIA, USDARecently, I was the lead author on a paper demonstrating that about 40 years and many millions of dollars of US nutritional surveillance data were fatally flawed. In most research domains, such a finding might be monumental; yet in nutrition epidemiology—the study of the impact of diet on health, hereafter referred to simply as “nutrition”—these results are commonplace. In fact, there is a large body of evidence demonstrating that the systematic misreporting of energy and macronutrient intake renders the results and conclusions of the vast majority of federally funded nutrition studies invalid.  

So what is going on? Is such research mere pseudoscience? And if so, how can the federal government continue to spend billions of taxpayer dollars on studies that are making no demonstrable progress in our nation’s fight against obesity and diabetes?

We may be witnessing the confluence of two inherent components of the human condition: incompetence and self-interest. Nutrition has had many colossal and costly failures. The list of dietary components claimed to reduce cardiovascular disease (CVD), prevent cognitive decline, and/or fight cancer that were later refuted via clinical trials is extensive. And while the self-correcting nature of science necessitates failure, the vast majority of nutrition’s failures were engendered by a complete lack of familiarity with the scientific method. This deficit is most apparent in the field’s reliance on self-reports of diet. Such information, to which nutrition researchers assign numeric caloric values, is rife with bias, and without the ability to corroborate or falsify the reports, the data should be considered pseudoscientific—outside the realm of scientific research.

Moreover, nutrition research fails to control for well-known, empirically supported, and in many cases obvious confounders. For example, the Food and Agriculture Organization of the United Nations and World Health Organization have repeatedly determined that human food energy requirements should be estimated using total daily energy expenditure, and that physical activity and basal energy expenditure are the primary determinants of this measure. Yet nutrition research rarely measures any form of energy expenditure or quantifies physical activity. This failure has led to a plethora of results that are suggestive of multiple and often divergent explanations, thereby obfuscating the examination of diet-health relationships. Nowhere is this fact more evident than the field’s inability to answer one simple question: What should we eat?

The responsibility for this unfortunate state of affairs rests squarely on the leaders of nutrition research. Rather than training graduate students in the scientific method, and allowing their research to serve the needs of society, the field’s leaders choose to train their mentees to serve only their own professional needs—namely, to obtain grant funding and publish their research. I have experienced these practices myself as I transitioned from student to graduate research assistant to research fellow, and colleagues continue to emphasize that this is how it must be, lest they fail to get funding and ‘feed’ their graduate students and families. But by not training mentees in the basics of science and skepticism, the nutrition field has fostered the use of measures that are so profoundly dissonant with scientific principles that they will never yield a definitive conclusion. As such, we now have multiple generations of nutrition researchers who dominate federal nutrition research and the peer review of that work, but lack the critical thinking skills necessary to critique or conduct sound scientific research.

The apparent self-interest that is driving research in this field is not limited to raising students to merely follow the herd. The subjective data yielded by poorly formulated nutrition studies are also the perfect vehicle to perpetuate a never-ending cycle of ambiguous findings leading to ever-more federal funding. The National Institutes of Health spent an estimated $2.2 billion on nutrition and obesity research in the 2012 fiscal year, a significant proportion of which was spent on research that used the pseudoscientific methods described above. The fact that nutrition researchers have known for decades that these techniques are invalid implies that the field has been perpetrating fraud against the US taxpayers for more than 40 years—far greater than any fraud perpetrated in the private sector (e.g., the Enron and Madoff scandals).

When anti-science rhetoric occurs at a Kansas school-board fight over creationism, we can nod our educated heads in silent amusement, but if multiple generations of nutrition researchers have been trained to ignore contrary evidence, to continue writing and receiving grants, and to keep publishing specious results, the scientific community as a whole has a major credibility issue. Perhaps more importantly, to waste finite health research resources on pseudo-quantitative methods and then attempt to base public health policy on these anecdotal “data” is not only inane, it is willfully fraudulent.  

The solution to this dilemma is quite simple: funding agencies must stop funding flawed nutrition research, and the editors of nutrition journals need to stop publishing the results. Given the immense amount of money invested in this field each year, this goal is much easier to state than to accomplish. Nevertheless, the health of our nation depends on nutrition finding a scientist that can disperse the wolves and lead the overly credulous nutrition flock to more productive pastures and empirically supported public health policies.

Edward Archer is a research fellow at the Arnold School of Public Health at the University of South Carolina. He recently coauthored a PLOS ONE article on this topic.

Advertisement

Add a Comment

Avatar of: You

You

Processing...
Processing...

Sign In with your LabX Media Group Passport to leave a comment

Not a member? Register Now!

LabX Media Group Passport Logo

Comments

Avatar of: Robo407

Robo407

Posts: 1

October 22, 2013

Thank you Edward for this article calling all nutrition researchers to reach for a higher standard.  All scientist should be as critical and demanding of our colleagues a higher standard.

Avatar of: Neurona

Neurona

Posts: 33

October 22, 2013

Bravo. 

Avatar of: Paul Stein

Paul Stein

Posts: 119

October 22, 2013

I commend Doctor Archer for authoring a truly brave article.  Aside from the scientific issues in nutrition research, he raised a particularly biting criticism of the trend in basic medical science research that spans almost all disciplines save possibly immunology:  to select and perform research and to train mentees "to serve their own professional needs-namely, to obtain grant funding and publish their research."  Everyone knows how this playing-it-safe attitude of solely working on what is fundable today produces little of value in the end.  While this money and paperchase does generate some laudable careers in the cloystered scientific society circles, including major lecture awards at national conferences, after these folks retire from the scene, there is little worth in all that paper, and their work is quickly relegated to dusty shelves.  For me personally, trying to translate basic science research into medical devices, there isn't much of worth from the past few decades.  For instance, I am continually disappointed with the seeming lack of real scientific advancement year after year at the annual Experimental Biology conference.  Everything presented seems to be a baby-step or a seen-that or a nothing-new or a makes-sense-based-on-what-I-remember-back-in-graduate-school-circa-1981,-but-big-deal.  

"Inane" is a totally proper term for the state of biomedical science today.  Everyone knows it, but it is to be seen if there is the proper level of gonadal fortitude to change the course of this monster battleship.  Frankly, I don't think that will happen until major outside influences collide.  I believe that when the federal money stream contracts to a trickle, and when the Chinese tuition money totally runs out when that country's leaders finally realize that it can self-generate their own scientists and don't need the U.S. anymore, the entire foundation of the scientific endeavor will crack.  Then, a realization of what is truly important to study will finally take hold.  The scientific landscape will be unrecognizable to all but the most ancient of researchers.

Avatar of: mightythor

mightythor

Posts: 40

October 22, 2013

Can anyone suggest a comprehensive general discussion (preferably recent) of sources of error/bias in nutritional studies, suitable for first year students in the health sciences professions? I try and teach students to read the literature critically, but I am neither an epidemiologist nor a professional nutritionist. I'd like to be able to be more specific on what to look for, and what to look out for, in the literature. Thanks!

Avatar of: James V. Kohl

James V. Kohl

Posts: 152

October 22, 2013

Re: "When anti-science rhetoric occurs at a Kansas school-board fight over creationism, we can nod our educated heads in silent amusement...."

Research results support the single species theory of Homo's nutrient-dependent pheromone-controlled adaptive evolution and there is also support for the claim that.the majority of genetic  variants, including potentially harmful ones, were picked up during the past 5,000–10,000 years. There is no experimental evidence that supports mutation-initiated natural selection.

Why are the amused educated heads silent when reports in Science and in Nature appear to contradict the entirety of mutations theory? Nutrition research, like all research that demands knowledge of molecular mechansims, will remain fatally flawed until a reasonable model is adopted the links epigenetic effects to genetic predispostions as detailed in:

Nutrient-dependent/pheromone-controlled adaptive evolution: a model

 

October 23, 2013

Sorry, this is wrong. While I agree most nutrition studies done today are either useless or biased, we have long known that people underrreport caloric intake, even trained health professionals know this. The discrepancies found in the NHANES data are no different than previously reported in many other studies. Good news is that there seems to be some consistency in their inaccuracy which has been maintained over time, which makes the data inaccurate but reliable. In other words, whenever anyone says one glass of wine, we know it is most often 2.  Not 3-7.  So, the data is consistently inaccurate and in that, somewhat reliable.  It is like a home body fat scale. It is not accurate but it is consistently inaccurate so you can still use it to see your trends.  It may always under report my true body fat by about 5% but that 5% error will be consistent.   The bad news, if anything, is that this means all the data we have on how bad the American diet is -- is under reported and the actual number are much, much worse. 

But the conclusion here that we can't answer the question of "what to eat" is profoundly wrong. Either that, or all these inaccurate reports and people lying have all produced the same dietary patterns in the Blue Zones and in long lived populations. 

Clearly there is nutritional research which is the exception to what is being asserted here, like research on plant-based diets...where the composition of the diet is controlled like Pritikin, Mcdougall, Barnard, Ornish, Essy and so on. These aren't just some vague recall surveys trying to match a given health outcome with a given food, but looking at a very specific diet subjects are given and at specific diseases to gauge the impact of that diet. Those are some of the few studies that clearly are not pseudo science. So I would say the issue is that much of nutritional science is a wank because research showing how to use a clinical diet to reverse and prevent specific diease is largely ignored and not reported to the public.

The root problem is that most researchers aren't really interested in the question What dietary measures produce the best consistent results? They are interested in seeing if there might be some tiny benefit to Food X in the diet, and the Food X companies are very eager for this kind of pseudo science because it aids in marketing. The researcher gets published, Food Company X gets some b.s. to try to snow the public and make money, and the beat goes on.

Students shouldn't follow this herd, but should see that Type 2 Diabetes, heart disease, obesity and autoimmune diseases can be arrested and reversed with healthy plant-based diets, and then move public policy in that direction. Except that's not where the money is, in science, and therein lies the problem.

 

Avatar of: gpinzone

gpinzone

Posts: 1

October 25, 2013

mightythor: Go on YouTube and search for "Science for Smart People" by Tom Naughton. He gives a great run down of the scientific method and examples of the past mistakes made.  There was also a great episode of Catalyst in Austrilia that went over how the dangers of saturated fat are wrong. Google "Catalyst: Heart of the Matter Part 1" and you can watch the episode online.

Avatar of: Micki

Micki

Posts: 2

October 26, 2013

Small quibble with this:

"We may be witnessing the confluence of two inherent components of the human condition: incompetence and self-interest. Nutrition has had many colossal and costly failures. The list of dietary components claimed to reduce cardiovascular disease (CVD), prevent cognitive decline, and/or fight cancer that were later refuted via clinical trials is extensive."

Indeed, we are witnessing conflicting interests when it comes to food and nutrition. However, amid the so-called components that were refuted to be beneficial to various health endpoints in clinical trials, there are some issues that were poorly addressed due to their narrow focus.

Using one nutrient, often as a synthetic isolate w/o all the naturally occurring forms of this nutrient as found in real food does NOT exclude benefits of a nutrient. It shows that a tocophorol (for example) as a synthetic isolate form of vitamin E - and I would argue it is an aberrant form - did not show benefits. E in foods may present an entirely different result.

However, there is a nutrient that deserves much more consideration and is often dubbed 'the forgotten vitamin.' Vitamin K, especially vitamin K2, is getting much more appreciation these days. Like many nutrients, especially the fat soluble ones, this vitamin comes in multiple forms and it works with other nutrients in many processses. Along with the effects of this nutrient in myriad processes, interest in the receptor for osteocalcin (one of the so-called K-dependent proteins), interest in the enzyme/gene for biosynthesis of short chain K2 (UBIAD1 or TERE1) and other aspects of this nutrient deserve real trials and real consideration. The diversity of foods containing long chain menaquinones is vast and these foods are always found in the so-called Blue Zones. Unseen, they eat fermented foods, organ meats, or highly bioavailable vitamin K1 (olive oil) and THIS is a huge benefit to multiple diverse cultures. Americans do not eat long chain menaquinones. They block biosynthesis of MK-4. This topic, this nutrient, this hormone, this antioxidant - found in diverse foods containing no-fat and high-fat and everywhere in between - needs to be looked at more closely.

Who will fund a study of a nutrient? Some are selling supplements, so they promote them. Some nations are using this nutrient therapeutically. But Americans don't eat it, don't supplement it, and are paying a dear price. We could have this nutrient in food, but our cheese is now made in a couple weeks, not the year and a half or more of fermented cheese, so this more traditional American fermented food is now altered. We shun organ meats. We eat few whole bivalves or insects. We are put off by the 'funk' of sauerkraut or natto. We eat greens, but not many, and most of the K1 is not bioavailable which is poorly understood by nutritionists (yes, kale is high in K, but we get almost none of it when we eat it unless we have a fat and even better...we cook it).

Along with too much sugar and too many refined grains, we lost a valuable fat soluble nutrient that is vastly undeappreciated. Vitamin K2.

Lets fund a study to look at this nutrient in all its permutations with no one to benefit but the entire public! Help spread the word about vitamin K2. And lets use good scientific practices. Lets understand bioavailability of all forms. Lets understand how this nutrient/hormone affects health.

Avatar of: Micki

Micki

Posts: 2

Replied to a comment from mightythor made on October 22, 2013

October 26, 2013

This may not be exactly what you are looking for, but I found Vitamins in the Prevention of Human Diseases to be an excellent textbook.

It is not so much about sources of error/bias as looking at individual nutrients via articles from diverse experts of such nutrients. It is pretty fascinating and insightful.

Avatar of: ef

ef

Posts: 1

October 28, 2013

if one carefully reads his PLoS One study, which he mentions at the bottom, and read the study's fine print..... tons of corporate $$$.... plus...   "Funding for the study was provided by an unrestricted research grant from The Coca-Cola Company." http://www.plosone.org/annotation/listThread.action?root=73017   ...says its all. 
Avatar of: Kymus

Kymus

Posts: 1

Replied to a comment from Colonel Beefeater made on October 23, 2013

October 30, 2013

But the conclusion here that we can't answer the question of "what to eat" is profoundly wrong. Either that, or all these inaccurate reports and people lying have all produced the same dietary patterns in the Blue Zones and in long lived populations. 

There is very strong data that shows that SFA do not corrleate with CHD. Despite this, the USDA, ADA, AHA, etc. still claim that there is a link and that SFA should be reduced to avoid heart disease. Likewise, I'm seeing more and more data coming out suggesting that salt isn't the health danger that it's been made out to be, either.

Certainly RD's can tell their clients what to eat, but it's not really based on prospective data. This goes back to what Mr. Archer said: the data we have is unreliable and unscientific. This is similar to, but different from, your next point.

Clearly there is nutritional research which is the exception to what is being asserted here, like research on plant-based diets...where the composition of the diet is controlled like Pritikin, Mcdougall, Barnard, Ornish, Essy and so on. These aren't just some vague recall surveys trying to match a given health outcome with a given food, but looking at a very specific diet subjects are given and at specific diseases to gauge the impact of that diet. Those are some of the few studies that clearly are not pseudo science. So I would say the issue is that much of nutritional science is a wank because research showing how to use a clinical diet to reverse and prevent specific diease is largely ignored and not reported to the public.

The problem with the PCRM's (Physician's Comittee for Responsible Medicine) doctors is that their conclusions are not scientific. It is as if they have ignored the scientific method and honesty to encourage others to adopt their moral beliefs (veganism) while masking it with flawed data that they say shows that animal products correlate with disease (as apposed to other potential variables).

The problem with this conclusion is that no one in the PCRM has ever produced research that difinitively shows that animal products are a disease trigger; they fail the scientific method to come to absolutist conclusions.

Dr.'s Esselstyn, and Campbell are perhaps the most well known for their research (Dr. Essesstyn's intervention trials and Dr. Campbell's cohort "The China Study" and his biochemical isolation study of casein and gluten). Dr. Esselstyn takes patients who are on a Standard American Diet and then puts them on to a very low fat vegan diet which eschews extracted oils, animal products, sugar, and refined and processed foods as well as encourages healthy lifestyle activities. This is a range of variables and it could very well be any one of these that is associated with the positive health effects that are reported. The problem, however, is that Dr. Esselstyn (along with his colleagues whom have done similar intervention trials) does not account for these variables in a follow-up RCT; his bottom line is just that: animal products are bad.

Dr. Esselstyn has never (at least to my knowledge) conducted an RCT which addresses the variables in his diet. It is really the meat? Is it the milk? Is it eating more plants? Is it the extracted oils (and if so, is it the PUFA, SFA, or MUFA)? Is it the refined and processed foods? Is it the sedentary lifestyle? I do not doubt that his patients have improved, but the primary reason for these imrpovements has never been proven by Dr. Esselstyn or any of his colleagues. Despite this, they spread FUD; and coincidentally this FUD is in line with their moral beliefs.

Dr. Campbell's is also guilty of spreading misinformation based on incomplete data. Ignoring for a moment that correlation does not prove causation in epidemiology (something I'd expect such an accomplished researcher as Dr. Campbell to know, understand, and uphold), and likewise ignoreing that it is not scientific to apply the results of a study done on powdered casein to everything containing casein (coincidentally, animal products) data from his studies do not fit his conclusions.

This is the very definition of pseudoscience! Are people improving on this low-fat, no oil, no salt, no sugar, no refined foods, no processed foods, active lifestyle diet (with some getting stress management and fish oil supplimentation)? Certainly. But I could easily design a similar diet that is omnivorous, vegetarian, low carb, etc. and have the same results and conclude that it's because food X was removed (when it was really foods X,Y, Z, A, B, C). It would still be pseudoscience.

Avatar of: BBaars

BBaars

Posts: 1

November 12, 2013

I've seen enormous errors being made both ways: Excessive skepticism and excessive commitment to popular hypotheses. I am always mindful of that when I read critiques like this. 

I do regret that the author has taken to a popular science medium to voice his critique. It has not gone through normal peer review. Frequently that is a sign that a certain passionately held opinion would not pass peer review. Alternatively, it is ALSO possible that the author is right, and that the nutrition establishment is driven by grants and in-groups. 

You only find out by rigorously examining the evidence. There is no other way. A non-technical medium is not the best place to do that. 

In my perception of good science, popularization always leads to the danger of massive corruption. I have seen it multiple times. 

Yet it is also true that stuck establishments are less than optimal. 

The important thing is to support an open and rigorous DIALOGUE on these matters. 

Epidemiological studies have well-known risks, because correlation is not causation, and statistical solutions like regression methods are not guaranteed to take into account all the covariates of interest. In recent years time series analysis and causal analysis have improved the tool kit for analyzing real bodies of data in the world. However, that gives us better methods but not specific solutions for specific datasets. 

A mix of experimental and epidemiological studies is always better. I personally favor a much wider inclusion of animal and human experimental datasets along with epidemiological studies. With a new age of genomic data on the basic biology of nutritional interventions, animal studies down to C Elegans have taken on a direct relevance that was previously unimaginable. For example, caloric restriction (CR) has been shown to extend life in a range of species, but not (yet) in humans. We are therefore blessed with a much more set of statistical AND biological tools for nutritional research. Sometimes this allows us to take animal work and make plausible inferences about humans. The speed of experimental work therefore accelerates. 

It took decades to find solid proof that cigarette smoking caused cancer, because longitudinal and naturalistic studies are inherently difficult and expensive. We can study such causal questions much more easily today. But it is by no means a done deal. It is still difficult. 

For all these reasons I would be a little cautious about torpedoing the nutritional science establishment. Let the debate occur in the journals, making sure that alternative voices are clearly heard. 

We have made Type I and Type II errors, in some case with disastrous results. We should listen to the critics. Then we should listen to the answers from those who are criticized. A one-sided critique fails on that score. Both sides need to be heard. 

Bernard Baars

Avatar of: JoeThePimpernel

JoeThePimpernel

Posts: 1

November 12, 2013

Same thing applies to AGW/climate change research.

If you want to get grants, your research better not question the politically correct dogma.

Avatar of: George Henderson

George Henderson

Posts: 1

November 13, 2013

This study has always seemed to me to be trying hard to avoid the errors mentioned above. It is a less fallible way of doing what is much the same kind of research. It uses a real disease diagnosis as an endpoint, not just a "risk factor", and measures the dietary habit in question using a serum marker.
From 2010:
http://www.ncbi.nlm.nih.gov/pubmed/21173413

Also, from 2013:
http://www.ncbi.nlm.nih.gov/pubmed/23407305

Follow The Scientist

icon-facebook icon-linkedin icon-twitter icon-vimeo icon-youtube
Advertisement

Stay Connected with The Scientist

  • icon-facebook The Scientist Magazine
  • icon-facebook The Scientist Careers
  • icon-facebook Neuroscience Research Techniques
  • icon-facebook Genetic Research Techniques
  • icon-facebook Cell Culture Techniques
  • icon-facebook Microbiology and Immunology
  • icon-facebook Cancer Research and Technology
  • icon-facebook Stem Cell and Regenerative Science
Advertisement
Advertisement
Life Technologies