© STUDIOM1/ISTOCKPHOTO.COMThe media commonly reports on what appear to be shocking contradictions and reversals in studies of diet and health. One day oat bran works for lowering cholesterol, the next day it doesn’t. One day the perils of butter drive up sales of margarines, until the risks of trans fats in margarine swing sales back in the other direction. For decades “low-fat” was the public-health mantra for weight loss, until a series of studies seemed to indicate “low-carb” was at least as effective. In truth, the findings from these studies are rarely as diametrically opposed to one another as the media portrays in a bid to capture the attention of their audience.
The recipe for creating controversy is built into the complexities linking food and nutrition to health and disease. The main ingredients include the target population, the type of diet or food or nutrient of interest, the...
Ultimately and ideally, the general question about the healthfulness of oat bran would take into account all possible combinations of target population, type of diet/food/nutrient, dose, duration, and outcome. But no single study could answer all those questions simultaneously. In 1990, two research studies, published nine months apart, reported apparently contradictory conclusions (N Engl J Med, 322:147-52, 1990; Am J Clin Nutr, 52:495-99, 1990). The later study reported that oat bran lowered blood cholesterol; the earlier one reported that it did not. The study reporting a benefit was conducted in an older population with elevated blood cholesterol and room for improvement. The study reporting no benefit was conducted in a younger, healthier population with normal blood cholesterol levels and limited room for improvement.
Contradictory? No. The two target populations were different. When combined, do the two studies address all the possible combinations mentioned above? No. Do the studies address health? Yes, but only a small slice of the oat-bran-effects-on-health pie—far from the whole enchilada. Controversial? Sure, if you want them to be, because they reach opposite conclusions about whether eating oat bran lowers blood cholesterol. But truly controversial? No, these just represent two different pieces that both fit into a larger puzzle.
The findings from nutrition studies are rarely as diametrically opposed to one another as the media portrays. The recipe for creating controversy is built into the complexities linking food and nutrition to health and disease.
Ideal study designs: the problems
For most nutrition topics, there are several types of study designs that can be used to generate useful evidence for presence or absence of benefit. Each type of study design has multiple components with options that range from more to less ideal. Comprehensive coverage of this area is beyond the scope of this discussion, but several of the more important sources of variability are described below.
Type of study. Observational studies involve simply observing the participants, their dietary habits, and their health status. When dietary components are linked to health outcomes in observational studies, the conclusions are always framed as “associations,” as opposed to “cause and effect.” Intervention studies are used to establish cause and effect. Knowing about cause and effect is more ideal than knowing about associations, but observational study is often the only realistic option.
Participant adherence. A central component of intervention studies is that participants must change their diet in order to determine if this causes a subsequent change in a health outcome. Ideally, a participant would keep everything else in his life constant and change just one nutrition component of interest. When the component is a single nutrient, like a vitamin or mineral, this can be achieved fairly easily using a dietary supplement and a placebo. However, if the dietary component of interest is food (e.g., garlic, soy, cruciferous vegetables, nuts), or more complicated yet, a dietary pattern (e.g., Paleo, vegetarian, Mediterranean), adherence becomes critical. A common problem in this area of research is low or only partial adherence to the particular food or diet pattern being tested.
Assessment of adherence. The typical approach to assessing adherence has multiple sources of error. Study participants, when asked to report what they ate during a particular time period, often provide answers that include some combination of overestimation, underestimation, forgetfulness, and lies. Another common source of error arises from the frequent occurrence that the collected nutrition data, while generally accurate for a particular day, represents an atypical food day for the participant. A third source of error occurs when the reported dietary intake of the study participants is converted to an estimation of grams, milligrams, and micrograms of nutrients by a database based on the same type of food reported by the participant, but not the same actual food.
Study outcome. The choice of the outcome being measured—blood cholesterol, dental caries, life or death, for instance—is critical to the conduct of an ideal study. Some health outcomes can be measured more easily and accurately than others. Whichever study outcome is selected, there will be a corresponding ideal number of enrollees and an ideal amount of time to track them. Unfortunately, many studies are conducted with outcomes that are difficult to measure at all or to measure accurately, with sample sizes that are too small, and durations that are too short. These issues often arise due to funding limitations.
“Ideal” studies are rare, since pursuing the ideal leads to choosing among competing priorities. When we prioritize generalizability, we lose rigor and control. When we prioritize the most accurate measurement, we lose affordability. When we seek ideal participants, we often lose generalizable individuals. Being aware of these trade-offs, we can seek to design the most ideal study possible.
Technology and a more ideal study
While the most important and complex nutrition questions will likely remain dauntingly problematic to resolve for many years to come, technological advances could address some of the challenges of designing better studies, particularly in the areas of diet adherence, assessment of adherence, and cost effectiveness.
Recruiting participants who understand and adhere to the study guidelines may be the greatest challenge of all. Typically, study staff need to explain the dietary guidelines at the outset, and then continue to field follow-up questions or provide further clarification. Much of this is repetitive, since many participants have similar questions or have similar problems with adherence. Internet-based technology can help in several ways that can decrease both the workload and the number of staff required to run a study: Make the answers to frequently asked questions available quickly and easily through a Web-based platform accessible by computer or smartphone; make study staff more accessible through e-mail or Web-based appointments and interactions; and use technology as a part of the intervention. An example might be exploring how diet reminders could be initiated through smartphone programming.
Assessment of diet adherence is another challenge that may be tackled with technology. Using a smartphone to take photos of eating episodes could improve the accuracy of reported food consumption by having study participants review their photos while reporting on their dietary intake to study staff. There have even been reports of attempts to use photographs of food to automatically estimate caloric and nutrient content of the items in the photo, using algorithms for calculations.
The latest technology can also benefit nutrition research by making such studies more cost-effective. A limiting factor in nutrition studies is often sample size. Recruiting, enrolling, intervening, and following up with large numbers of study participants typically require large staff sizes, which can be prohibitively expensive. Online screening tools have already helped investigators decrease the staff time necessary to recruit and enroll participants (e.g., Survey Monkey, Qualtrics). Web portals allow participants to schedule themselves for various study requirements (e.g., clinic visits, labs) and to fill out questionnaires online, rather than using paper copies that require subsequent data entry. Funds from these staff savings can be redirected to recruiting larger sample sizes, which then have the capacity to generate results that are more conclusive.
There are likely many other ways that technological advances could help to improve the design and process of nutrition studies, and it is clear that such advances have already had an impact. These approaches will likely continue to improve dietary adherence and accuracy in the assessment of adherence by study participants. These advances, combined with larger sample sizes made possible by greater cost-effectiveness, will allow for more-definitive and less controversial findings. These are confusing but also exciting times in the field of nutrition research.g
Christopher Gardner is Director of Nutrition Studies at the Stanford Prevention Research Center and a professor of medicine at Stanford University. He is in the process of developing a new interdisciplinary Stanford Food Systems Initiative. Michael Stanton is a postdoctoral fellow at the War Related Illness and Injury Study Center (WRIISC), VA Palo Alto Health Care System, and at Stanford University School of Medicine. A clinical psychologist specializing in behavioral medicine, he studies the development of compulsive eating and complementary medicine–based interventions for obesity and diabetes.
Interested in reading more?
Become a Member of
