New JAMA Study Raises Research Questions

A study published in the December 5 issue of the Journal of the American Medical Association (JAMA) has raised important philosophical issues regarding nutrient research and demonstrated the confusion when new research contradicts earlier work.

“The scientific community-from scientists to journalists to consumers-wants conclusive answers, consequently we’re always looking at what the ‘study du jour’ tells us and trying to make it answer all questions,” said Andrew Shao, Ph.D., vice president, scientific and regulatory affairs, Council for Responsible Nutrition. “But the reality is that science doesn’t always move forward-there is some back and forth-and while research may seem to contradict itself, that should not be interpreted to mean one type of study trumps another, particularly when different studies ask and answer different questions.”

The JAMA review set out to evaluate the change over time in the quantity and content of citations for two observational studies. One concerned the major cardiovascular benefits associated with vitamin E supplementation. The other examined the protective effects of beta-carotene on cancer and estrogen on Alzheimer’s disease. Subsequent articles citing the studies were analyzed. The researchers gathered and reviewed 172 articles on vitamin E and cardiovascular health, 16 articles on beta-carotene and cancer prevention and 47 articles on estrogen for dementia prevention, rating them as favorable, equivocal and unfavorable. The sampled articles were published in 1997, 2001 and 2005 (before, early and late after publication of refuting RCTs) related to vitamin E and cardiovascular health and in 2006, which referenced highly cited articles proposing benefits associated with beta-carotene and cancer.

Pertaining to vitamin E the researchers found that, despite the large RCTs that received a great deal of media attention, more than half of the articles that cited the observational studies were favorable towards a beneficial effect of supplemental vitamin E. The study authors write, “Even among articles that cited the contradicting HOPE [RCT] trial, rather than the positive epidemiological studies, the majority in 2005 still could not conclude that vitamin E was ineffective.”

“This suggests that researchers interpret research differently, depending on their bias and expertise,” said Shao. “For pure scientific purposes, here’s a valid hypothesis to test: conduct a trial on secondary prevention in heart patients with a lifetime of bad habits that likely contributed to their heart disease to determine if a nutrient might provide some benefit. But it’s not valid to conclude from the results of that study that the nutrient doesn’t work. We can’t expect a simple vitamin supplement to reverse heart disease. So if that doesn’t happen, we must interpret the results appropriately by placing the study in the proper context and acknowledge that the results don’t answer the question of whether supplemental amounts of vitamin E in a healthy population could have prevented heart disease had it been used consistently over time in combination with other antioxidants.”

The findings from the JAMA literature review related to beta-carotene and cancer and estrogen and dementia were similar, with the researchers finding that more than 62% of the articles in each of the subsets were favorable.

“The RCTs with negative results attempted to answer the question, ‘can a supplemental nutrient treat or reverse a disease or a lifetime of unhealthy habits in patients who are also taking prescription medications?’” said Shao. “The observational studies with positive results attempted to answer the question, ‘if we start with a mostly healthy population generally free of disease, can we identify various diet/nutrient and lifestyle factors that make them more or less prone to disease?’ These are very different questions, making the studies incompatible for direct comparison and demonstrating that one type of study can’t necessarily be used to refute the other. We firmly believe that RCTs should not be thought of as the only rigorous research approach. As the study authors point out, ‘when randomized and observational studies disagree, it is incorrect to assume that nonrandomized studies are always wrong.’ Rather, we should put studies into the appropriate context and evaluate the total body of evidence, which includes RCTs and observational studies, and other types of research. We hope that the publication of this analysis will encourage all of those within the scientific community to become more involved in this discussion.”