Research: Why Nutrient Randomized Controlled Trials Often Appear Inconclusive

Too often, scientific bodies (EFSA, here’s looking at you) reject health claims, calling the scientific evidence “inconclusive.” But is the evidence truly inconclusive, or is another problem the cause, such as the wrong type of study?

Regulatory bodies and policy makers exclusively prize and prioritize randomized controlled trials (RCTs) as the “gold standard” of scientific evidence. But although RCTs are common for drug and disease claims, RCTs are far more difficult to properly conduct for food nutrients in relatively healthy populations. In fact, precisely because of this difficulty, too often collective RCT nutrient evidence will appear inconclusive, said Professor Robert Heaney, MD, a professor of endocrinology at Creighton University, and a guest speaker at the Council for Responsible Nutrition’s The Workshop and The Conference held in October. (This year’s sessions were an in-depth discussion of evidence-based nutrition and how various study models-observational studies, animal studies, and the like-fit the nutrient model.)

“We will never find a complete body of data in which all the results were positive,” Heaney continued. “We might if we had a pharmaceutical agent in the treatment…but we’re never going to find that for nutrients for the kinds of endpoints we’re talking about. So we should get used to the fact of inconsistency.” Yet, he said, EFSA and other regulatory bodies fail to take this inconsistency into account.

But why are RCTs so problematic for nutrient studies, as it is so often said? To be clear, Heaney did not say that RCTs are bad studies-quite the opposite, of course. RCTs, when done correctly, are indeed the only surefire way to confirm a strong causal relationship between a nutrient and efficacy.

But the RCT model fails nutrient studies often, Heaney said. First, there can be methodological flaws (technical problems when the study is conducted). But, even more importantly, Heaney said, there are biological flaws, such as not pinpointing the right dose needed to produce a measurable effect (nutrient effects are more subtle than drugs’, especially in healthy populations); failing to account for genetic heterogeneity within a study’s sample group; and failing to identify and account for nutrient interactions or the level of nutrients already present in subjects at baseline. “RCTs are indeed powerful, but these problems weaken or destroy that power,” he said.

Heaney pointed out that these problems “are much more common than we recognize or would like to recognize perhaps.” But those evaluating studies, such as regulators who tend to take an RCT’s word for fact, fail to realize that many RCTs do have these problems-and thus, those looking for surefire answers may not find conclusive evidence in many nutrient RCTs. Even large, multimillion-dollar, government-funded RCTs fall victim to flaws. Heaney pointed to the NIH-funded Women’s Health Initiative study, which found no conclusive impact from supplemental calcium because subjects already had high baseline calcium levels; or the Trial of Calcium for Preeclampsia Prevention, which failed to supplement enough vitamin D to show a difference. For both studies, results were deemed null, and it was because the RCTs had biological flaws.

“But null is not negative,” Heaney also reminded. Just because a study didn’t come to a definitive conclusion doesn’t mean it doesn’t provide information. Instead of writing off a nutrient completely due to a negative RCT, we should give greater credence to the collective body of scientific evidence for a nutrient, including a possible range of results as well as other study types besides RCTs-and build from there.

If, however, regulators continue to demand only RCTs, “It means that we’re only going to have stagnation; we won’t have progress, and we won’t get evidence. That means we should think about some possible options,” Heaney said.

We can still build a body of evidence for nutrients, as long as we move away from RCT bias and can accept less evidential certainty than would be appropriate for drugs, he said. After all, compared to drugs, nutrients tend to have a lower risk-to-benefit ratio. “Randomized trials usually cannot-and, I stress, cannot-provide the evidence needed to ground nutrient health claims or nutrient intake recommendations,” Heaney said. “As a result, progress cannot be made until the policy establishment accepts this fact and agrees upon alternatives to RCTs.”

Now, how we get regulators there is another matter. 

Jennifer Grebow
Editor-in-Chief