Approaches to clinical research for substantiating brain health products

Photo © AdobeStock.com/Atlas

Cognitive health is an important but controversial category for the dietary supplement industry. While consumers respond to marketing that offers them support for outcomes such as memory, focus, and mood, manufacturers need to be able to back up cognitive health claims to avoid regulatory scrutiny. One recent example is Prevagen, a well-known brand of cognitive health supplements, whose maker, Quincy Bioscience (Middleton, WI), had been sued by the Federal Trade Commission (FTC) and the New York Attorney General (NY AG) who alleged that the company made false claims about memory and brain health in their marketing.¹ The suit has stalled in the courts, but the company settled a separate class action lawsuit in 2020.²

While Prevagen did in fact have a study that the company said supported its claims, FTC and the NY AG did not feel the study sufficiently demonstrated the products ability to improve memory. At the end of the day, this is one of the major challenges of doing research to substantiate product claims; how much is enough, and what can I say about my product based on the study’s results? Industry stakeholders and regulators may disagree to what extent a study’s results can be interpreted to support a product’s claims.

According to FTC’s health products compliance guide, FTC wants “competent and reliable scientific evidence” that include “tests, analyses, research, or studies that (1) have been conducted and evaluated in an objective manner by experts in the relevant disease, condition, or function to which the representation relates; and (2) are generally accepted in the profession to yield accurate and reliable results.”³ 

The gold standard for substantiation is, of course, the randomized, placebo-controlled trial (RCT), but this can be quite expensive and time consuming to complete. That should not discourage companies from pursuing any research, however, because having any knowledge about how your products impact its users, or how they function, is valuable. At the end of the day, the value of research is not in the individual studies but the overall body of evidence.

From a business perspective, having some scientific validation behind an ingredient can be a significant advantage, particularly as the marketplace becomes more saturated with products making brain health-related claims. Bad actors in the marketplace, and negative headlines about products like Prevagen have made many people question the integrity of the dietary supplement industry, as well as the value of dietary supplements for promoting one’s health. Good research can offer peace of mind about a product’s safety, and offer a point of differentiation in the marketplace, says Joshua Baisley, HBSc, senior vice president, clinical operations at Nutrasource (Guelph, ON, Canada), a contract research organization (CRO).

“Ingredient manufacturers and brands typically invest in research to validate the efficacy and safety of their ingredients or products, differentiate themselves in the market, substantiate health claims for marketing purposes, and ultimately build consumer trust and credibility,” explains Baisley.

In the world of brain health, having research to back up claims is important. While dietary supplements are not pharmaceuticals, and should not be treated as such, there are consumers that believe in their value as a way to reduce the risk of age-related cognitive decline, or mitigate the effects of this decline. There are also many young people interested in products marketed for energy and mental focus either for gaming or as a daily pick-me-up. As a result, the consumers buying these products may be a vulnerable population susceptible to unscrupulous sellers that make big promises but poor quality products.

That said, conducting research is not without its risks. One cannot guarantee positive results, but if you’re working with a quality ingredient, with sound scientific rationale, and the right study design, you’re bound to learn something valuable. It’s also a long-term investment. You don’t just jump into an RCT, you work your way up.

“While results can’t be guaranteed, risk of failure can be mitigated through a phased approach such as conducting a pilot study to understand the products effect and potential effect size prior to conducting a pivotal trial where the goal is statistical significance and clinical relevance,” explains Baisley. “In the end it comes down to a risk benefit ratio. There is a risk that a product does not show a greater effect than placebo, but perhaps there is an improvement in quality of life or other unexpected beneficial result. Even with a null result in terms of efficacy, all studies are helpful in building the safety profile of the product.”

The return on investment can also be substantial. “For branded ingredient companies, having two studies per ingredient can benefit their B2B sales to finished supplement companies,” says Palak Vaval, senior associate, international business and marketing for the CRO Vedic Lifesciences (Mumbai, India). Having a validated ingredient is attractive to finished product manufacturers because they want to be able to tout that ingredient and its benefits to consumers, while also having a competive advantage over other brands that can’t make the same claims.

So, where do you start? Most brands don’t have the resources to conduct research in-house. This is where CROs come in. They can help you plan, refine, and conduct research, but it helps to have a base of knowledge to start from.

“Brands should have a clear research objective, understand their target market, claim(s) they wish to make and study, regulatory requirements, have preliminary data or at least a rationale for the research, and be prepared with a reasonable budget and timeline,” says Vaval.

Other information or resources that are helpful for CROs as they start working with a company, says Baisley, are the mechanisms of action of a product or ingredient, whether the study is for marketing or regulatory purposes, expectations of effect size for the results, a gap analysis of unmet needs as well as competitor analysis, the pharmacokinetic profile of the product, as well as a product with consistent quality in adequate quantities made using an established manufacturing process.

Design Considerations

Designing a study for your product really depends on what the product is and its hypothesized impacts. Budget is definitely a factor, but a number of variables need to be considered to create the right study, as opposed to most affordable study.

“While budget considerations are important, consideration of the ingredient’s mechanism of action (if known), expected effect size, and duration of time until the product is expected to have an effect, are all warranted to provide the greatest chance for a successful trial,” says Baisley. “Variability in expected response between participants can also lead to increased sample size and as a result cost and duration. This can sometimes be mitigated using crossover trials to reduce interindividual variability; however, warrants consideration of the response being studied as well as washout period required to prevent carryover effects.”

In crossover studies, two groups of subjects take both the active product and placebo at separate times, with a preset washout period in between interventions. In this case, a crossover design helps solve one problem while adding a layer of complication. When it comes to study design, it’s easy to get into the weeds about different designs and their potential strengths and weaknesses. So, let’s start at the beginning: What is the desired outcome you want to study? Do you have an herb that is known in traditional medicine to offer stress relief? That would be tested very differently from a product that is believed to support memory or overall cognition. Within the cognitive health category, there are a ton of areas to cover, and they each have their challenges in terms of investigation.

“Cognitive support has got some diverse claims substantiation in times post-COVID. Depression, anxiety, stress, and mood are some of the common areas to start with. These can be great primary objectives that are simple, yet impactful to advertise for the companies,” explains Vaval. “One can opt for standardized questionnaires, like DASS-21 (Depression, Anxiety & Stress Scale – 21), POMS (Profile of Mood States), PSS (Perceived Stress Scale), STAI (State-Trait Anxiety Inventory), etc., for assessing the emotional states per se.”

These questionnaires are a good way to calculate subjective measures, but they are subjective and therefore not as reliable. More sophisticated tests can be used as well, such as measuring biomarkers that are associated with mood and stress physiologically. There are also more objective tests that are used to calculate executive functioning and different types of memory.

Digital cognitive assessments such as CANTAB, or ANT – Attention Network Test, “are highly precise and sensitive, objective measures of cognitive function,” explains Vaval. In terms of biomarkers, she points to salivary cortisol for assessing stress and gut microbiome analysis for assessing changes in the Gut-Brain Axis as valuable tools. Some tests are even more dialed in, such as specific ePRO questionnaires and biomarkers that are “laser-focused” in the event that someone wants to make specific claims, says Vaval.

“A mix of impactful subjective and objective measures and respective tools with safety profiles assessed is recommended for cognitive health studies,” advises Vaval.

Some tests are more difficult to administer than others, which can influence the number of personnel involved in the study, as well as how many people may or may not drop out of the study over time. “Digital assessments are trickier to conduct with the level of precision they offer, yet their data is more quantifiable. Their sensitivity is crucial to the study hence the process has to be carefully drafted in the protocol and trained to the site staff and volunteers,” explains Vaval. “On the other end, incorporating questionnaires is relatively easier, but the outcomes may be affected by other external factors, like too many ePROs added to one study, the environment and mood of the participants, and the varied levels of understanding of the questionnaire ratings.”

You also have to consider the population. Memory and executive function outcomes, for example, can vary widely depending on age group being studies. Therefore, you may want to deploy different tests for different age groups. There’s also the question of testing healthy or unhealthy populations.

“While dietary supplement studies often focus on ‘healthy’ populations to support general wellness claims, investigating at-risk populations, rather than diseased populations can also be considered,” says Baisley. “The study of diseased populations can be valuable for understanding specific therapeutic benefits if a manufacturer has interest in pursuing a pharmaceutical product, or to gain more aggressive claims in jurisdictions such as Canada. In such cases, this requires sound regulatory knowledge and strategy to ensure compliance, especially within the US where dietary supplements should not claim to treat, prevent or mitigate a disease.”

Sometimes, it’s more difficult to get an ideal outcome from a study in a healthy population than a diseased, but you have to use these results carefully, lest you be accused of making disease claims. The use of at-risk populations, as Baisley calls them, or what Vaval calls “borderline” populations may offer an ideal compromise. As Vaval explains, these subjects “are neither fully healthy, nor sick enough to go to a doctor.” This is a common practice in joint health studies in which subjects have joint problems but do not have a formal diagnosis of arthritis or other disease. In the case of brain health, at-risk subjects could be elderly people with complaints about their memory, or younger people dealing with chronic stress. 

Should I publish?

Suppose you did not get the statistically significant results you were hoping for. That does not mean the study was a waste of time, or has no value. “All findings, including negative or inconclusive results, contribute to the scientific understanding and should be considered for reporting,” says Baisley. “Transparency in research fosters trust and the data garnered from studies informs future studies and adds to a products safety profile.”

“The FTC Guidelines of December 2022 encourages reporting of outcomes that didn’t meet statistical significance,” adds Vaval. “We strongly believe that p values don’t necessarily have to meet a rigid less than 0.05 for the results to be usable. The totality of the other outcomes must be viewed and trends assessed especially if the sample (n) made available to the researchers was limited. Supplement researchers don’t always have the luxury of huge budgets that pharma enjoys. It is a problem only if the study fails to meet several of its key objectives, not just one or two.”

There are times when post-hoc or sub-group analysis are used to find patterns and determine between-group differences after the data has been completed. While it may offer useful insights for planning future studies, says Vaval, the results of these analyses should not be relied on to communicate a product’s potential benefits. The over-reliance on post-hoc analysis is, in fact, what got Prevagen in trouble with FTC.

While the quality of results certainly inform whether or not to publish results in a journal, there are other reasons, says Vaval. “The decision often weighs the benefits of public knowledge against competitive advantage and intellectual property rights,” she explains. It’s not unusual to see companies announce the results of research that has not been published or subject to peer-review. The study may help with sales, but the lack of peer-review may also be a detriment. That said, the peer review process can be time-consuming and tedious, taking anywhere from three to nine months, sometimes upwards of a year.

Baisley encourages the publication of all research. “We are not sure how journals react to manuscripts where the study failed to meet multiple endpoints, because we have had very few such opportunities!” says Baisley. “We believe that the scientific community may still benefit if negative results are boldly reported, while the totality of evidence from past studies is discussed and attempts are made to explain why only this particular study failed. No product is a panacea. A failed study or missed endpoints guide formulators, consumers, and practitioners about reasonable product limitations.”

The more people participating in sciences the better. Are you ready to do your part?

References

1. FTC, New York State Charge the Marketers of Prevagen With Making Deceptive Memory, Cognitive Improvement Claims. FTC. January 9, 2017. https://www.ftc.gov/news-events/news/press-releases/2017/01/ftc-new-york-state-charge-marketers-prevagen-making-deceptive-memory-cognitive-improvement-claims (accessed 2023-02-27).
2. Degman, W.J.; Barraza L.; Farland, L.V.; Klimentidis, Y.C.; Jacobs, E.T. Strengthening the Regulation of Dietary Supplements—Lessons from Prevagen®. FDLI. https://www.fdli.org/2021/11/strengthening-the-regulation-of-dietary-supplements-lessons-from-prevagen/#_edn28 (accessed 2023-02-27)
3. FTC. Health Products Compliance Guidance. December 2022. https://www.ftc.gov/business-guidance/resources/health-products-compliance-guidance#_edn28 (accessed 2023-02-27)