Registered dietitian and moderator of our Journal Club, Juliette Kellow, reveals her top tips for evaluating the credibility of scientific studies supported by food industry funding, and shares a case study from the Almond Board of California, highlighting how academia and the food industry can work together to expand nutrition knowledge.
Juliette Kellow is a freelance nutrition consultant and regularly works with sectors of the food industry to support media and PR activities. She has worked with the Almond Board of California on a consultancy basis since 2017 and has been financed by them to write this blog for MyNutriWeb.
Nutrition research has recently been put under the spotlight – and not just for its role in advancing our knowledge on the relationship between diet and health. Increasingly, there are concerns that some nutrition research funded by the food industry may result in a conflict of interests, potentially leading to bias and bringing the credibility, transparency and trustworthiness of the findings into question.
Frameworks are increasingly being developed to help researchers negotiate and interact with commercial partners to reduce bias. For example, the Food Research Risk (FoRK) guidance and toolkit from the Medical Research Council Epidemiology Unit at the University of Cambridge helps researchers evaluate potential interactions with the food industry. Meanwhile, the Institute for the Advancement of Food and Nutrition Sciences (IAFNS) Assembly on Scientific Integrity has updated its Guiding Principles for Funding Food Science and Nutrition Research to help reduce bias and promote integrity in research funded by industry. These types of guidance may play an important part in reducing conflicts of interest in future nutrition research, although it will be up to researchers to decide whether to use them.
As health professionals, it’s vital we keep up to date with nutrition research to ensure we give clients current, relevant and evidence-based advice. So, what should we look for when reading existing, newly published or future studies that may not have employed such frameworks? Here are my tips for evaluating the credibility of industry-funded research.
BEFORE YOU BEGIN
1. Understand that funding is needed
Funding is an essential part of the research process. Undertaking good-quality scientific studies is expensive, so without financial support, research projects wouldn’t happen. Research will nearly always have an external funder – or group of funders. The two main funding pathways are:
- Non-commercial organisations such as charities, national academies, research councils and government health or agriculture departments.
- The commercial food industry, which encompasses the whole food supply chain, from organisations that support farmers of specific food commodities, such as the Almond Board of California (ABC), to global businesses that manufacture, market and sell food.
2. Look for collaborations
Increasingly, research is bringing together academia, industry and policymakers. For example, in 2023 the Biotechnology and Biological Sciences Research Council (BBSRC) with support from the Department for Environment, Food and Rural Affairs (Defra), Innovate UK and the Medical Research Council (MRC) set up six new innovation hubs in the UK as part of a new Diet and Health Open Innovation Research Club (OIRC). Such collaborations could bridge the gap between researchers and commercial partners, ensuring all parties have the same objective, report results accurately and work together to improve public health. While there’s no guarantee this will avoid bias, there’s limited evidence to suggest studies with combined funding sources have a lower risk of bias in certain areas, so collaborations could be a step in the right direction.
3. Put funding bias into perspective
Funding bias does exist. In a study looking at published papers in 10 nutrition journals, more than half (55.6%) with industry involvement reported findings favourable to certain food industry interests, compared to less than one in 10 (9.7%) with no industry involvement. But bias doesn’t just result from food industry funding. It exists throughout the entire research process, from the initial concept to getting a paper published and communicating the findings to health agencies, peers, media and consumers. The Catalogue of Bias from the University of Oxford and Centre for Evidence-based Medicine identifies 65 types of bias that can occur throughout the research process, but estimates the complete list of potential entries could be around 250. Industry Sponsorship Bias – defined as ‘a tendency for the methods and results of a study to support the interests of the funding organisation’ – is just one of these. Other examples include:
- Confounding – A distortion that modifies an association between an exposure and an outcome because a factor is independently associated with the exposure and the outcome.
- Hawthorne effect – When individuals modify an aspect of their behaviour in response to their awareness of being observed.
- One-sided reference bias – When authors restrict their references to only those works that support their position.
- Positive results bias – The tendency to submit, accept and publish positive results rather than non-significant or negative results.
- Publication bias – When the likelihood of a study being published is affected by the findings of the study.
- Spin bias – The intentional or unintentional distorted interpretation of research results, unjustifiably suggesting favourable or unfavourable findings that can result in misleading conclusions.
When looking at research, it’s important to understand bias can occur throughout the research process and isn’t always or solely related to funding.
4. Recognise your own biases
Our own biases, based on personal experiences, attitudes, interests and beliefs, have the potential to affect our views on a scientific study. Known as reader bias, this can influence what nutrition research we choose to read and, if we have an opinion on the funding provider, may sway our views on a study before we’ve even read the abstract. Reader bias may be implicit or unconscious, where we’re unaware of our attitudes and beliefs, or explicit, meaning we are aware of these on a conscious level. There’s limited information about reader bias, especially when evaluating nutrition research, but there’s useful (non-nutrition related) information here.
5. Look at where the research is published
Is the publication in a peer-reviewed journal? The peer-review process means research is reviewed by experts who have expertise in the same field but no conflicts of interest with the researchers or funder. This quality control exercise provides an additional layer of review from independent experts to improve quality of papers and screen out poorly conducted studies. Recent years have seen an increase in online journals that don’t carry out peer reviews before publication – sometimes known as predatory publishing – so checking this is important before evaluating a paper.
6. Look for pre-registration status
Papers reporting results of clinical intervention studies should state whether the research was registered with an official body such as clinicaltrials.gov before the study began. Pre-registration sets out the research plan, including the hypothesis, primary and secondary outcome measures, the methodologies and analyses to be used. This helps to stop them being changed during the research process or at publication to benefit the funder or the researcher. Plus, if researchers have stated what they want to measure, but then don’t find any significant differences in primary outcomes, they must still report them.
EVALUATING THE RESEARCH
7. Consider whether the research is needed
Look at the rationale for undertaking the research and consider:
- Is the research justified?
- Is it important or necessary to progress knowledge in a specific area?
- Is it in the best interests of public health?
- Will it only benefit the organisation funding the research?
8. Identify the source of funding
Research papers should identify where funding has come from and any role the funder plays in the design of the study – regardless of whether it’s from industry, a charity, government or a research council – and report any conflicts of interest.
9. Check out the authors
Look at who the authors are, where they work, and whether they report any conflicts of interest. This will identify whether researchers from a funding body have been involved in the development of the study. Increasingly, research papers outline the roles of individual authors at various stages of the study, for example, ‘author x extracted the data’, ‘author y analysed the data’, ‘all authors contributed to the writing of the article and the study design’.
10. Consider the study design
Ask yourself:
- Does the study design make sense and address the research question?
- Is it fit for purpose and robust?
- For interventional studies, does the control group provide an appropriate comparison to the intervention?
- Have potential confounders been addressed?
- Did many people drop out of the study and, if so, why?
- Was the analysis done blindly, so interpretation of the results wasn’t swayed?
11. Review the results, limitations and conclusion
When it comes to looking at research findings:
- Check results are accurately reported and supported by the data.
- Make sure primary outcomes aren’t overlooked in favour of secondary outcomes that may have greater benefits for funders.
- Consider whether the findings are clinically relevant – research papers may reveal a significant result but consider whether this would make a difference in clinical practice.
- Consider whether the results are what would be expected based on current and existing literature in the area. Research can be ground-breaking, so don’t automatically dismiss findings that don’t sit neatly within the current literature. But do question whether they could be caused by bias if they deviate from current thinking, especially in a direction that benefits funders.
- Reflect on whether findings will have a positive or negative impact on public health.
- Identify what you think the limitations could be and see if the paper communicates them properly.
- Check the conclusion is supported by the main study findings.
12. Consider how research findings are publicised
If you have access to press releases, look at how research is reported by the academic institute or funding bodies. Does this reflect what was actually found in the research? A recent paper looking at the science communication of nutrition and dietetic research found funders, journals and researchers’ institutions often exaggerate when reporting on study findings, for example, inferring health benefits in humans from a lab study.
13. Making your mind up
Deciding whether industry funding has influenced nutrition research is often down to individual interpretation. There are positive steps being taken to reduce bias, thanks to greater scrutiny and awareness of the issue, and because affiliations with industry and conflicts of interest are required to be disclosed in most situations, aiding transparency. Together with having a greater understanding of what to look for when evaluating nutrition research, all these factors can help us decide on the credibility of industry-funded studies.
Case Study: The Industry Perspective
The Almond Board of California (ABC) has 30+ years’ experience of funding research. To date there are more than 200 published studies that support the benefits of eating almonds for heart health, weight management, satiety, type 2 diabetes, gut health, skin health, physical activity and more.
We talked to Elena Hemler, Associate Director, Nutrition Research for the Almond Board of California, to learn about ABC’s approach to funding nutrition research.
Why does the Almond Board of California fund research?
Research is a core part of ABC’s work. As a non-profit Federal Marketing Order overseen by the US Department of Agriculture (USDA) and funded by almond farmers and processors, ABC promotes California almonds by funding studies about almonds’ health benefits, as well as ensuring sustainable agricultural practices and food safety. The Almond Board of California’s research has helped advance understanding of almonds’ health benefits related to heart health, diabetes, weight management and more.
Industry-funded studies have been pivotal in contributing to current knowledge about healthy dietary patterns. From 1980-2000, the U.S. Dietary Guidelines recommended that people “avoid too much fat,” or “choose a diet low in fat.” It was research, some funded by ABC, which led to the current understanding that different types of fats have different health effects. These findings have had dramatic impacts on the changing guidance on fat – which now acknowledges the crucial role of unsaturated fats (like the monounsaturated fats abundant in almonds) in improving health and encourages replacement of saturated fat with unsaturated fat.
How does the Almond Board of California pay for research?
Almond farmers and processors fund the Almond Board of California each year based on the quantity of almonds grown, and part of this budget is used to support nutrition research. Research projects may be exclusively funded by the Almond Board of California or partly funded by other organisations, such as government bodies like the U.S Department of Agriculture.
What kind of research do you fund?
We fund research in many areas, including sustainable agricultural practices, soil health, pollination and bee health, food quality and safety, and, of course, nutrition science. We prioritise transparency and high quality research and work with top reputable researchers at leading academic institutions. We take great pride in the calibre and strength of the nutrition research we fund, and always disclose when we have funded research.
Where do research ideas come from?
ABC periodically releases requests for proposals inviting researchers to submit their research ideas. We purposely keep our requests for proposals general as we don’t want to limit ideas, set the research agenda, or move researchers in a certain direction. Once received, we send research proposals for peer review by independent scientists who are experts in the proposal topic area to ensure the scientific methodology outlined is robust. Proposals are then reviewed by the Almond Board’s Nutrition Research Committee, made up of volunteers including almond growers and scientists, who determine which proposals to support.
ABC’s Nutrition Research Committee identifies broad areas we are keen to explore and where there is potential for almonds to benefit health. For example, in the last few years there’s been an interest in learning more about the role almonds may play in improving skin health, prediabetes, gut health, and exercise performance and recovery.
What is the role of the Almond Board of California during the research process?
Once a research study is funded, the researchers work independently on the project design and implementation. ABC is not involved in study design, data collection, analysis or results interpretation. Researchers are required to give progress reports at different stages of the project, such as if participant recruitment is on schedule, but are not asked about results. ABC requires researchers to submit their findings to a peer-reviewed journal, regardless of the results of the studies.
How does the Almond Board of California communicate its research findings?
Everything ABC says publicly about research results is reviewed by USDA to ensure study findings are put in the correct context and not distorted, oversimplified, or generalised beyond the population that is being investigated. When sharing the findings, we always disclose the funding source and focus on how this research will benefit the health of consumers and how they can action it. As well as writing media releases for consumer and health press, we communicate new research to the scientific and healthcare professional community through our Nutrition Bulletin newsletter, website, online roundtables and at scientific conferences..
In all communications, we provide detail on the study design, subjects, full results, conclusions, and limitations. Plus, we stick to reporting on what the study found without making assumptions or exaggerating the results.
What research has the Almond Board of California recently funded?
Recent research published in the American Journal of Clinical Nutrition, led by Professor Kevin Whelan from King’s College London, set out to determine the impact of whole almonds and ground almonds on the composition of gut microbiota, gut microbiota diversity and gut transit time. In the study, 87 healthy adults consumed either 56g whole almonds, 56g ground almonds or an energy-matched muffin for four weeks. While there were no significant differences in transit time or the composition or diversity of gut microbiota, the researchers found almond consumption significantly increased stool output (a sign of a well-functioning gastrointestinal system). When pooling results for whole and ground almonds, researchers saw increased levels of butyrate, a beneficial short chain fatty acid that’s been linked to fighting inflammation and lowering the risk of colon cancer. These findings suggest almonds could have a role to play in supporting gut health and provide a foundation for future studies.
Another study published in Clinical Nutrition ESPEN, led by Dr. Anoop Misra, Professor and Chairman of Fortis-C-DOC Centre of Excellence for Diabetes, Metabolic Diseases, and Endocrinology in New Delhi, found that eating almonds before breakfast, lunch and dinner was able to reverse prediabetes among one quarter of research participants. In the study, 30 Asian Indian adults with prediabetes and overweight or obesity, consumed 20g almonds before breakfast, lunch and dinner each day for three months. Compared with a control group, almond eaters had statistically significant reductions in body weight, waist circumference, fasting blood glucose, insulin resistance, HbA1c and total cholesterol. Furthermore, almost one in four participants in the intervention group (23.3%) saw raised blood glucose return to a normal level, suggesting that eating almonds before meals has potential for reversing prediabetes.
Disclaimer: This blog has been written in collaboration with the Almond Board of California nutrition team. Approval of each sponsor and activity is carefully assessed for suitability on a case by case basis. Sponsorship does not imply any endorsement of the brand by MyNutriWeb, its organisers, its moderators or any participating healthcare professional, or their association. Sponsorship funds are reinvested into the creation and promotion of professional development opportunities on MyNutriWeb.
