This article has been published by the peer-reviewed Journal of Applied Nutrition and cannot be reprinted without permission. I share the copyright and can publish it here.
LAND OF CONFUSION:
How Poor Science and Misleading Media Coverage Create Public Confusion About How Dietary Supplements Affect Health
Levin N. Land of Confusion: How Poor Science and Misleading Media Coverage Create Public Confusion About How Dietary Supplements Affect Health. J App Nutr, Vol 55, No. 1, 2005 8-15
© 2006 Neil E. Levin and the Journal of Applied Nutrition
By Neil E. Levin, CCN, DANLA
Certified Clinical Nutritionist, Diplomate in Advanced Nutritional Laboratory Assessment
Poor scientific work done by physicians and scientists - plus a lack of proper filters in journalism’s coverage of science - is contributing to misleading and contradictory dietary supplement health information seen by consumers, leading to them making poor health choices. This is a review of some of the problems that have occurred, which will continue until they are exposed and challenged.
POOR SCIENCE
A. WRONG DOSE: The clinical study dose needs to be somehow related to that used in previous studies and to common doses taken by consumers. An appropriate dose for a mild condition will probably not work for a more severe condition, which should not be presented as a failure of the supplement. 19-21
B. WRONG DURATION: A long term, chronic problem will probably not be corrected by a short term use of a dietary supplement. Longer term studies may be necessary to correctly determine efficacy. 15, 27, 37
C. WRONG MATERIAL: The species and preparation of botanicals need to be the same as those available on the market and used for nutritional support of specific health issues. 24
D. WRONG PATIENT POPULATION: Age and illness related effects need to be taken into account. For example, incidences of myocardial infarction and coronary heart disease associated with intake of arginine by seniors is not relevant for athletes that use arginine for strength enhancement. And there is evidence that people with certain conditions, like cardiovascular disease, tend to take more supplements than the general population, but it is unscientific to assume that the supplements are therefore responsible for these advanced disease conditions in the absence of any real evidence . 4, 6, 10, 28, 36-37
E. AMNESIA: The most recent study does not invalidate all previous studies. The FDA agrees, asserting that the totality of the evidence must be taken into account when interpreting studies. 19
F. GARBAGE IN: Meta-analyses mathematically integrate a number of clinical studies. However, when the analyses fail to account for differences between the studies, and are conducted by statisticians that don't understand these differences, garbage emerges. 4, 5, 10
G. BREEDING IGNORANCE: Poorly conducted studies on nutrition are often done by physicians or statisticians who have inadequate training in nutrition. 4, 10, 28 Dietary supplements may have either "actives" e.g. glucosamine, or provide nutrition, e.g. vitamins. Clinical studies for these two different classes of supplements need to be different by virtue of mode of action, end points, etc. But many researchers don't know the difference, much less how to distinguish and set up these distinct types of studies.
H. DATA BIAS: Author or journal bias affects interpretation of data. For example, small benefits from vitamins have been dismissed as probably occurring by chance, while equally slim risks are noted, highlighted and magnified in importance. 4, 8, 10, 22, 24, 28
I. ADVERSE EVENT REPORTS AREN’T EVIDENCE: Adverse Event Reports (AERs) are preliminary, unscreened possible associations between a substance and a side effect. They are intended as an early warning system so toxicologists, pathologists and other experts can try to isolate a dose-related cause and effect. AERs are NOT appropriately screened to justify using them as actual evidence of a substance having caused someone harm, particularly when most AERs involving dietary supplements are actually associated with combinations of pre-existing combinations, drugs and dietary supplements. Yet a bias that we should use these unsubstantiated AERs as primary evidence to ban certain dietary supplements is quite common, as was argued in banning the herb Ephedra. There is a false assumption by many government and medical authorities that most dietary supplement AERs are true, whereas drugs are actually protected against being more tightly regulated solely on the basis of their own, far more numerous and far more deadly AERs that are known to kill over a hundred thousand people every year. 11-14, 18, 22, 32, 33
J. TALK TO THE HAND: Journal editors often fail to publish criticisms of peer reviewed work. (No references, but isn’t that the point?)
K. DATA CREEP: Abstracts or conclusions that do not closely reflect the study’s data. 4, 8, 10, 21, 24, 25, 28
L. CONFLICTS OF INTEREST: Failure to properly screen studies or panels of peer-reviewers to eliminate conflicts of interest. 8
M. VITAMINS AREN’T DRUGS: Failure to anticipate synergies that nutrients require to be safe or effective. 1-3
N. VARIABLE CREEP: Failure to properly isolate and measure variables beyond the supplements that could affect study outcome 1-3, 21, 28
O. TIME TRAVEL: Reliance on preliminary study methods (test tube, epidemiological) when animal and human science has already ruled out certain mechanisms of action that may be expressed in vitro. 26
P. QUANTUM LEAP: Assuming that injections are equivalent to oral administration of supplements without evidence that this is likely or despite evidence that it is unlikely. 25
Q. POOR SPORTS: Authors do not correct their own published work even in the face of overwhelming criticism or subsequent science that reveals problems with their work. 1-3, 10, 28
R. TUNNEL VISION: Nutrient intake is often measured but not blood levels, which should also be noted for the presence of synergists or agonists that might affect the results. 1-3, 10
S. POISON PEN: Assumption that dietary supplements, including essential vitamins, are inherently toxic and that people need to be protected from them. This may betray an institutional bias by toxicologists, against the large body of safety evidence. 7, 18, 21
T. WRONG END OF THE MICROSCOPE: Failure to note that the majority of Americans are deficient in more than one essential nutrient. 9
POOR JOURNALISM
A. HYPE: Misleading or sensationalistic headlines and reports that position a study’s outcomes as being more meaningful than they are or that otherwise inaccurately depict a study’s data and conclusions. (Vitamin E, Beta Carotene)
B. AMNESIA: The most recent study does not invalidate all previous studies. In fact, it might be a poorly designed “rogue study”. The FDA says that the totality of the evidence must be taken into account when interpreting studies. (Vitamin E)
C. ARE WE HOME YET?: Headlining a study and then failing to publish critiques of that study. Also, ignoring subsequent studies that would discredit a previously covered study. (Vitamin E, Beta Carotene)
D. NOT MY DEPARTMENT: Failing to cover new developments that would significantly impact people’s view on a topic that was previously covered and continuing to refer to the original, incorrect story. (Vitamin E, Beta Carotene)
E. MISTAKEN IDENTITY: Reporting on failure of ingredients to act as expected because the studies were done on severely ill patients, whereas most dietary supplements are taken by people with mild to moderate symptoms. 1, 27 (Vitamin E, Beta Carotene, Glucosamine)
F. RED HERRING: Failure to note the relative safety of dietary supplements versus drugs – in fact, dietary supplements are even safer than many other categories of food products - while reporting on the supposed “dangers” of supplements. 11-14, 18, 21, 23 (Many supplements)
G. WRONG DIRECTION: Failure to note that most Americans are deficient in one or more essential nutrients while continuously warning against rare, typically non-fatal toxic effects of vitamins. 9 (Vitamins A, D, E)
H. PRETZEL LOGIC: Headlining a supplement “failure” while largely ignoring the similar failure of a drug that was also tested in the same clinical trial, when the drug is actually approved for treating that condition and the supplement is not even allowed to claim to treat any disease. Which one really failed: the supplement or the approved drug? 15-17 (St. John’s Wort vs. amitriptyline, Glucosamine vs. ELEVEN different drugs)
I. FACT CHECKING IS PASSE: Repeating inaccurate statements about dietary supplement dangers and regulation. (“Dietary supplements are unregulated.”) 18, 21
J. WEIRD SCIENCE: An appropriate dose for a mild condition will probably not work for a more severe condition, and a short term experiment may not quickly relieve a chronic condition, which should not be reported as a failure of the supplement. 19, 27 (Glucosamine, St. John’s wort)
K. IS ‘THE PRESS’ REALLY JUST A COPY MACHINE?: Assuming that all press releases by researchers or government offices are true or that it is responsible journalism to simply broadcast/publish them without added context or opposing views, even when reporting on controversial topics (includes all points listed above).
L. MAGNIFYING GLASSES BURN: An insignificant result on a study means that the study failed to produce statistically meaningful results, thus invalidating the data. Yet reporters mistakenly report on these studies and even magnify the insignificant “results” as headline news, serving only to mislead the public. In realty, the study design or execution probably failed, not the foods or supplements. (B-Vitamins and cancer risk (NORVIT trial), Low-fat diets don’t work, Vitamin E is dangerous, etc.) 10, 38, 39, 40
Example 1
Beta-carotene: Myth and Fact
Some years ago an antioxidant study in Finland was halted early because of a widely reported increase in cancer rates among male smokers taking beta-carotene. 1 Headlines associated this supplement with cancer risk. Despite objections that the study was flawed, beta-carotene use dropped.
A later analysis published in July 2004 took another look at that same Finnish smokers' study data, but now taking into account total antioxidant intake, which cleared away the scientific controversy. The smokers’ risk of getting lung cancer was inversely associated with total antioxidants in the diet, meaning that more total antioxidants resulted in fewer cancers. 2
A composite antioxidant index was generated for each of the 27,000 men over 14 years. The calculated amounts of carotenoids, flavonoids, Vitamin E, selenium and Vitamin C were compared to actual lung cancer rates, with a clear result: the combination of antioxidants lowered lung cancer risk in male smokers. Beta carotene was not the ‘bad guy’ that preliminary results suggested.
Another large study has noted that high carotenoid intake, confirmed by measures of blood levels, was associated with lower mortality rates among the elderly over a ten year period. 3
Still, news reports - and even a recent NIH scientific panel - continue to refer to beta-carotene as potentially harmful, having somehow missed the newer, better evidence. The “media myth” continues long after the science has moved on, with even scientists missing the line of evidence exonerating beta carotene.
Example 2
Vitamin E: Bogus Warnings
The warning about taking doses over 400 IU of Vitamin E is based on bad science from a flawed, heavily criticized meta-analysis that has been largely discredited. 4 A major review in the American Journal of Clinical Nutrition looked at the same data much more carefully, ultimately reporting that doses of up to 1,600 IU daily are safe. 5
The Food and Nutrition Board of the Institute of Medicine sets the upper tolerable intake of Vitamin E at 1,500 IU, showing a strong scientific consensus that lower doses, such as the popular 400 IU capsules, are not dangerous. 9
Vitamin E reduced cardiac events by 34% in the US Nurses' Health Study and 39% in the US Health Professionals' Follow-up Study, also reducing cardiac mortality by 47% in the Iowa Women's Health Study. 29 Among 90,000 nurses the incidence of heart disease was 30% to 40% lower among those with the highest intake of vitamin E from diet and supplements. Researchers found that the apparent benefit was mainly associated with intake of vitamin E from dietary supplements. 29, 31 The 10-year SENECA study reported blood levels of Vitamin E were not associated with all-cause or cause-specific mortality. 30
Reports of increased death rates occur mostly in meta-analyses that combine dissimilar studies imperfectly due to faulty statistical models or added variables. This type of review study needs to be critically reviewed before accepting the results, but is rarely definitive. 38
Example 3
Killer Ephedra?
Rand Corporation researchers reported that up to 155 reported deaths possibly linked to Ephedra were contradicted by its own review of published Ephedra studies that found no deaths, strokes or any serious side effects reported from using the herb. The Rand report also stated that ephedra, with or without caffeine, provided a statistically significant increase in short-term weight loss compared to placebo: about 2 pounds per month for up to 6 months. 32
The final FDA report that banned Ephedra in 2004 lowered the number of deaths possibly linked to Ephedra, stating that a total ban on the herb would possibly prevent only an estimated 0.7 to 1.2 deaths per year out of an estimated 3 billion doses taken by 20 million consumers. In banning Ephedra, the FDA said: “There is no requirement that there be evidence proving that the product has caused actual harm to specific individuals, only that scientific evidence supports the existence of risk.” 33 The agency also admitted that the U.S. General Accounting Office (GAO) had previously “criticized FDA's reliance on adverse event reports (AERs) as the basis for the proposed restrictions on dosage, frequency and duration of use”.
This shows that the FDA banned an herb with no proven deaths attributed to it. Only unproven Adverse Event Reports (AERs), which the FDA screeners virtually eliminated upon closer examination, were used to show a theoretical risk from ephedra. In fact, the FDA justified its ban only by making several assumptions in its cost-benefit analysis: claiming that there were no health benefits from using ephedra (ignoring decades of science, including the Rand report), that the vast majority of AERs were valid (without any specific evidence given, also contrary to the historic record) and that there were no increased risks to users who would be forced to switch from ephedra to pharmaceutical drugs for weight loss (no documentation was given to justify this assumption). 33
The FDA report contrasts greatly with the Rand report and with several other published studies. Researchers from the Stroke Program at the University of Texas’ Department of Neurology reported that, “Ephedra is not associated with increased risk for hemorrhagic stroke, except possibly at higher doses.” 34
A toxicology journal review concluded that, “…the majority of the published nonclinical and clinical studies, and history of use, support the safety of ephedra at the proposed use levels”. 35
Millions of users, billions of doses, no piles of bodies, no hard evidence regarding serious adverse events such as heart attacks and strokes. Ephedra is safer than some other foods. Even the consumption of peanuts is reported to kill about 100 people a year. 18 Yet ephedra is illegal, at least for now.
REFERENCES:
1. N Engl J Med. 1994 Apr 14;330(15):1029-35. The effect of vitamin E and beta carotene on the incidence of lung cancer and other cancers in male smokers. The Alpha-Tocopherol, Beta Carotene Cancer Prevention Study Group.
2. July 2004 American Journal of Epidemiology Development of a Comprehensive Dietary Antioxidant Index and Application to Lung Cancer Risk in a Cohort of Male Smokers. Margaret E. Wright , Susan T. Mayne, Rachael Z. Stolzenberg-Solomon, Zhaohai Li, Pirjo Pietinen, Philip R. Taylor, Jarmo Virtamo and Demetrius Albanes
3. Am J Clin Nutr 2005;82:879–886. Plasma carotene and alpha-tocopherol in relation to 10-y all-cause and cause-specific mortality in European elderly: The Survey in Europe on Nutrition and the Elderly, a Concerted Action (SENECA). Buijsse B, Feskens EJ, Schlettwein-Gsell D, Ferry M, Kok FJ, Kromhout D, de Groot LC.
4. Edgar R. Miller, III, MD, PhD; et al. High-dose vitamin E supplementation may increase all-cause mortality, a dose response meta-analysis of randomized trials. Annals of Internal Medicine: Online: Nov. 10, 2004: Print: 4 January 2005 Volume 142 Issue 1
5. John N Hathcock, et al. REVIEW ARTICLE: Vitamins E and C are safe across a broad range of intakes. American Journal of Clinical Nutrition, Vol. 81, No. 4, 736-745, April 2005.
6. Satia-Abouta J, et al. Dietary supplement use and medical conditions. Am Journal Preventive Medicine 24:43-51, January 2003.
7. The Lewin Group, DaVanzo, J. et al. Improving Public Health, Reducing Health Care Costs: An Evidence-Based Study of Five Dietary Supplements. September 22, 2004.
8. A study conducted by USA Today found that more than half of the experts hired to advise the government on the safety and effectiveness of medicine had a direct financial interest in the drug or topic they were asked to evaluate. An analysis of financial conflicts of interest at 159 FDA advisory committee meetings from January 1, 1998, through June 30, 2000, found that at 92% of the meetings, at least one member had a financial conflict of interest, while at 55% of meetings, half or more of the FDA advisers had conflicts of interest. These conflicts included helping a pharmaceutical company develop a medicine, then serving on an FDA advisory committee that judges the drug.
9. Food and Nutrition Board, Institute of Medicine. Dietary reference intakes for vitamin C, vitamin E, selenium, and carotenoids. A report of the Panel on Dietary Antioxidants and Related Compounds, Subcommittees on Upper Reference Levels of Nutrients and Interpretation and Uses of Dietary Reference Intakes, and the Standing Committee on the Scientific Evaluation of Dietary Reference Intakes. Washington, DC: National Academy Press, 2000.
10. Lonn E, et al. Effects of long-term vitamin E supplementation on cardiovascular events and cancer: a randomized controlled trial. JAMA. 2005 Mar 16;293(11):1338-47. PMID: 15769967
11. (Average 1982-1998): According to researchers, approximately 32,000 hospitalized patients (and possibly as many as 106,000) in the USA die each year because of adverse reactions to their prescribed medications. Source: Lazarou, J, Pomeranz, BH, Corey, PN, "Incidence of adverse drug reactions in hospitalized patients: a meta-analysis of prospective studies," Journal of the American Medical Association (Chicago, IL: American Medical Association, 1998), 1998;279:1200-1205, also letters column, "Adverse Drug Reactions in Hospitalized Patients," JAMA (Chicago, IL: AMA, 1998), Nov. 25, 1998, Vol. 280, No. 20.
12. Lazarou J, Pomeranz BH, Corey PN. Incidence of adverse drug reactions in hospitalized patients: a meta-analysis of prospective studies (JAMA): Incidence of adverse drug reactions in hospitalized patients: a meta-analysis of prospective studies. JAMA. 1998 Apr 15;279(15):1200-5. PMID: 9555760
13. FDA recent Recalls, Market Withdrawals and Safety Alerts: http://www.fda.gov/opacom/7alerts.html
14. FDA regulation of supplements: http://www.cfsan.fda.gov/~dms/supplmnt.html
15. Wheatley D. LI 160, an extract of St. John’s wort, versus amitriptyline in mildly to moderately depressed outpatients-a controlled 6 week clinical trial. Pharmacopsychiatry 1997; 30(suppl.):77–80.
16. Woelk H. Comparison of St. John’s wort and imipramine for treating depression: randomized controlled trial. BMJ 2000 Sep;321:536–9.
17. A total of 11 studies have compared SJW preparations with conventional antidepressants (7 tricyclic; 4 SSRI) concluding that SJW is effective for mild to moderate depression with a low side effect profile (Kasper, 2001). (Herbalgram) Kasper S. Hypericum perforatum– Review of clinical studies. Pharmacopsychiatry 2001;34 Suppl. 1:S51–5.
18. WATSON ET AL ■ 2003 AAPCC ANNUAL REPORT The American Journal of Emergency Medicine (22(5):335-404, 2004)
19. Bent S, Kane C, Shinohara K, et al. Saw palmetto for benign prostatic hyperplasia. NEJM. Feb 9, 2006;354(6):1-10.
20. Turner RB, Bauer R, Woelkart K, Hulsey TC, Gangemi DJ. An evaluation of Echinacea angustifolia preparations in experimental rhinovirus infections. N Engl J Med 2005;353:341-348
21. Blumenthal M, Farnsworth NR. Echinacea angustifolia rhinovirus infections [letter]. N Engl J Med. Nov.3, 2005;353(18):1971-1972.
22. 21 C.F.R. Pt. 119, Final Rule Declaring Dietary Supplements Containing Ephedrine Alkaloids Adulterated Because They Present an Unreasonable Risk (Published February 11, 2004) (Effective April 12, 2004) available at http://www.fda.gov/ohrms/dockets/98fr/1995n-0304-nfr0001.pdf
23. Szegedi A, Kohnen R, Dienel A, Kieser M. Acute treatment of moderate to severe depression with Hypericum extract WS® 5570 (St. John’s wort): randomized, controlled, double-blind, non-inferiority trial versus Peroxetine. BMJ 2005, BMJ Online First.
24. Taylor JA, Weber W, Standish L, Quinn H, Goesling J, McGann M, Calabrese C. Efficacy and safety of Echinacea in treating upper respiratory tract infections in children: A randomized controlled trial. J Amer Med Assn Dec 3, 2003;290(21):2824-30.
25. Fugh-Berman A, Myers A. Citrus aurantium, an ingredient of dietary supplements marketed for weight loss: current status of clinical and basic research. Exp Biol Med (Maywood). 2004 Sep;229(8):698-704. Review. PMID: 15337824
26. Xinhe Wang et al. Mechanism of arylating quinone toxicity involving Michael adduct formation and induction of endoplasmic reticulum stress. Proc. Natl. Acad. Sci. USA, 10.1073/pnas.0510962103 (2/27/06)
27. Glucosamine, chondroitin sulfate, and the two in combination for painful knee osteoarthritis. N Engl J Med. 2006 Feb 23;354(8):795-808. PMID: 16495392
28. Effects of Long-term Vitamin E Supplementation on Cardiovascular Events and Cancer. JAMA. Vol. 293 No. 11, Vol. 293 No. 11, March 16, 2005
29. Emmert DH, Kirchner JT. The role of vitamin E in the prevention of heart disease. Arch Fam Med. 1999 Nov-Dec;8(6):537-42.
30. Buijsse B, Feskens EJ, Schlettwein-Gsell D, Ferry M, Kok FJ, Kromhout D, de Groot LC. Plasma carotene and alpha-tocopherol in relation to 10-y all-cause and cause-specific mortality in European elderly: the Survey in Europe on Nutrition and the Elderly, a Concerted Action (SENECA). Am J Clin Nutr. 2005 Oct;82(4):879-86. PMID: 16210720
31. Stampfer MJ, et al. Vitamin E consumption and the risk of coronary disease in women. N Engl J Med 1993;328:1444-9
32. Shekelle PG, et al. Efficacy and safety of ephedra and ephedrine for weight loss and athletic performance: a meta-analysis. JAMA. 2003 Mar 26;289(12):1537-45. Epub 2003 Mar 10. PMID: 12672771
33. Federal Register: February 11, 2004 (Volume 69, Number 28).
34. Morgenstern LB. Use of Ephedra-containing products and risk for hemorrhagic stroke. Neurology. 2003 Jan 4;60(1):132-5. Erratum in: Neurology. 2003 Mar 25;60(6):1055. PMID: 12525737
35. Soni MG, Carabin IG, Griffiths JC, Burdock GA. Safety of ephedra: lessons learned. Toxicol Lett. 2004 Apr 15;150(1):97-110. Review. PMID: 15068827
36. Schulman SP. L-arginine therapy in acute myocardial infarction: the Vascular Interaction With Age in Myocardial Infarction (VINTAGE MI) randomized clinical trial. JAMA. 2006 Jan 4;295(1):58-64. PMID: 16391217
37. Clegg DO, et al. Glucosamine, chondroitin sulfate, and the two in combination for painful knee osteoarthritis. N Engl J Med. 2006 Feb 23;354(8):795-808. PMID: 16495392
38. Ioannidis JPA (2005) Why Most Published Research Findings Are False. PLoS Med 2(8): e124
39. Low-fat diets do not protect women against heart attacks, strokes, breast cancer or colon cancer, a major study has found, contradicting what had once been promoted as one of the cornerstones of a healthy lifestyle. Low-Fat Diet's Benefits Rejected. Study Finds No Drop In Risk for Disease. Washington Post. Wednesday, February 8, 2006 http://www.washingtonpost.com/wp-dyn/content/article/2006/02/07/AR2006020701681.html
40. European Society of Cardiology http://www.escardio.org/initiatives/News/events/esc_congress/DietSupplements.htm
