The most common way to misuse a good study is not to misread its statistics. It is to apply it to the wrong person. A trial can be well designed, adequately powered, honestly reported, and still say nothing useful about the client sitting across from you — because it studied a different population, a different product, a different regulatory environment, or a different question than the one you are being asked.
May 2026 produced an unusually clear run of that problem. A 2,199-patient meta-analysis found that a Chinese patent herbal medicine meaningfully reduced atrial fibrillation recurrence after ablation, with a corroborating 920-patient trial in the European Heart Journal behind it. The finding looks solid. It is also, in the authors’ own words, something for which “extrapolation to Western populations requires caution” — and the product is not available through any regulated supply chain outside China. A Lancet Neurology review synthesized decades of mechanism for ketogenic therapy in epilepsy, while noting that Cochrane-graded trial certainty remains limited and that the evidence base is concentrated in paediatric refractory disease.
Two more papers this month illustrate the subtler version of the same problem, where the population studied is not foreign but self-selected. Adherence to a planetary-health diet was associated with lower cancer incidence across 1.1 million people; adherence to that diet also travels with not smoking, exercising, and seeking preventive care. Organic produce substitution was associated with lower postmenopausal breast cancer risk in a French internet cohort that is 75% female and systematically wealthier and better educated than the population it is drawn from.
And one paper is about the evidence itself rather than any clinical question: an audit of how well black cohosh trials reported their methods and their harms. Its findings are not about whether the herb works. They are about whether we can tell.
Studies at a Glance
Real Findings, Different Patients
Three studies whose results are credible and whose populations are not yours — a cardiac herbal medicine studied and sold only in China, a mechanism review whose evidence base is paediatric refractory epilepsy, and a microbiome map drawn from a narrow, healthy cohort but framed as personalized nutrition.
Atrial fibrillation recurs in a substantial fraction of patients after catheter ablation, and adjunctive antiarrhythmic therapy has limited durable benefit. Shensong Yangxin is a twelve-herb Chinese patent medicine used widely in Chinese cardiology for this indication. This meta-analysis pools the randomized evidence.
Systematic review and meta-analysis of 9 randomized trials comprising 2,199 patients undergoing first catheter ablation for atrial fibrillation. Eight databases searched through 1 July 2025. Shensong Yangxin capsule as add-on to antiarrhythmic drugs versus antiarrhythmic drugs alone. Outcomes were early and late recurrence, analyzed separately for overall and persistent atrial fibrillation. Prospectively registered (PROSPERO CRD420251012453), with GRADE applied.
Show full methodology
The author team and institutions are entirely China-based. Blinding status, publication language, and trial-level industry funding for eight of the nine pooled trials could not be verified from accessible sources — a material gap for this literature, where unblinded single-centre trials with manufacturer involvement are a documented systemic concern. One identifiable component trial, SS-AFRF (European Heart Journal, 2024), was large, multicentre across 66 Chinese centres, double-blind and placebo-controlled at n=920, and reported a concordant effect (HR 0.6, 95% CI 0.4–0.8). Its inclusion in this pool is likely but unconfirmed. Publication bias assessment was not reported.
Both pooled estimates are statistically strong with no detectable heterogeneity, and adverse events did not differ from control. The GRADE ratings are the authors’ own signal of caution — moderate certainty for late recurrence and only low for early. An I² of exactly zero across nine trials is worth noting in both directions: it may reflect genuine consistency, or it may reflect that the pooled trials share design, setting, and reporting conventions closely enough that real variation never surfaces.
Strengths
- 2,199 patients across 9 randomized trials
- Prospectively registered with GRADE applied
- No detectable statistical heterogeneity
- A large, rigorous corroborating trial exists in a major cardiology journal
- Adverse events tracked and reported
Limitations
- Blinding, language, and funding unverified for 8 of 9 pooled trials
- Entirely China-based author team and trial populations
- Authors explicitly caution against Western extrapolation
- GRADE certainty only low-to-moderate
- Publication bias not assessed
- Product not available through regulated Western supply chains
This is not an actionable protocol for a practitioner outside China, and the reason is safety rather than skepticism about the data. Shensong Yangxin is not a standardized product available through regulated pharmacy or supplement channels in the US. Recommending an unregulated imported preparation as adjunctive therapy for a serious arrhythmia means unknown product quality, unknown interactions with the client’s actual antiarrhythmics and anticoagulants, and no oversight of sourcing. The correct posture is to watch this space for Western validation trials rather than to act on it.
A 45% reduction in recurrence reads like a prescribable herbal cardiac protocol. It is not one here. The stakes in post-ablation atrial fibrillation are real, the authors’ own caution about extrapolation should be treated as a hard stop rather than a footnote, and the pooled evidence base has unverified blinding and funding for most of its component trials.
Ketogenic diet therapy for epilepsy dates to the 1920s and remains in active clinical use, which makes the persistent thinness of its randomized evidence base genuinely unusual. This review synthesizes the mechanistic understanding rather than generating new outcome data.
Narrative review, not a systematic review or meta-analysis. No registration, no search protocol, no formal quality appraisal of included literature. The review covers classic ketogenic diet, modified Atkins diet, and low glycaemic index treatment, synthesizing mechanism across metabolic shift to ketone bodies, mitochondrial function, neuroinflammation, neurotransmitter modulation, and microbiome-mediated effects.
Show full methodology
Funded by NICHD and NINDS. Conflict disclosures are substantial and worth noting: one author reports an investigator-initiated grant from Jazz Pharmaceutical, paid advisory roles for Longboard Pharmaceuticals, data safety monitoring board fees from Praxis Precision and Lundbeck, textbook royalties, and a patent relating to cannabidiol use in a febrile epilepsy syndrome. As a narrative rather than systematic review, it is inherently subject to selective citation, with no PRISMA flow or explicit search strategy to constrain it.
The review’s substantive contribution is mechanistic: it maps how ketone metabolism, improved mitochondrial function, reduced neuroinflammation, and microbiome-mediated neurotransmitter changes plausibly converge on seizure control. Its most important sentence for a practitioner is the epistemic one — that Cochrane reviews and meta-analyses rank the certainty of randomized evidence as limited. The authors themselves call for rigorous large-scale comparative effectiveness studies accounting for age, epilepsy type, dietary modality, sociodemographic background, and care delivery context.
Strengths
- Authoritative venue and comprehensive mechanistic synthesis
- Explicitly flags the evidence gap rather than overstating certainty
- Covers multiple dietary modalities rather than one
- Authors call for the specific study designs the field needs
- Public NIH funding
Limitations
- Narrative review — no search protocol or quality appraisal
- Generates no new efficacy evidence
- Evidence base concentrated in paediatric refractory epilepsy
- Substantial pharmaceutical conflicts among authors
- Selective citation risk inherent to the format
This is useful for explaining to a client or a referring clinician why ketogenic therapy plausibly works. It is not new evidence that it does. The population where the evidence actually sits is paediatric drug-resistant epilepsy, delivered with coordinated medical and dietetic supervision given the restrictiveness and monitoring burden. Extrapolation to adult-onset or non-refractory epilepsy is much thinner than the century of clinical use implies. The KIWE trial (Lancet Neurology, 2023), which randomized 136 infants with drug-resistant epilepsy across 19 UK centres, found the ketogenic diet neither more nor less effective than adding another antiseizure medication.
A mechanism review in a prestigious journal can read as confirmation that the efficacy question is settled. It is not. The review says plainly that randomized certainty is limited, and the closest modern head-to-head trial found ketogenic therapy roughly on par with simply adding another medication — not superior to it.
Mapping which foods associate with which gut organisms, at species rather than genus resolution and at population scale, is a genuine advance over the small cross-sectional studies that dominate microbiome literature. This analysis draws on a deeply phenotyped Israeli cohort with metagenomic sequencing and app-based dietary logging.
Cross-sectional and 4-year longitudinal observational analysis of 10,068 participants in the Human Phenotype Project. App-based dietary logging paired with shotgun metagenomic sequencing at species and pathway resolution. Machine-learning predictive modeling with feature attribution, false discovery rate correction at 0.05 across species- and pathway-level tests.
Show full methodology
The parent cohort recruits generally healthy, ethnically homogeneous, higher-educated Israeli adults, excluding severe or chronic illness at recruitment. Age, sex, and BMI breakdowns, recruitment dates, attrition for the longitudinal subsample, and the covariate adjustment set were not reported in accessible sources. The personalized-intervention component is explicitly exploratory and in silico — it simulated dietary shifts and their predicted microbiome associations. No participant received an intervention. Conflict disclosures note one author is an employee of Pheno.AI and another is a paid consultant to it — a company positioned to commercialize personalized microbiome-based nutrition.
Specific food-organism pairs were strong and biologically sensible: coffee with Lawsonibacter asaccharolyticus at r = 0.43, yogurt with Streptococcus thermophilus at r = 0.42, milk with Bifidobacterium species at r = 0.31–0.36. Degree of food processing emerged as a predictor of diversity, with minimally processed intake the strongest positive and ultra-processed the strongest negative. The 92.4% figure requires careful reading. It reports how many species reached statistical significance after correction, not how much variance diet explains. At a sample size above ten thousand, very small associations clear significance thresholds routinely; the actual correlation coefficients sit between 0.24 and 0.43.
Strengths
- 10,068 participants with shotgun metagenomic sequencing
- Species-level rather than genus-level resolution
- FDR correction across a very large testing burden
- Four-year longitudinal tracking component
- Biologically plausible and consistent food-organism pairings
Limitations
- Single-country cohort of generally healthy, higher-educated adults
- App-based self-reported dietary logging
- Correlation coefficients are weak to moderate (0.24–0.43)
- The personalization component is simulated, not tested
- Two authors have financial ties to a personalized-nutrition company
- Covariate adjustment not reported
This supports the message you were already giving: minimally processed, varied whole-food diets associate with greater microbial diversity, and specific staples like coffee and fermented dairy track with specific organisms. It is useful for client education and framing. It is not a validated tool for prescribing an individualized microbiome-optimizing diet, and it should not be cited as clinical validation for commercial microbiome-based diet services — particularly given the authors’ commercial ties to exactly that market.
“Diet predicts 92.4% of gut species” sounds like diet controls the microbiome and that personalizing intake will reliably reshape it. The figure counts statistically significant associations in a very large sample, not explained variance. The correlations are modest, the cohort is narrow and healthy, and the personalization component was simulated rather than tested in a single participant.
The Healthy-User Problem
Two large observational analyses where the exposure travels in a bundle with everything else that lowers risk. Both are methodologically careful. Neither can fully separate the diet from the person who chooses it.
The EAT-Lancet reference diet was designed as a planetary-health framework rather than a clinical one, and its health outcomes are being evaluated retroactively. This meta-analysis pools cohort evidence on cancer incidence and mortality.
Systematic review and meta-analysis of 8 cohort studies comprising 1,103,953 participants. PubMed, MEDLINE, Embase, Cochrane, and EBSCO searched. Random-effects modeling with dual independent extraction. Both cancer incidence and cancer mortality assessed as co-primary outcomes. Prospectively registered (PROSPERO CRD42024520028), PRISMA 2020-compliant. Funded by a Chinese provincial higher-education science fund; no conflicts declared.
Show full methodology
Individual cohort names, countries, sample sizes, follow-up durations, and the covariate adjustment sets applied within each cohort were not reported in accessible text. I² values, publication bias assessment, GRADE certainty, and subgroup analyses by cancer type or region were likewise not reported. The authors concede that “given the limited number of available studies, further high-quality research is warranted” — an acknowledgment that eight cohorts is a thin base for a claim of this scope.
Both associations are statistically significant and in the expected direction. The incidence interval is notably wider than the mortality interval, consistent with more heterogeneity or less precision in that pooling — though I² was not reported, so this is inference. The evidence base is moving quickly and not converging cleanly: an independent meta-analysis of the same question published shortly afterward, pooling 15 cohorts rather than 8, reported cancer incidence at HR 0.90 (95% CI 0.84–0.95) with I² of 89%, and explicitly flagged inconsistent operationalization of the EAT-Lancet index across studies.
Strengths
- Over 1.1 million pooled participants
- Prospectively registered and PRISMA-compliant
- Dual independent data extraction
- Both incidence and mortality assessed
- Authors concede the evidence base is immature
Limitations
- Only 8 cohorts underpin a very large participant count
- I², publication bias, and GRADE all unreported
- Healthy-user confounding largely unaddressed
- Cohort-level covariate adjustment not standardized
- A larger independent meta-analysis already reports a different estimate
- EAT-Lancet index operationalized inconsistently across studies
For clients already drawn to plant-forward, sustainability-minded eating, this supports the direction of that choice. It does not support a quantified prevention claim. The central interpretive problem is that adherence to a planetary-health dietary pattern travels in a bundle with not smoking, exercising, drinking less, and engaging with preventive care — and cohort models adjust for some of those inconsistently and none of them completely. That the point estimate moved from 0.78 to 0.90 between two meta-analyses within months is the practical signal: treat the direction as informative and the magnitude as unsettled.
“The planetary health diet cuts cancer risk 22%” attributes to a dietary pattern what may substantially belong to the lifestyle cluster it accompanies. Eight observational cohorts, no reported heterogeneity statistic, and a competing larger analysis with a materially different estimate all argue for directional rather than numeric claims.
Whether organic produce confers health benefit beyond the produce itself is among the most-asked and least-well-evidenced questions in practice. This analysis is methodologically novel: rather than scoring how often someone buys organic, it models substituting organic for conventional at a fixed total intake, isolating the substitution question.
Prospective cohort analysis within NutriNet-Santé (NCT03335644). 31,179 participants, 75% female, followed a mean 7.3 years for 227,660 person-years, yielding 1,718 total cancers including 284 postmenopausal breast cancers. Exposure was substitution of conventional with organic fruit and vegetables at fixed total intake, per 100 g/day increment and by quintile. Multivariable Cox models plus marginal structural models to address time-varying confounding.
Show full methodology
The substitution design and use of marginal structural models are both genuine methodological improvements over this group’s widely cited 2018 JAMA Internal Medicine analysis, which scored organic consumption frequency and drew substantial criticism for uncontrolled socioeconomic confounding. The specific covariate adjustment set and the additional adjustments tested in sensitivity analysis were not reported in accessible text. No multiple-comparison correction was reported across the several cancer sites examined. Authors report no conflicts; no funding source was listed.
The postmenopausal breast cancer association held under both quintile analysis and marginal structural models. The overall cancer association did not — its upper confidence bound touches 1.00, and quintile analysis found no inverse association for overall cancer or for any other site. That selective pattern admits two readings. It may reflect a genuinely site-specific mechanism, which is biologically plausible given endocrine-disruptor hypotheses linking pesticide exposure to breast tissue. Or it may be one significant result among several sites tested without correction.
Strengths
- Substitution design isolates the organic-versus-conventional question at fixed intake
- Marginal structural models address time-varying confounding
- 227,660 person-years of follow-up
- Methodologically stronger than the group’s widely cited 2018 analysis
- Detailed questionnaire distinguishing organic from conventional purchases
Limitations
- Self-selected internet volunteer cohort, 75% female
- Organic buyers differ systematically in income, education, and overall diet quality
- Overall cancer association is borderline and did not survive quintile analysis
- No multiple-comparison correction across cancer sites
- Covariate adjustment set not disclosed
- Self-reported organic-versus-conventional purchasing is error-prone
For a client asking whether to buy organic, the honest answer is that adequate produce intake is the established lever and organic substitution offers, at most, a smaller and less certain additional benefit concentrated in one cancer site. This should not displace the primary recommendation, and it should not be framed as strongly evidence-based. The residual confounding concern is not hypothetical: within a self-selected cohort of health-engaged volunteers, organic purchasers differ from conventional purchasers in income, education, and general diet quality, and the marginal structural models are a real but unverifiable attempt to handle that.
“Organic produce lowers cancer risk” overstates a result whose overall estimate was borderline and whose one robust finding was a single subgroup among several tested without correction. The methodology here is better than its predecessor’s, which makes the modesty of the finding more informative, not less.
When the Headline Outruns the Result
Three papers whose most-quoted claim is the one their own data least supports — a bone-density benefit that was null, a mediation finding drawn from 1,400 candidates, and an efficacy conclusion drawn from a paper that measured no efficacy at all.
Creatine is increasingly promoted to menopausal women for muscle, strength, and bone, and the bone claim in particular has outrun its evidence. This meta-analysis pools the randomized trials specific to this population.
Systematic review and meta-analysis of 7 randomized placebo-controlled trials, 608 women randomized, mean age approximately 62, intervention durations 12 to 104 weeks with a median of 38. Six databases searched from 2000 to August 2025. Random-effects modeling using the Paule-Mandel estimator with Hartung-Knapp-Sidik-Jonkman adjustment, with 95% prediction intervals reported alongside confidence intervals. Cochrane RoB 2 and GRADE applied.
Show full methodology
The review was explicitly not prospectively registered, which the authors disclose; data were deposited on OSF afterward. Per-outcome samples are much smaller than the headline 608: lean mass rests on 5 trials and 338 women, leg-press strength on 3 trials and 111. Risk of bias was mostly rated “some concerns”. Conflict disclosures are extensive: one author chairs a scientific advisory board supported by Alzchem, a creatine manufacturer; another is chief executive of the society publishing the journal, which receives funding from creatine manufacturers; and Alzchem covered the article processing charge.
Both significant effects are small in absolute terms. The prediction interval for lean mass crosses zero (−0.10 to +0.84), meaning a new trial could plausibly find no effect at all — a distinction the confidence interval alone conceals, and one this review deserves credit for reporting. Benefits were evident specifically at 5 g/day or more combined with resistance training; trials at 3 g/day or less without training showed nothing. Bone mineral density, the outcome most likely to motivate a menopausal client, was null overall.
Strengths
- Conservative statistical methodology appropriate for few pooled trials
- Reports 95% prediction intervals, not just confidence intervals
- Cochrane RoB 2 and GRADE applied
- Dose and training subgroups explored rather than assumed
- Data deposited openly
- Safety and renal indices reassuring
Limitations
- Not prospectively registered, as the authors disclose
- Per-outcome samples of 338 and 111, far below the headline 608
- Prediction interval for lean mass crosses zero
- Bone density null, and possibly underpowered at a 38-week median
- Risk of bias mostly “some concerns”
- Creatine-industry funding of the article charge and author advisory roles
Creatine monohydrate at 5 g/day or more, paired with structured resistance training, is a reasonable recommendation for postmenopausal women seeking modest lean mass and lower-body strength gains, with a reassuring safety profile including renal indices. Set expectations honestly: roughly a third of a kilogram of lean mass, and individual response genuinely uncertain given a prediction interval that crosses zero. Plan for at least the 38-week median duration. Do not promise bone benefit. Note that an earlier landmark trial by Chilibeck did find femoral neck preservation over 12 months at a higher dose — that signal washes out when pooled with shorter, lower-dose trials, which is itself instructive.
“Creatine builds bone in menopause” is the claim this paper most directly contradicts — bone mineral density was unchanged. The real findings concern lean mass and strength, both modest. The industry-adjacent funding and authorship should temper the confident framing of the conclusion, even though nothing suggests the data itself is unsound.
Socioeconomic position is among the most consistent predictors of cognitive decline, and disentangling it from the health behaviours it travels with is genuinely difficult. Mendelian randomization offers one approach, using genetic variants as instruments to approximate randomization. Whether it works for an exposure as socially constructed as income is the question this paper raises.
Two-part design. First, a multistate Markov model on 13,457 UK Biobank participants with recorded cognitive transitions over a median 8.96 years, covering 14,040 transitions. Second, a two-sample, two-step Mendelian randomization using genetic instruments from the IEU OpenGWAS project, testing 17 dietary factors and 1,400 gut metabolites as mediators.
Show full methodology
Critical methodological detail is unavailable in accessible text: which MR estimator was used, how many instrument SNPs, F-statistics, pleiotropy and heterogeneity tests, and Steiger filtering are all unreported, as is any multiple-testing correction across the 1,400-metabolite screen. Mendelian randomization rests on three assumptions — that the instruments are robustly associated with the exposure, that no confounding links instrument to outcome, and critically that the instruments affect the outcome only through the exposure. That last assumption is difficult to defend for income, whose associated variants plausibly track cognitive ability directly. No funding was listed; authors declare no competing interests.
The observational arm found higher income associated with lower risk of cognitive deterioration, in line with a large existing literature. The MR estimate is where scrutiny belongs. An odds ratio of 2.14 for a socioeconomic exposure on cognition is far larger than the hazard ratios of roughly 1.3 to 2.3 typical of observational socioeconomic-dementia literature — and a substantially larger causal effect than the observational estimates should generally raise suspicion of instrument invalidity rather than confirm an unusually strong effect. Cheese and coffee were reported as significant mediators, with cheese acting via tryptophan, tyrosine, carnitine, and ergothioneine pathways.
Strengths
- Large, richly phenotyped cohort
- Triangulates observational and genetic approaches
- Novel use of a broad gut-metabolomic panel to probe mechanism
- Multistate modeling captures both deterioration and improvement
Limitations
- Exclusion-restriction assumption hard to defend for income
- Reverse causation plausible — cognitive ability influences earnings
- MR estimator, instrument count, and pleiotropy tests all unreported
- No stated correction across 17 foods and 1,400 metabolites
- UK Biobank healthy-volunteer selection bias, with collider bias risk
- No funding source reported
There is little here a practitioner can act on, and that is the honest assessment. Raising a client’s income is not a clinical intervention, and the mediation findings that would be actionable rest on the weakest part of the analysis. The defensible takeaway is a reminder to attend to the modifiable diet and lifestyle factors that correlate with socioeconomic position when working with lower-income clients on cognitive risk — which was already good practice.
Two leaps to refuse. The first is reading “MR shows income causes cognition” as settled causal inference; income is entangled with education, occupation, and reverse causation in ways that make the core MR assumption very hard to satisfy. The second is treating cheese and coffee as cognitive-protective interventions. Two mediators surfacing from 17 foods and 1,400 metabolites with no reported correction is the textbook shape of a multiple-testing artifact.
Black cohosh is among the most widely used botanicals for menopausal symptoms, and its efficacy literature is genuinely contradictory — a 2012 Cochrane review found insufficient evidence, while a 2020 meta-analysis specific to one standardized extract reported a large effect. This review asks a different and prior question: how well were those trials reported in the first place?
Systematic review grading 14 clinical trial records of oral Actaea racemosa for climacteric complaints against two reporting standards — the CONSORT Extension for Herbal Interventions and the CONSORT Harms Extension. Seven databases searched with an initial search in April 2023 and an update in March 2024, supplemented by citation searching. Search terms included named preparations BNO 1055, Remifemin, and Ze 450.
Show full methodology
This is a methodological audit, not an efficacy analysis. It reports no effect size for symptom relief, and any claim about whether black cohosh works drawn from this paper would be a fabrication. The trials span multiple chemically distinct preparations that are not interchangeable. No registration was reported, and the scoring methodology and inter-rater reliability were not detailed in accessible text. One co-author is employed at a company that owns a brand marketing a black cohosh product — a direct commercial interest in this herbal category, though the conflict bears less on a reporting audit than it would on an efficacy analysis.
The range is the finding. Some trials reported thoroughly; others reported adverse events at adherence as low as 6%, meaning the safety data from those trials is close to uninterpretable. That matters more than it sounds, because black cohosh carries an unresolved hepatotoxicity signal flagged by regulators in Australia, Canada, and the EU on the basis of roughly fifty case reports. A claim that black cohosh is safe, built on trials that barely reported harms, is not a claim the underlying data can support.
Strengths
- First application of CONSORT herbal and harms extensions to this literature
- Broad multi-database search with two waves and citation chasing
- Directly identifies which trials can bear clinical weight
- Authors conclude honestly that higher-quality investigation is needed
Limitations
- Measures reporting quality only — says nothing about efficacy
- No registration reported
- Scoring methodology and inter-rater reliability not detailed
- Pools chemically distinct preparations
- A co-author is employed by a company marketing a black cohosh product
Three practical consequences. Insist on extract-specific evidence: a trial of one standardized preparation does not validate another manufacturer’s product at a different standardization. Treat the overall efficacy literature as lower-confidence than headlines suggest, since this audit explains structurally why two prior reviews reached opposite conclusions — they pooled non-interchangeable extracts reported to inconsistent standards. And given that harms reporting was inadequate in a substantial share of trials, counsel on liver function monitoring, particularly for clients with existing liver disease or on hepatically metabolized medications.
Two opposite misreads are both available here. One is concluding the herb does not work — the paper does not measure efficacy. The other is dismissing it as a paperwork exercise. It is neither: when harms reporting adherence bottoms out at 6% in a botanical with a live hepatotoxicity signal, the safety reassurance built on those trials is worth considerably less than it appears.
Synthesis & Emerging Themes
Transferability Is a Separate Question From Validity
The Shensong Yangxin meta-analysis is, on its own terms, better evidence than most botanical literature achieves: 2,199 patients, no detectable heterogeneity, prospective registration, and a large double-blind trial in a major cardiology journal behind it. None of that makes it usable by a practitioner in Philadelphia, because the product is not available through any regulated channel and the arrhythmia it treats has real stakes if adjunctive therapy goes wrong. Validity and transferability are different properties, and a paper can have the first in full while offering none of the second. The reverse also happens: the ketogenic review contains little new evidence but transfers directly, because it explains mechanism to a clinician who already has patients on the therapy.
The Denominator Behind a Big Number
Two of this month’s most impressive-sounding figures shrink considerably on inspection. “Diet predicts 92.4% of gut species” counts how many associations cleared significance in a sample of ten thousand, not how much variance diet explains — the actual correlations run from 0.24 to 0.43. “Creatine improves outcomes in 608 postmenopausal women” describes a pooled randomization total; the lean mass estimate rests on 338 women and the strength estimate on 111. Neither paper misrepresents anything. Both invite a reader to carry forward the largest number in the abstract.
Mediators Found in Very Large Haystacks
The income-and-cognition analysis tested 17 dietary factors and 1,400 gut metabolites, reported no correction procedure, and surfaced cheese and coffee as significant mediators. Whatever the underlying biology, that is the shape a false positive takes. It is worth naming as a general reading habit: when a paper reports a surprising, specific, highly quotable mediator, the first question is how many candidates were screened to find it.
What Remains Unresolved
Whether black cohosh works, and whether it is hepatically safe, both remain genuinely open — and this month’s audit explains why, which is more useful than another contradictory efficacy review would have been. Whether organic produce carries a real, site-specific benefit for postmenopausal breast cancer or whether that signal is confounding and multiplicity is unresolved and, in a self-selected cohort, may not be resolvable. And whether creatine preserves bone at higher doses over longer durations is a live question that this pooled null does not close, given that a 12-month trial at a higher dose did find femoral neck preservation before being averaged away.
“Before asking whether a study is true, ask who it was true about.”
A discipline worth carrying into next month: for every finding you plan to act on, name the population it was established in and say out loud how it differs from the client in front of you.