top of page
Independent Meta-Analysis Reproducibility & Integrity Audits

We don't publish new meta-analyses.
We verify them.

A published conclusion is only as sound as the trials, assumptions, and studies pooled into it. We independently reproduce an existing meta-analysis, methodically changing one thing at a time, and report whether the headline conclusion actually holds

The Reproducibility Portfolio

1

Evidence Integrity Audit

Can questionable, retracted, or fabricated trials change the published conclusion?

​

RISK: Undetected for years

 

A prolific bone researcher had 14 papers retracted for fabricated data — but not before his trials had already been pooled into published meta-analyses. Two of those meta-analyses were later retracted outright because they rested entirely on that data; a third needed a formal published revision. Nobody checked the trials' integrity until a whistleblower did, years after the fact.

 

WE PROVIDE:

  • Every included trial checked against retraction databases and journal notices

  • Pooled estimate re-run with flagged trials excluded

  • Documented verdict: does the headline conclusion move, and by how much


Case reference: Vitamin K & fracture risk — Sato research-fraud case

2

Statistical Robustness Audit

How sensitive is the conclusion to the analyst's own assumptions?

​

RISK: Assumption-dependent

​

Meta-analyses of hydroxychloroquine for COVID-19 reached opposite conclusions depending on whether they pooled randomized trials only or blended in observational data — and the field was destabilized further when Surgisphere-sourced studies, briefly influential enough to pause a WHO trial arm, were retracted within days of independent scrutiny. 

​

WE PROVIDE:​

  • Inclusion criteria varied: RCT-only vs. all study designs

  • Effect model varied: fixed-effect vs. random-effects

  • Subgroup and outlier handling stress-tested, one change at a time

​

Case reference: hydroxychloroquine for COVID-19 scandal

3

Living Evidence Update

Does the original conclusion hold up over time when new research is added?

​​

OUTCOME: Re-affirmed, updated

​

When the USPSTF updated its 2016 statin review in 2022, three newly eligible trials were added and the recommendation held — with more precision, not less. This is the model for a Living Evidence Update: rerun the original pooling logic against the full current literature and report whether the answer changed.

​

WE PROVIDE:​

  • Updated literature search & evidence synthesis

  • Documented verdict: does the headline conclusion move, and by how much​

​

Case reference:

Statin Use for the Primary Prevention of Cardiovascular Disease in Adults

How an audit runs

The four-step procedure, published in full alongside the verdict, so a third party could rerun it and get the same answer.
1 / SCOPE

Pin down the exact meta-analysis, version, and claim under review.

2 / REPRODUCE

Rebuild the original analysis from its own stated data and methods.

3 / SENSITIVITY ANALYSIS

Change one disclosed thing at a time — a trial, a model, a study design filter — and rerun.

4 / REPORT

A plain-language verdict plus a full methods trail anyone else could rerun.

Who this is built for:

Everyone asking "Is this evidence reliable?"
1 / Journals & systematic review teams

An independent second pass, run to fact-check and prevent the risk of a retraction notice.

2 / Pharma & medical affairs

Independent auditing to catch important and costly errors before launching an internal or public-facing product 

3 / Policy, HTA & guideline bodies

Know whether the evidence continues to supports existing recommendations, keeping policies up-to-date with literature as time progresses and our body of knowledge updates

3 / Researchers catching AI-fabricated references

A fast "re-do" of an article after eliminating any AI-hallucinated or retracted studies, when corrections are needed

bottom of page