Skip to main content
Alma Health AI
Evidence & Methodology

What we've measured. How we measured it. What's available under NDA.

A working AI platform should be transparent about the methodology underneath the metrics. This page is our running statement of where Referral Review stands today — operating data, validation approach, and the documents available to diligence reviewers.

What we've measured

Operating data from eight production sites.

40%+

Reduction in total referrals, appropriately

Observed against pre-deployment routing patterns; the engine routes guideline-concordant primary-care-manageable conditions back to PCP-LED management. The figure is a directional operating metric, not yet a validated outcome. It is the primary signal that the engine is doing the work it was designed to do.

Eight production sites
88%+

Clinician concordance

Of the engine's recommendations reviewed by deploying clinicians, 88%+ concord with the engine's pathway. Coverage spans cardiology, endocrinology, GI, dermatology, orthopedics, and twenty-plus other specialties; the remainder are documented overrides with reason captured in the audit log.

Eight production sites · concordance audit
99%+

Engine uptime

Production batch mode. Sub-10-second decision latency under typical load. Failover and audit log integrity preserved through deploy windows.

operational measurement
0

Missed safety escalations

Across the production window. The 39 hardcoded patient-safety rules fire before any other engine logic and cannot be bypassed by any user. Closed-loop accountability requires immutable audit records — the safety rule layer fires upstream of every recommendation, and the audit log proves it.

v2.0.0 production window · audit log review
Validation methodology

What "validation" actually means — and where we are.

Operating data tells us the engine is doing what it was designed to do. Validation tells the world. Here's the framework — and our honest position on each piece.

Cohort definition

Specifying who's in and who's out

Production-validated cohorts are pre-specified — patient population, time window, exclusion criteria, denominator basis. Status: documented for the operating dataset; comparator-arm definitions in development.

Comparator structure

What the engine is being compared against

Operating measurement uses pre-deployment routing patterns at the operating sites. A retrospective comparator with matched-cohort design is the next step. Status: pre-deployment baseline measured; matched-cohort design in development.

Statistical approach

Significance, confidence intervals, sensitivity

Effect size, CI, sensitivity to cohort definition, robustness across specialties and sites. Status: descriptive statistics complete on the operating dataset; inferential analysis pending matched-cohort design.

Cohort scope

Production cohort

The current measurement cohort is eight production sites, with multiple additional sites expected to come online. Status: multi-site production deployment complete and expanding.

What's available under NDA

Documentation status, by document.

  • Validation methodology documentNDA Available
  • Engine architecture overviewNDA Available
  • Audit log schema and integrity proofNDA Available
  • Safety escalation list (39 rules)NDA Available
  • Operating dataset extractNDA Available
  • Specialty matrix list (40 specialties)Public
  • Condition coverage list (400+ conditions)Public

Buyers, plan diligence teams, and clinical advisors regularly ask to see the underlying work. Most of it exists in defensible form today and is available under a standard mutual NDA.

The validation methodology — authoring by board-certified physicians, stress-testing against the production cohort, ongoing pilot review by 20+ practicing primary care clinicians, and twice-yearly minimum update cadence — is documented and available under NDA.

Request documents under NDA →

Skeptical? Good.

The right diligence question for any healthcare AI platform isn't whether we'll show you the metrics — it's whether we'll show you the methodology underneath them. Ask us. The answer is yes.

Request validation documents →