Evidlab
Independent safety testing
for medical AI
When regulators, investors, or enterprise buyers ask whether your AI is trustworthy and ready for regulatory scrutiny, internal testing is rarely enough. We provide independent, credible evidence to support enterprise adoption, investor due diligence, and regulatory review.
Free safety scan: 250 scenarios, findings included. Full evaluation is a paid engagement. No spam. Ever.
Illustrative example · fictional product · synthetic data
How it works
From model access to evidence dossier in four steps.
Connect your model
Secure API access or on-prem deployment, whichever fits you better
We run the evaluation
Multi-turn patient conversations, adversarial safety probes, and realistic messy inputs — scored against physician-validated scenarios built on named clinical guidelines.
Findings
A findings report ranks every failure with the exact transcripts and quotes that triggered it. You can fix the issues — re-run after fixes is included.
Receive your evidence dossier
A citable report mapped to your framework — FDA, US state AI laws, RUAIH, NIST, EU AI Act, MDR, and more — with methods, results, and limitations, ready for due diligence, procurement, or regulators.
The full client journey in 83 seconds, no audio. Illustrative example · fictional product · synthetic data.
Evaluation methodology
Every model is measured the same way, every time. Our protocols are versioned, auditable, and designed to survive scrutiny.
Clinical validity — missed red flags, unsafe advice, and care-level routing, scored against physician-validated scenarios
Safety & hallucination — crisis protocol adherence, scope boundaries, and adversarial probing for dangerous advice
Robustness — real-world phrasing: typos, dialects, atypical presentations, consistency across repeated runs
Compliance behaviors — deterministic checks for AI disclosure, crisis resources, and required disclaimers
Scoring — deterministic rules + LLM judge (powered by Anthropic's Claude)
Illustrative. One measured behavior, mapped to every framework in scope — acceptance thresholds set per engagement, every status traceable to a dossier section.
Built for scrutiny
Evidence is only useful if regulators, buyers, and boards trust how it was produced.
Privacy handling
We use carefully produced test data, restrict access to each evaluation, and sign Business Associate Agreements when required.
Regulatory-ready outputs
Reports structured to support USA (FDA, US state-laws, etc.) and EU (AI Act, MDR) technical documentation.
Independent by design
We never build or fine-tune the models we evaluate. No equity, no consulting conflicts — just evidence.
CEO
Founded 2026 · Houston, Texas
FAQ
What kinds of models do you evaluate?
Patient-facing conversational AI: symptom checkers, triage bots, mental-health and companion chatbots, care-navigation and intake tools, voice agents. Clinical scribes and clinician-facing Q&A systems are on our radar as well — talk to us if that's you.
Who is the evidence dossier for?
It is written to serve three readers at once: investors running diligence, regulatory notified bodies, and hospital procurement committees comparing vendors. This dossier documents how your product behaved under our evaluation — measured safety behavior, methods, and limitations. Compliance and certification decisions stay with regulators and your organization; we produce the evidence they rely on.
Do you need access to our training data?
No. We evaluate the deployed model behavior only. You can grant API access or we run inside your environment — weights and training data stay with you.
How long does an evaluation take?
A standard evaluation across our core clinical suites takes 2-3 days, including report drafting.
What is free and what is paid?
The safety scan is free: 250 scenarios, with findings included. The full evaluation and evidence dossier is a paid engagement, scoped to your product and frameworks. Re-run after fixes is included.
Is Evidlab itself independent?
Yes. We take no equity in evaluated companies, do no model-building consulting, and our dataset is not used for training.