95% of AI Pilots Fail: The Missing Metric is Trust

MIT study shows adoption collapses when AI lacks trustworthiness — and ROI follows.

Author: Dr. Shakira J. Grant

Date: September 4, 2025

Executive Summary

A recent MIT study shows that 95 percent of AI pilots fail, not because of broken algorithms but because products lack trustworthiness. The collapse of Babylon Health illustrates how quickly billions in value can evaporate when clinical rigor, data security, governance, and operational readiness are missing. For investors and business leaders, trust is not a soft value, it is the decisive factor that protects capital, drives adoption, and delivers ROI.

The Rise and Fall of an AI Unicorn

In 2017, Babylon Health was a $4.2 billion AI unicorn poised to disrupt health care. Six years later, it collapsed into bankruptcy. Its downfall was not caused by a lack of funding or innovation. It collapsed because trust, the foundation for adoption and ROI, was missing.

Reports revealed troubling practices: regulators questioned the safety of its AI, Excel files were presented as "clinical workflows," a data breach exposed private patient consultations, and financial losses cost the NHS millions. When trustworthiness fails, adoption collapses, and investment is lost. Babylon's story is a stark reminder that in the AI race, the hardest number to get right isn't the code, it’s trust.

Trust vs. Trustworthiness: The Critical Distinction

Trust is often defined as a belief in the reliability or intentions of a person or organization. In health care AI, this means whether physicians, patients, and payers are willing to rely on a tool.

But belief alone is fragile. Trustworthiness is different. It’s about whether a product and company actually deserve that trust. Trust may secure the first adoption. Trustworthiness is what sustains it over the long term. It is built on a consistent track record of reliability and integrity.

For investors, this distinction is critical: trust drives initial uptake, but trustworthiness sustains ROI.

The Silent Killer of AI Pilots

While Babylon's story is an extreme example, it highlights the risks that cause up to 95 percent of enterprise AI pilots to result in a zero return. This data from a recent MIT study highlights the stark reality: only 5% of custom AI tools result in a measurable impact. The issue is not weak algorithms, but weak trustworthiness, poor integration, messy data, and weak governance that kill business value before scale is reached.

Distrust translates directly into business risk:

• The "Confidently Wrong" Problem: Many AI systems provide incorrect or nonsensical outputs, forcing employees to spend more time fact-checking the results than the tool saves. This erodes trust and adoption.

• The Workflow Gap: Pilots are often treated as standalone "add-ons" rather than being deeply integrated into existing workflows. If an AI doesn’t fit into how work actually gets done, it won't be used.

• Misaligned Strategy: Companies chase AI for the sake of being "innovative," not because it solves a specific, high-value problem. This leads to pilots with no clear key performance indicators (KPIs) or connection to strategic goals.

• Messy Data: AI models need clean, organized data to be effective. Many companies have fragmented or low-quality data that prevents models from being trained and scaled.

  
  

These factors compound, resulting in sunk capital, delayed revenue streams, reputational harm, and increased regulatory scrutiny. For investors, the lesson is simple: pilot success is not product success.

The Investor Checklist: Aligning Trustworthiness with ROI

Before you write the check, go beyond the pitch deck and market projections. Your due diligence must answer one core question: Does this company truly deserve my trust? This isn't just a matter of opinion. The recommendations below are grounded in global best practices from leading organizations. From Microsoft's Responsible AI principles to the WHO's ethics and governance guidance for AI in health, these checklists are shaping how responsible AI is built and evaluated. Treat this checklist as due diligence insurance. If a company cannot answer these questions with evidence, your capital is at risk.

1. Clinical Reliability and Fairness

• Has the product been validated in real-world clinical settings across diverse populations?

• Are there peer-reviewed results that demonstrate safety and unbiased performance?

  Why it matters: Clinical rigor protects adoption and prevents regulatory setbacks.

  
  

2. Data Privacy and Security

• How is sensitive patient data protected?

• Does the company have a strong track record of compliance, security, and transparent data handling?

  Why it matters: Privacy safeguards prevent costly breaches and reputational damage.

  
  

3. Ethical Governance and Accountability

• Does leadership demonstrate transparency and accountability?

• Are there clear safeguards to ensure responsible development, bias mitigation, and ethical oversight?

  Why it matters: Strong governance builds confidence and protects long-term valuation.

  
  

4. Operational Inclusiveness and Transparency

• Can the tool seamlessly integrate into clinical workflows and scale beyond pilots into sustained adoption?

• Does the company demonstrate inclusiveness in how its tools serve different populations and settings?

  Why it matters: Operational readiness ensures scalability and sustainable ROI.

  
  

If a company cannot demonstrate strength across all four dimensions, it may still generate headlines, but it will struggle to scale, and your investment will carry outsized risk.

The Smarter Strategy

The collapse of Babylon Health shows the downside of neglecting trustworthiness. But the inverse is equally true. Building trustworthiness isn't an added cost; it's a competitive advantage. Companies that embed these principles into their DNA are not only more likely to avoid catastrophic failure but are also better positioned to scale faster, attract stronger partners, and establish lasting market credibility.

For investors and business leaders, demanding alignment with responsible AI principles isn't just an ethical choice. It is the smartest strategy to protect your capital and build a legacy of sustainable growth. The next time an AI pitch lands on your desk, don't just ask about the tech or the total available market (TAM). Ask about trust. Because, for health AI trust is not a soft value. It is the hardest number in ROI.

Want to stay on top of the latest insights in health care AI? Join our community to receive updates on our blog series and other resources.

Image Credits

Text to image generated by ChatGPT (Sora), September 4, 2025, OpenAI, https://chat.openai.com