From Pilot to Payoff: Why Trust Architecture, Not Hype, Drives AI ROI

Author: Dr. Shakira J. Grant

Date: 9-18-2025

Midway into the visit, I was already being recorded when my clinician paused to ask permission, not the other way around, and the trust in the room shifted. During a webinar this past week with community members at a church in West Raleigh, North Carolina, the questions were blunt: “What happens to the recording?” “Are they allowed to do that?” “What about telemedicine? Are they recording those visits too?” and another person said, “That sounds scary. What about privacy?”

What we are talking about

Ambient AI scribes record the visit audio during a patient’s encounter with a healthcare professional, summarize the medical conversation, and generate a draft note in the electronic health record (EHR). In several large deployments, the audio itself is not retained; clinicians review a transcript and a draft summary, then move on. Even so, the questions from this group of community members remain: what happens to the raw data, and do telehealth providers also use AI scribes?

Figure 1. Ambient Intelligence in action: Capturing the data without breaking the connection

Why these questions matter

They are not side issues; they decide whether pilots become contracts and whether they scale across the enterprise. When people understand what is recorded, why, and for how long, trust rises, conversations become more candid, and clinicians get the whole story. That candor can unlock access to the care people need, referrals, social support, and adherence help that often depend on sensitive details. It also means that the EHR becomes a more accurate representation of a patient’s complexity, risks, and goals, which improves decision support, care coordination, and outcomes.

"Make trustworthiness the product, then the ROI shows up."

What that means in the boardroom

If trust makes rooms more candid and the EHR more complete, it also makes AI deployments more valuable. That is the context for a broader argument you could hear from executives at the Wall Street Journal’s Technology Council Summit this week: stop fixating on classical ROI for AI, start building the conditions that let value show up. Governance, strong data foundations, and focused use cases come first, then the numbers follow. In healthcare, the translation is straightforward; ROI means procurement speed, clinical safety, and equity, and you achieve these goals most effectively when trustworthiness is the product.

Privacy is the first gate

Too often, consent is ad hoc, timing and wording left to the moment, which nudges the patient-clinician relationship at exactly the wrong time. Productize it, offer a plain language pre-visit opt-in through the portal or SMS, add waiting room signage with a one-minute explainer, and publish a clear data use and retention policy.

Minimize data to move faster

If you do not keep raw audio, you shrink the blast radius of any breach, you shorten security reviews, and you lower perceived risk. Pair that with consent by design, and you have something patients and clinicians can understand and support.

There is also a safety backdrop that buyers cannot ignore. Transcription errors do not just slow clinicians; they can also miss medication changes, dosages, or key symptoms, and the risk is not evenly distributed. The accuracy of AI scribes often drops for clinicians or patients who are not native English speakers, and for regional or international dialects, which turns a technical shortfall into an equity problem. Vendors need to show parity testing across accents and languages, publish the results, and monitor production with clear escalation and rollback paths. If they cannot, those gaps become safety flags in procurement, the pilot stops at the threshold, and the ROI disappears.

Don’t let your AI strategy become a safety liability. At CROSS Global Research & Strategy, we help healthcare organizations and tech developers build Responsible AI that prioritizes patient safety. If you are deploying LLMs in oncology or chronic care, let’s discuss how to engineer the right guardrails. Schedule your AI Strategy session today!

Follow Dr. Grant and CROSS Global Research & Strategy on LinkedIn.