Why AI fails when it ignores the workflow

Preeti Vaidya of Morgan Health on why useful AI starts with workflows, not chatbots, and what healthcare can teach enterprise teams about adoption.

I recently recorded a podcast episode with Preeti Vaidya, who works on AI at Morgan Health, part of JPMorganChase. Morgan Health focuses on improving outcomes for people covered by employer-sponsored insurance, which is a much larger population than many people realize. Roughly 160 million Americans receive health insurance through their employers, which means this is not a niche healthcare problem. It includes employees, families, benefits teams, clinicians, insurers, and the companies trying to help them all make better decisions.

Preeti sits at the intersection of healthcare, AI, data, clinical workflows, and employer-sponsored benefits. Naturally, we spent a lot of the conversation talking about AI in healthcare: care navigation, clinical decision support, radiology, fragmented health records, claims data, and the shortage of physicians. But the more we talked, the more I felt that healthcare was exposing a much broader enterprise AI problem.

Most AI programs still assume that the user wants to chat, and that’s wrong

Listen to the episode now on YouTube, Apple Podcasts, and Spotify.

---

That assumption works well for a very specific kind of user. It works for the person who is already curious about AI, already trying new tools, already comfortable writing prompts, and already willing to inspect the answer before trusting it. Preeti called these people “canary users.” Every organization has them. They are the early adopters who will test anything, find useful patterns, and prove that something is possible.

The mistake is treating their success as proof that the average user is ready.

Canary users are not the average user

Every company has people who are excited about new technology. They are not just the first users of AI. They are the first users of almost every new tool. They ask for access before the tool is formally rolled out. They try things outside the organization and then come back asking whether they can use them internally. They are motivated, curious, and willing to tolerate friction because they believe there is value on the other side.

These users are incredibly important. They help teams discover what a model can do, where the workflows might change, and which use cases are worth pursuing. But they can also create a false signal. When an AI pilot works with canary users, it can look like the organization is ready for broad adoption. In reality, all you have proven is that motivated users can make a flexible AI interface useful.

That is not the same thing as building a product the average user will actually use.

The average user usually does not want to become good at prompting. They do not want to understand model behavior. They do not want to learn how to inspect every answer for hallucinations. They do not want another destination they need to visit during the day. They want help doing the job they already have.

This is where many AI programs start to break. They give everyone access to a chatbot, point to the success of the power users, and then wonder why the rest of the organization does not adopt it with the same energy.

The wrong default is “here is a chatbot”

ChatGPT, December 2022

The original GenAI experience was ChatGPT, so it makes sense that a lot of people still imagine AI through that interface. There is an empty box. You type something. The model answers. You refine. You ask again. Sometimes it works incredibly well.

But that is not how most operational work happens.

Preeti gave the example of a nurse in an emergency room. A nurse does not need a blank chatbot in the middle of a clinical workflow. She does not have time to stop, describe the situation, ask the model what to do, evaluate whether the model is hallucinating, and then figure out how to translate the answer back into care. The better experience is much more embedded. If a marker changes, surface it. If a patient needs attention, alert the right person. If relevant context exists, put it in front of the clinician at the point of decision.

That distinction matters. The value is not in making the nurse “use AI.” The value is in helping her deliver care more effectively inside the workflow she already has.

I see the same pattern in analytics. Giving a business user a blank “chat with your data” box sounds powerful, but it often creates more work. The user asks a vague question. The model makes assumptions. The answer looks confident. Then someone has to verify what happened, inspect the SQL, check the metric definition, and decide whether the answer can be trusted.

At that point, AI did not remove work from the system. It moved the work somewhere else.

This is why the product question cannot be, “How do we get everyone to chat with AI?” The better question is, “Where in the existing workflow would better information, surfaced at the right moment, change the outcome?”

Start with the workflow, not the model

Preeti’s approach was very practical. Before deciding where AI belongs, you need to understand the workflow as it exists today. Not the idealized version. Not the version described in a strategy document. The actual workflow, with all the handoffs, shortcuts, delays, repeated checks, and moments where people rely on judgment.

That means talking to the people doing the work. What does the nurse do today? What does the radiologist do today? What does the benefits administrator do today? Where do they lose time? Where do they need more context? Where are they overloaded with alerts? Where are they forced to make a decision with incomplete information?

Only after that mapping does it make sense to decide where AI should enter.

This sounds obvious, but a lot of enterprise AI programs do the opposite. They start with a model, vendor, or platform, then go looking for places to apply it. That usually produces impressive demos and disappointing adoption. The demo shows what is possible in a controlled setting. The workflow reveals what is useful in the real world.

The better path is less flashy but much more durable. Understand the workflow. Find the decision point where better context matters. Insert AI there. Make the output understandable to the person doing the work. Do not force that person to become an AI operator just to benefit from the technology.

In healthcare, that might mean clinical decision support. In analytics, it might mean surfacing the right governed metric, explaining a dashboard, generating SQL against an approved semantic model, or warning the user that the question is ambiguous before producing a number.

The pattern is the same: AI works better when it meets people where they already work.

AI should support decisions, not pretend to make them

A lot of AI conversations eventually turn into the same question: will AI replace people?

In healthcare, that question often becomes: will AI replace doctors?

Preeti’s answer was clear. Healthcare is deeply personal. There are things AI can do extremely well, and there are things it should help with immediately. It can process large bodies of medical literature. It can help radiologists identify subtle changes in images. It can summarize relevant context. It can reduce administrative burden. It can help clinicians spend less time typing notes at the end of an exhausting shift.

But the human still matters.

A doctor may notice that something feels off even when the lab results look normal. A nurse may see a patient in a way the data does not capture. A clinician brings judgment, empathy, and accountability to the room. AI can support that work, but it should not be framed as a replacement for it.

That framing is useful outside healthcare too. In analytics, the goal should not be to pretend the AI is the analyst, the data team, and the business decision-maker all at once. The goal should be to help people make better decisions with less friction. That means giving them relevant context, governed definitions, clear assumptions, and answers that can be inspected and trusted.

Decision support is a stronger product promise than decision replacement. It is also a more honest one.

Data fragmentation is not one problem

At one point in the conversation, I asked Preeti about something I personally feel as a patient: my healthcare data is everywhere. I have multiple portals across different providers. There are labs, clinical notes, insurance claims, wearable data, family history, and information I may or may not remember to share. Even if I personally want all of it connected, the system does not make that easy.

So the question was simple: how can AI work if the data is so fragmented?

Preeti’s answer was basically yes, data fragmentation is a huge problem, but no, we do not need to solve all of it before AI can be useful.

I liked that answer because it avoids two bad extremes. One extreme is pretending the data problem does not matter. The other is waiting for a perfectly unified data world before building anything useful. Neither is realistic.

Preeti broke the problem into several layers. There is system fragmentation, where different providers and systems hold different parts of the patient picture. There is temporal fragmentation, where claims data may be useful but delayed by 30 to 90 days. There is also semantic fragmentation, where different systems describe things differently, and where clinical notes, claims codes, lab results, and provider judgment do not automatically line up just because the data has been collected.

That last layer is especially important.

Semantic fragmentation is not just a healthcare problem. Every enterprise has it. One team says “customer,” another says “account,” and another says “workspace.” A revenue metric means one thing in finance and something slightly different in sales. A dashboard contains logic that is not fully documented. A dbt model contains logic that is technically correct but not obvious to a business user. A Slack thread explains the exception, but the AI cannot see it.

Then someone asks an AI analyst a question and expects it to know what the business means.

This is why AI for analytics cannot just be a Text2SQL problem. Generating SQL is not enough. The system needs business meaning. It needs definitions, relationships, assumptions, governance, and context about how the data is actually used. Without that semantic layer, the AI may produce an answer that is syntactically correct and operationally wrong.

Build for incomplete context

One of the most important points Preeti made was that AI builders need to understand not only what their data can do, but what it cannot do.

That is easy to say and hard to implement. Most AI demos are built around the happy path. The model has the right data. The user asks a clean question. The answer exists. The output looks impressive.

Production is not the happy path. Production is missing context, ambiguous language, conflicting definitions, lagged data, undocumented business logic, and users who do not know exactly how to phrase what they need. The system has enough information to sound confident, but not always enough information to be right.

A useful AI system needs to recognize that gap.

In healthcare, that might mean knowing that it has today’s lab result and the current visit notes, but not the full patient history. In analytics, it might mean knowing that there are two definitions of gross margin and asking which one should apply. In both cases, the system becomes more trustworthy when it can explain what it knows, what it does not know, and which assumptions it is making.

This is one of the places where I think semantic engineering becomes essential. The goal is not to create a perfect data world. The goal is to make enough meaning explicit so that AI can behave responsibly in an imperfect one.

That includes knowing when to answer, when to ask a clarifying question, when to use a governed definition, and when to say that the data does not support the request.

The best AI may feel less like AI

One of the interesting patterns in the conversation was that the best healthcare AI examples did not sound like standalone AI tools. They sounded like better workflows.

Care navigation that appears inside the channels people already use. Radiology support that helps clinicians review images more effectively. Clinical context that appears when it is needed. Documentation support that reduces after-hours work. These are not experiences where the user goes somewhere else to “do AI.” They are experiences where AI makes the existing work easier.

That may be a good test for enterprise AI products.

If adoption depends on users leaving their workflow, learning a new behavior, and becoming good at prompting, the bar is very high. If AI is embedded into the place where the work already happens, the bar is different. The user does not need to believe in the AI transformation story. They just need the next step to be easier, faster, or more reliable.

This also changes how we should evaluate success. The question is not how many people logged into the AI tool. The question is whether the workflow improved. Did the clinician save time? Did the patient find the right provider? Did the analyst avoid repetitive work? Did the business user get a governed answer instead of creating another ad hoc spreadsheet? Did the data team reduce the amount of manual clarification required before answering a question?

AI adoption is not the outcome. Better work is the outcome.

The real promise is less friction and better decisions

The optimistic version of AI in healthcare is not that we all get an AI doctor and never see a human again. The optimistic version is more practical and, I think, more compelling.

Clinicians spend less time on administrative work. Patients get better navigation. Benefits become easier to understand. Radiologists get support spotting subtle changes. Doctors get faster access to relevant research. People in underserved areas get more consistent support. The shortage of clinicians becomes less painful than it otherwise would have been.

AI does not need to replace the system to improve it. It needs to reduce friction in the right places.

That is the lesson I took from the conversation with Preeti. It is also the lesson I think applies to almost every enterprise AI effort. The future will not belong to the flashiest chatbot. It will belong to systems that understand the workflow, respect the limits of the data, expose their assumptions, and help humans make better decisions.

That is true in healthcare.

It is true in analytics.

And it is probably true everywhere AI actually needs to work.

Listen to the episode now: YouTube, Spotify, Apple Podcast.