Four uncomfortable truths from the trenches, for the teams quietly wondering if they're the only ones whose AI pilot can't seem to make it to production.
Everyone's shipping AI right now. Or at least, everyone's trying to ship AI right now. The gap between those two things is where most of the interesting lessons live, and it's where we've spent the last year. Building visiontypes, prototypes, and production systems with clients across insurance, financial services, government, and legal.
Here's what we keep learning the hard way.
1. Data-readiness is not AI-readiness.
Everyone says clean your data first. We've watched teams with pristine data warehouses stall out at pilot, and scrappy teams with messy CSVs ship real results. So what's actually going on?
A story that sticks with us. An organization spent a full year building a beautiful data lake. Multiple tables, every join optimized, every column typed. Glorious work. Then they tried to build an AI chatbot on top of it and discovered the data their customers were actually asking about wasn't in the lake at all. It was in emails. And PDFs. And a few unloved CSVs sitting on someone's desktop.
.png)
The data lake wasn't wrong. It was just answering a question nobody was asking anymore.
The shift we'd offer: stop thinking about clean data and start thinking about useful evidence. Sit with the human who's going to use the AI. Ask where they apply judgment, where they copy-paste between systems, where they reach for help. That's the readiness work that actually matters.
2. Anyone can get AI to perform. The hard part is getting it to behave.
If there's one line we want you to walk away with, it's that one. Perform vs. behave.
Perform means you gave the AI a prompt and it did the thing. Once. In a demo. With friendly inputs. Probably on a Friday afternoon when everyone clapped.
Behave means it does the right thing every time, in production, knowing the terminology, respecting the rules, catching the edge cases, and not hallucinating when a real customer asks the question slightly sideways.
.png)
Performing is a prompting exercise. Behaving is an architecture exercise.
This is where most pilots get stuck. The first time you build an AI feature, of course it works. You fed it one use case, friendly data, and your own brain filling in the gaps. Production is full of weird inputs, weird users, and conversations long enough that the model starts forgetting things from earlier. You don't fix that with a better prompt. You fix it with curated architecture: business rules, policy documents, guardrails, and a system that can tell you why it failed when it fails, instead of silently shrugging.
If your AI is a black box that's wrong sometimes and nobody knows why, you don't have an AI problem. You have an architecture problem. That's actually good news, because architecture is something we already know how to do.
3. The best roadmaps came from building first.
The old way: plan for six months, then build.
What's working now: run a small experiment, learn something real, then build the plan from evidence instead of assumptions.
.png)
A real example. We had a client where 15 stakeholders across finance, compliance, marketing, and sales each wanted something different from a customer-facing app. Three weeks later, they were holding a working version of it on their phones.
- Week 1: A hyper-specific survey, with the responses fed into an AI-built context dashboard. The chaos sorted itself into themes.
- Week 2: A one-page narration. Not a 50-page spec, but a story, in plain English, of what the experience would feel like for a real user. That single page aligned every stakeholder.
- Week 3: Curated architecture, real code, real app, installable on a phone.
The roadmap that came out of that prototype was infinitely better than anything that could have been planned in advance, because it was informed by something real instead of something assumed.
The trick is picking small. One niche problem. Five fields. Two stakeholders. Resist the urge to solve the whole organization on day one.
4. The unlock wasn't AI. It was knowing where to put the human.
This is the one we want to leave you with.
Most AI conversations default to one framing: human-in-the-loop. The AI does the work, the human checks it like a hall monitor at the end. It's fine. It's also missing the point.
The framing we prefer is human-in-the-cockpit. The human isn't checking the AI's homework after the fact. The human is flying the thing, with the AI as the most powerful instrument panel they've ever had. The AI extends what the human can see, reach, and do. But the human is still the one making the decisions that matter.
.png)
The biggest breakthroughs we've watched land in production weren't about making the AI smarter. They were about getting the boundary right. Designing exactly where the AI stops and the human starts. Where does it take initiative? Where does it ask? Where does it surface options instead of just answers? Where does it step back entirely because the moment is human?
Take claims processing during a crisis. AI can do an enormous amount in claims. But when someone has just lost their home, no one wants to talk to a chatbot. They want a person. The AI's job in that moment isn't to handle the call. It's to free up the human's calendar so they can be fully present for it.
Get the boundary right and everything else falls into place. The trust shows up. The adoption shows up. Compliance gets comfortable. Customers actually use the thing.
Get it wrong and it doesn't matter how good your model is. You've built something nobody wants.
Working through any of this? We'd love to compare notes. [Get in touch →]
About the author
Tripti Mishra builds AI systems that survive contact with production. As a Solution Architect for Data & AI at SingleStone, she works with clients across insurance, financial services, government, and legal on the unglamorous middle of AI work: turning prototypes into things that behave, not just perform. She'd rather show you a working app than a roadmap. Find her on LinkedIn or reach out through SingleStone.