The Real Challenge of Scaling AI Beyond the Pilot

The AI industry has been celebrating increasingly capable demonstrations: a model beats a benchmark, an autonomous vehicle completes a route, an AI agent handles a task that once required a person. But a successful demonstration answers only one question: Can the technology work?

Commercialization demands much harder answers: Can it work every day? What happens when it fails? Who is responsible? Can the surrounding operation absorb those failures? And do the economics still work when real customers are paying for the result? Nevada’s recent approval of robotaxi networks for Tesla, Waymo and Uber opens the door to thousands of autonomous vehicles operating commercially, and brings those questions into sharp focus.

Members of the Senior Executive AI Think Tank bring perspectives from machine learning, enterprise technology, product management, data architecture, financial services, infrastructure, healthcare and design to this next stage of the AI conversation. Below, they examine what it takes to move beyond a working model and build a system that can withstand real-world complexity—from managing failures and infrastructure to earning regulatory and public trust, establishing accountability and making the economics work at scale.

Build for the Failure State, Not Just the Success Case

For Dhyey Mavani, AI and Computational Math Researcher at Amherst College, the central lesson is that commercialization begins after the demonstration has succeeded.

“The lesson from robotaxis is that commercialization begins where the demo ends,” Mavani says. “At scale, the hard problems become uptime, regulation, insurance, edge cases, incident response and public trust instead of just model accuracy.”

That matters for any executive evaluating an AI deployment. A pilot tends to measure whether the system can produce the desired result. A commercial system must also answer what happens when it produces the wrong result, becomes unavailable or encounters a circumstance it was not designed to handle.

“Thousands of revenue-generating vehicles also mean thousands of opportunities for rare failures to become operational failures,” Mavani says. “Many AI companies still underestimate this last mile: Building a system that works is different from building one that can fail safely, recover quickly and remain economically viable every day.”

Treat Physical Complexity as Part of the AI Product

Uttam Kumar, Engineering Manager at American Eagle Outfitters, sees the Nevada expansion as evidence that AI commercialization depends on integrating technology into complex physical operations.

“Transitioning from controlled pilots to thousands of revenue-generating vehicles signals that regulatory confidence and algorithmic accuracy are just entry stakes,” Kumar says.

For organizations outside transportation, the equivalent physical complexity might be a warehouse, hospital, factory, store or field-service network. The model can be excellent while the surrounding workflow remains incapable of supporting it.

Kumar says AI firms frequently underestimate “the massive friction of real-world operational readiness”—including vehicle maintenance, insurance frameworks and fleet utilization dynamics.

“The old adage reminds us that ‘In theory, there is no difference between theory and practice; in practice, there is,’” Kumar says. “Scaling AI requires conquering physical complexity.”

Executives can apply that lesson by mapping every dependency surrounding an AI use case before moving from pilot to production: infrastructure, staffing, vendors, physical assets, customer workflows, compliance requirements and service-level expectations.

“At scale, AI stops being a technology product and becomes an institution that must be governed.”

– Dr. Aditya Vikram Kashyap, Vice President of Firmwide Innovation at Morgan Stanley

SHARE IT

Turn AI Into an Enterprise Operating System

Dr. Aditya Vikram Kashyap, Vice President of Firmwide Innovation at Morgan Stanley, sees robotaxis as a model for a much broader transformation: Once AI reaches scale, governance becomes part of the product itself.

“Robotaxis reveal the real commercialization test for AI: not whether it works in a demo, but whether it can operate safely, repeatedly and economically in the physical world,” Kashyap says.

He argues that scaling autonomy requires more than improving the underlying model.

“Regulatory permission, insurance, maintenance, incident response, telemetry, human escalation and public trust must all scale with the software,” Kashyap says. “Most AI companies still underestimate this ‘last mile of intelligence.’”

He notes that while the model may be the breakthrough, the operating system around it becomes the business.

“At scale, AI stops being a technology product and becomes an institution that must be governed,” he says.

For executives, that means AI governance cannot remain a specialist function bolted onto deployment after the fact. Ownership, escalation, auditability and risk management should be designed alongside the system itself.

Make Reliability a System Property

Venkata Kondepati, Manager of Data Architecture and Engineering at Ascentt, approaches the issue from the perspective of enterprise platforms and infrastructure.

“The shift from pilots to thousands of revenue-generating autonomous vehicles shows that commercializing AI is ultimately an operations challenge, not just a model-performance challenge,” Kondepati says.

The difference between a pilot and a production platform is cumulative. Every dependency introduces another potential point of failure.

“A system can look impressive in a controlled pilot; at scale, it must handle edge cases, regulation, safety, uptime, insurance, infrastructure, customer support and economics—every day,” he says.

Kondepati identifies the gap between technical capability and dependable production as the critical “last mile.”

“The winners won’t necessarily have the smartest models; they’ll have the strongest systems around them: monitoring, governance, human escalation, integration and unit economics,” he says. “At scale, even a 0.1% failure rate becomes a recurring operational problem.”

Remember That Adoption Is a Human Process

Goran Paun, Principal and Creative Director at ArtVersion, brings a different dimension to the commercialization question: Technology can reach technical maturity before people are ready to trust and normalize it.

“Autonomous vehicles may have had the longest and most visible exposure of any major AI application,” Paun says. “People have seen them on the streets, ridden in them and watched the technology mature in public.”

That visibility creates an important advantage, as society gets repeated opportunities to form opinions about the technology.

“Other industries, like logistics, healthcare and manufacturing, have been using increasingly autonomous systems for years, but much of that progress is less visible,” Paun says.

He believes the current moment reflects not just technical maturity, but social acceptance.

“What we are seeing now is commercialization as technology is reaching maturity, but also social acceptance,” he says. “I think many AI companies underestimate how long that normalization process takes. A technology can be ready before the market, regulation or public trust is ready with it.”

That should change how executives think about AI adoption. User education, transparency and experience design are not secondary communications exercises. They can determine whether customers and employees actually use a system.

Build a Trust Ledger Before You Need It

Hastimal Jangid, Co-Founder of RankRabbit AI, argues that commercialization is ultimately a process of accumulating evidence.

“The gap between a pilot and 7,000 paid vehicles isn’t technology; it’s regulatory trust compounding slowly,” Jangid says. “Tesla went from a 10-vehicle permit to 5,000 in weeks not because the AI improved overnight, but because months of supervised, incident-free operation built the record regulators needed.”

Jangid says companies need to stop treating evidence-building as an obstacle to innovation.

“Most AI companies underestimate that commercialization is a trust ledger, not a capability threshold,” he says. “They ship a working demo and expect scale to follow performance.”

Instead, scale follows demonstrated responsibility.

“It doesn’t, it follows evidence, accumulated publicly, under scrutiny, with real consequences for failure,” Jangid says. “Companies stuck in pilot mode usually treat regulators and skeptical incumbents as obstacles to route around, instead of the actual gate to scale through.”

“Just as automobiles required roads, gas stations, traffic laws, insurance and repair networks, AVs need an ecosystem capable of supporting them safely, reliably and economically.”

– Fredrick Redd, CEO of Metrocap Advisors™

SHARE IT

Design the Ecosystem, Not Just the Technology

Fredrick Redd, CEO of Metrocap Advisors™, puts autonomous vehicles in the historical context of infrastructure adoption.

“The automobile took roughly 40 to 50 years to move from invention to mass adoption—not because the technology didn’t work, but because an entire ecosystem had to scale around it,” Redd says. “Autonomous vehicles are following a similar path, although AI could dramatically compress that timeline.”

The analogy highlights why individual AI performance cannot determine the speed of commercialization.

“Moving from AI pilots to thousands of AVs marks a commercialization inflection point,” Redd says. “Just as automobiles required roads, gas stations, traffic laws, insurance and repair networks, AVs need an ecosystem capable of supporting them safely, reliably and economically.”

The same logic applies to enterprise AI. A company deploying AI agents may need new policies, data infrastructure, security controls, workforce training, procurement models and customer-support processes.

“That’s where many AI companies underestimate the challenge. Scaling from successful pilots to millions of real-world transactions requires systems thinking,” Redd says. “Technology may ignite the AV revolution, but the ecosystem built around it will ultimately determine the speed and scale of commercialization—and the path to mass adoption.”

Put Consequences at the Center of the Business Model

Muthukumarapandian Chandrasekaran of CitiusTech emphasizes that commercialization changes the question an AI company must answer.

“Nevada approving thousands of robotaxis isn’t really a technology story. It’s a trust and infrastructure story,” Chandrasekaran says. “Tesla, Waymo and Uber didn’t get here by building a smarter model.”

Instead, he points to the less visible work behind the deployment.

“They got here by grinding through regulatory approval, insurance frameworks, city permitting and years of real-world edge cases most people never see,” he says.

The lesson is particularly relevant in healthcare and other high-consequence environments, where the cost of an AI failure can be much higher than a bad customer experience.

“That’s exactly what most AI companies underestimate,” Chandrasekaran says. “They treat commercialization as a scaling problem: more compute, more data, more users.”

But compute and users do not resolve questions of responsibility.

“The real bottleneck is everything around the model,” he says. “Who’s liable when it fails. How you handle the 1% of cases that break the pattern. How you earn regulator and public trust before you earn revenue.”

His final point is one executives can use as a commercialization test: “A pilot proves a model works. Commercial scale proves an organization can be trusted with the consequences when it doesn’t.”

“It is relatively easy to prove an AI system can work; it is much harder to make it reliable and economical enough to run every day, with real customers depending on it.”

– Meghana Makhija, Senior Product Manager Tech at Amazon

SHARE IT

Make Reliability and Economics Equal Goals

Meghana Makhija, Senior Product Manager Tech at Amazon, brings the product perspective to the transition from experimentation to production.

“Getting from a successful pilot to thousands of vehicles on the road shows how different building AI is from actually operating it at scale,” Makhija says. “Once AI is making real decisions every day, you have to deal with failures, edge cases, monitoring, regulation, customer trust and clear ownership when something goes wrong.”

Makhija highlights the nonlinear nature of rare failures.

“Even a failure that happens 0.1% of the time becomes a real operational problem at sufficient scale,” she says. “And all of that has a cost.”

That last point is easy to overlook amid enthusiasm about AI capabilities. A system can demonstrate technical feasibility while remaining too expensive, unpredictable or labor-intensive to operate.

“It is relatively easy to prove an AI system can work,” Makhija says. “It is much harder to make it reliable and economical enough to run every day, with real customers depending on it.”

Build the Safety Infrastructure Around AI

Will Conaway, President of Tuxedo Cat Consulting, frames the commercialization challenge through healthcare, where promising technology must eventually operate reliably across complex, high-stakes environments.

“Nevada’s approval of paid robotaxi services shows that commercializing AI is not simply about moving from a strong demo to a larger fleet,” Conaway says. “It is about proving the system can operate safely, consistently and accountably in messy real-world conditions.”

His healthcare comparison makes the operational transition particularly clear.

“In healthcare terms, this is the shift from a promising clinical trial to broad patient care: Performance must hold up across edge cases, monitoring, regulation, incident response and public trust,” Conaway says.

As AI moves into decisions that affect customers, patients, employees and financial outcomes, these parallels become increasingly important.

“Most AI companies still underestimate the operational burden after launch: maintenance, human oversight, liability, data quality, cybersecurity and continuous validation,” Conaway says.

The answer, he argues, is to build the supporting infrastructure as deliberately as the AI itself.

“Scaling AI is less about autonomy alone and more about building the safety infrastructure around it,” he says.

Tips for Commercializing AI at Scale

• Design for safe failure, not perfect performance. Build recovery, escalation and incident-response processes into the AI system before expanding deployment.
• Treat physical and operational complexity as part of the product. Map the people, infrastructure, vendors, regulations and workflows that must function alongside the model.
• Make governance part of the architecture. Define accountability, monitoring and decision rights before AI reaches production.
• Measure the entire system, not just the model. Track uptime, edge cases, operating costs, human interventions and customer outcomes alongside accuracy.
• Invest in normalization and trust. Give customers, employees and regulators repeated evidence that the technology is understandable, dependable and safe.
• Build a trust ledger. Accumulate evidence through controlled deployments, transparent reporting and demonstrated compliance rather than expecting technical capability to automatically unlock scale.
• Think in ecosystems. Identify the infrastructure, policies, partnerships and support networks required for AI to work beyond the pilot environment.
• Make consequences part of commercialization planning. Before launch, establish who owns failures, how unusual cases are handled and what happens when the system reaches its limits.
• Make reliability and economics equal product goals. A system is not commercially ready until it can deliver dependable outcomes at a sustainable cost.
• Build the safety infrastructure around the AI. Continuous validation, human oversight, cybersecurity, maintenance and clear accountability need to scale alongside the technology.

The Hard Part Starts After the Demo

A robotaxi doesn’t become a business when it completes its first successful trip. It becomes a business when thousands of trips can happen safely, reliably and profitably—and when customers, regulators and the company itself are prepared for the trips that don’t go according to plan. That is the real leap from AI pilot to commercial scale.

The next wave of AI commercialization will be decided in the messy space between the model and the market. That is where regulation meets engineering, where customer expectations meet failure modes and where an impressive piece of technology has to prove it can carry the weight of a real business. Nevada’s robotaxi experiment offers a glimpse of that future—and a reminder that getting AI to work may be the easy part. Getting the world ready to rely on it is the bigger job.