The New AI Architecture: Balancing Data Sovereignty and Speed

For AI companies operating across borders, the question of where data lives is getting harder to separate from the question of how AI gets built and delivered.

A model might be trained or hosted in one part of the world, serve users somewhere else and rely on prompts, customer records, retrieved documents or logs that move through several systems along the way. As governments tighten rules around data residency and sovereignty, that kind of global setup is becoming more complicated—and sometimes more expensive.

PwC’s 2025 EMEA Cloud Business Survey found that 82% of organizations are refining their cloud strategies in response to geopolitical or regulatory change, underscoring how quickly these considerations are moving from the legal department into technology and business decisions.

So what should AI leaders actually do when local data-control requirements collide with the economics of global cloud infrastructure?

Members of the Senior Executive AI Think Tank—a curated group of experts specializing in machine learning, generative AI and enterprise AI applications—approach the question from different angles. Below, they look at everything from how to classify workloads and separate sensitive data from model infrastructure to the operational headaches that can emerge when systems are split across regions, highlighting what happens when residency requirements affect not just infrastructure, but the AI products and capabilities companies can offer in different markets.

“Maximum centralization gives you better economics; maximum localization gives you greater sovereignty and control.”

– Richie Adetimehin, AI Advisory and Transformation Delivery Consultant at Visani America

SHARE IT

Design Sovereignty Into the Architecture

Richie Adetimehin, AI Advisory and Transformation Delivery Consultant at Visani America, argues that organizations should resist the temptation to impose one global architecture on every AI workload.

“My approach is a federated model that keeps regulated data and sensitive inference within approved regional boundaries, while using global cloud capacity for workloads that don’t require the same controls,” Adetimehin says. “Add data classification, regional model gateways, encryption, workload routing and clear policies governing what data can cross which boundary.”

Adetimehin says executives should accept that sovereignty creates unavoidable economic choices.

“Maximum centralization gives you better economics; maximum localization gives you greater sovereignty and control,” he says. “Trying to optimize equally usually creates complexity and cost.”

He says the winning architecture will be able to answer one key question: “Where is the data, which model is processing it, who has authority and can we reroute or stop it when conditions change?”

Price Recovery Into the Sovereignty Model

Divya Parekh, Founder of executive coaching brand DivyaParekh.com, introduces a less-discussed operational issue: Regional isolation can make incidents harder to resolve.

“The trade-off that bites later is who’s allowed to look when something breaks,” Parekh says. “You separate the data properly, and then a region falls over at two in the morning. Your on-call engineer can’t read those logs, because that was the whole point of the design. So the fix waits for someone local to wake up, and the incident runs hours longer than it would anywhere else in your estate.”

If engineers responsible for a global AI platform cannot access a regulated environment during an outage, an architecture that looks excellent on a compliance diagram can produce significantly longer recovery times.

“Every jurisdiction holding sensitive data needs someone who can actually see inside it during an incident, on a rota, not as a favor,” she says.

That makes recovery performance a sovereignty metric, not merely an IT operations metric.

“The number I’d watch is time to recovery by region,” she says. “If one is consistently slower, that’s where the design is costing you, and it only ever shows up on a bad night.”

Match Infrastructure to the Regulatory Environment

Edward Morris, CEO and Lead Prompt Engineer at Enigmatica, brings an AI education and implementation perspective to the sovereignty question. He argues that there is no universal infrastructure answer.

“The right approach will depend heavily on both the sensitivity of the data and the operational environment,” he says. “In some cases, a single globally distributed cloud architecture may be too broad a fit for local regulatory requirements.”

Instead, he sees decentralization as a way to align technical architecture with local conditions.

“A more resilient model could be to decentralize the infrastructure, keeping data and compute within specific jurisdictions while deploying models that are adapted to local regulation, language and operational context,” Morris says.

Build for Sovereignty by Architecture

Dileep Rai, Manager of Oracle Cloud Technology at Hachette Book Group (HBG), says compliance should influence architecture before deployment—not become a remediation exercise afterward.

“AI organizations should design for sovereignty by architecture, not retrofit compliance after deployment,” he says.

His preferred model is hybrid and federated: “Keep regulated data, sensitive embeddings and critical inference within approved jurisdictions while using global cloud capacity for less-sensitive workloads.”

The architecture can then use “policy-based routing, encryption, regional model endpoints, data minimization and portable architectures” to balance competing requirements.

Rai acknowledges the fundamental trade-off.

“The hardest trade-off is efficiency versus control. Fully localized infrastructure improves sovereignty but increases cost and fragmentation; centralized platforms improve economics but raise regulatory exposure,” he says.

The architecture that succeeds, he notes, will “classify data and workloads by risk, then place each where it can operate most efficiently within regulatory boundaries.”

Make Governance a Global Standard

Gaurav Rastogi, Senior Director of Enterprise Data Analytics, Data Science and Strategic Insights at Hertz, makes governance a central part of his recommendation.

“Data sovereignty is ultimately a governance challenge, not just an infrastructure challenge,” Rastogi says.

His preferred model separates local data control from global standards: “Sensitive data remains local, while models, policies and governance are standardized globally.”

That distinction can allow companies to avoid rebuilding every governance mechanism independently in every geography. Regional processing and policy-driven data access can protect sensitive information while keeping inference close to users where latency matters.

But Rastogi warns executives against pretending the three primary objectives—compliance, cost and latency—can always be maximized simultaneously.

“You rarely get maximum compliance, lowest cost and lowest latency simultaneously,” he says. “Companies will have to choose where to optimize.”

For him, trust, governance, observability and compliance are nonnegotiable.

“Cost and latency can be optimized, but losing customer trust or standing is far more expensive than adding infrastructure,” he says. “Scalable AI requires balancing global efficiency with local accountability.”

“Orchestrating localized isolation zones with centralized controls respects regulatory boundaries without sacrificing global performance.”

– Vivek Kumkar, Sr. GenAI Leader at Amazon Web Services (AWS)

SHARE IT

Decouple Training From Inference

Vivek Kumkar, Sr. GenAI Leader at Amazon Web Services (AWS), has a particularly infrastructure-focused view of the problem.

“Directing multi-region AI infrastructure requires decoupling training topology from production inference architecture to balance data sovereignty with global performance,” Kumkar says.

Geographic requirements for building a foundation model do not necessarily have to match those for serving every production request.

“Replicating foundation models across every region destroys unit economics,” he says.

Instead, Kumkar recommends a workload-aware architecture: sensitive processing and fine-tuned adapters remain regional, while anonymized tasks can use centralized compute fleets.

He also points to localized caching as a way to improve performance without moving sensitive telemetry across borders.

“Deploy localized caching for frequent intent patterns, reducing cross-region latency while maintaining telemetry isolation,” he says.

The catch is resilience. 

“The necessary trade-off is accepting regional compute quotas over seamless cross-region failover,” Kumkar says. “Orchestrating localized isolation zones with centralized controls respects regulatory boundaries without sacrificing global performance.”

Treat Residency as a Recurring Decision

Markus Kopko, AI-PM Transformation Architect and CPMAI Authority at PMotion.ai, identifies a governance failure that can undermine even a well-designed architecture: Classification happens once and then stops.

“Sovereignty gets designed as an architecture decision and then broken as a project one,” he says.

As projects evolve, new retrieval sources, fine-tuning datasets and logging practices can quietly change the original data flows.

“Teams classify data and workloads once, at design time, and then ship for two years while the use case moves,” Kopko says. “New retrieval sources get added. Fine-tuning sets grow. Logging expands. Nobody re-runs the classification, because no phase gate requires it. The architecture stays compliant on paper and drifts in production.”

His answer is to turn residency into a recurring project gate.

“Treat residency as a recurring decision gate, not a design assumption,” he says. “Re-classify data flows at every phase transition, name one owner for that decision and tie deployment approval to it.”

The trade-off no one prices in? Delivery cadence.

“Organizations that accept that friction will ship fewer AI systems and keep the ones they ship,” he says.

Minimize What Has to Stay Local

Pon Murugesh Devendren, SAP Enterprise AI Architect at Deloitte, argues that sovereignty does not necessarily require organizations to localize everything.

“Sensitive customer data, prompts, retrieved documents and audit records should stay inside the country or region where they belong,” he says. “Shared standards and centrally managed model updates can be used where regulations allow.”

That distinction allows companies to reduce the volume of information that ever crosses a jurisdictional boundary.

“In practice, this means regional AI environments with local storage, local processing for sensitive workloads, and routing rules that decide where each request can be handled,” Devendren says.

He adds that small, local models, caching and collecting only necessary data can all reduce response time without the need to move protected information across borders.

“AI companies should stop treating data residency as a legal review at the end of a project. It needs to shape the system from the beginning,” he says.

Separate the Model Plane From the Data Plane

Blake Crawford, Partner and CTO at Fusion Collective, argues that executives often make sovereignty unnecessarily complicated because they treat the model and the information flowing through it as one thing.

“Keep the models global, keep the data local, stop logging things you’d have to explain to a regulator,” he says. “Separate the data plane from the model plane.”

The important point, he says, is understanding exactly what may become regulated.

“Model weights are not personal data and can live anywhere,” Crawford says. “What’s regulated is the stuff that flows through them.”

That can include prompts, retrieved context, outputs, logs and fine-tuning datasets.

“Once you draw that line clearly, the problem shrinks from ‘replicate everything everywhere’ to ‘keep the sensitive flows local,’” he says.

“An AI model that is limited to only local data is no longer the same product, even if it still goes by the same name.”

– Maitrik Patel, Sr. Engineering Manager at Apple

SHARE IT

Recognize That Sovereignty Can Change the Product

Maitrik Patel, Sr. Engineering Manager at Apple, says executives make a mistake when they view sovereignty strictly as a compliance constraint.

“Data sovereignty goes beyond having AI operate in a certain country,” he says. “It influences AI functionality in every region.”

That is because restricting a model’s access to information can change what the product is capable of doing.

“An AI model that is limited to only local data is no longer the same product, even if it still goes by the same name,” Patel says.

The implication is strategic as much as technical. Companies may need to decide whether a market is worth entering if localization requirements substantially alter the product experience.

“The biggest mistake that companies make is that they focus on the cost of infrastructure, instead of understanding that they have to sacrifice functionalities,” Patel says.

That makes sovereignty a product roadmap issue. Leaders need to evaluate not just the cost of regional infrastructure but what capabilities, data breadth and model performance may be lost by keeping information inside a particular boundary.

Building AI That Works Across Borders

• Classify AI workloads by risk rather than applying one global architecture. High-risk data and inference should remain local while lower-risk workloads can use centralized capacity.
• Design regional incident response before deployment. If local teams cannot access sensitive environments during failures, sovereignty can become a direct driver of downtime and recovery costs.
• Match infrastructure to local regulatory and operational realities. Jurisdiction, language, data sensitivity and business requirements should influence where compute and models operate.
• Build sovereignty into architecture from day one. Retrofitting residency controls after deployment creates cost, complexity and regulatory exposure.
• Make governance globally consistent even when infrastructure is local. Standardized policies, observability and accountability can provide enterprise-wide control without centralizing sensitive data.
• Separate training architecture from inference architecture. Foundation models do not necessarily need to be replicated everywhere if sensitive production workloads can be isolated regionally.
• Reassess residency throughout the AI lifecycle. New data sources, retrieval systems, fine-tuning datasets and logs can change the compliance profile after launch.
• Minimize the data that needs to cross borders. Local processing, smaller models and caching can preserve functionality while reducing data movement.
• Separate the model plane from the data plane. The objective should be to localize sensitive information flows rather than automatically duplicating entire AI stacks.
• Treat sovereignty as a product decision, not merely a compliance requirement. Regional restrictions can affect functionality, market entry, user experience and the competitive capabilities of an AI product.

Finding the Right Balance

The emerging consensus is not that global AI must become local—or that sovereignty requirements should be resisted. Instead, the most durable architecture is likely to be federated by design: local where regulation, risk or latency demands it and global where economics and scale provide a meaningful advantage.

That will require executives to accept that compliance, cost and latency cannot always be optimized simultaneously. The organizations that handle this well will be the ones willing to make some uncomfortable choices. That may mean paying more for local infrastructure, accepting slower recovery in certain regions or giving up some functionality to meet local requirements. The key is to make those choices deliberately—and early—rather than discovering the costs after the system is already in production.