The AI Fluency Trap: When Familiarity Looks Like Expertise

For many executives, AI has already become part of the daily workflow. They use it to summarize reports, find information, draft emails and work through routine tasks. But familiarity with the tools can create a false sense of fluency—and make it harder to recognize the gap between using AI and knowing how to use it strategically.

Microsoft research involving thousands of knowledge workers found that generative AI can reduce time spent on tasks such as writing, information retrieval and summarization. But those individual productivity gains do not automatically translate into changes in how organizations make decisions or operate. Research from McKinsey similarly finds that while AI adoption is widespread, most organizations are still struggling to turn it into significant enterprise-level impact.

The challenge, then, is not simply getting leaders to use AI more. It is learning to recognize where AI can change the work itself—and developing the judgment to know when, where and how to make that change.

So what does genuine AI fluency look like at the leadership level? Members of the Senior Executive AI Think Tank approach that question from different vantage points, spanning enterprise technology, research, finance, retail, design and AI implementation. Their experiences offer a closer look at what happens when leaders move beyond individual productivity and begin applying AI to decisions, workflows, business systems and organizational strategy.

“Comfort often comes from using AI frequently; fluency comes from knowing how to redesign work around it.”

– Dhyey Mavani, AI and Computational Math Researcher at Amherst College

SHARE IT

Redesign the Work, Not Just the Prompt

Dhyey Mavani, AI and Computational Math Researcher at Amherst College, works at the intersection of statistical learning, mathematical rigor and practical machine learning systems. His research perspective leads him to distinguish frequency of use from depth of understanding.

“I’ve seen this gap firsthand,” Mavani says. “Comfort often comes from using AI frequently; fluency comes from knowing how to redesign work around it.”

That redesign mindset is the important step. Rather than asking only what AI can accelerate, Mavani recommends asking what would change if intelligence became cheap and abundant.

“The shift happens when leaders stop asking, ‘What can AI help me do faster?’ and start asking, ‘What decisions, workflows or operating assumptions would I redesign if intelligence were suddenly cheap and abundant?’” he says.

His practical test is to choose one consequential workflow every week and push AI beyond drafting: interrogate assumptions, simulate alternatives, automate pieces of the process and evaluate whether the final decision improved.

“Fluency comes from repeated judgment, not prompt volume,” Mavani says.

Turn Confidence Into Capability

Gabriella Goddard, CEO and Director of AI Catalyst Leader™ at Brainsparker Ltd, has spent more than 20 years coaching executives, innovators and entrepreneurs. She sees a recurring pattern among nontechnical executives.

“In my experience, most nontechnical leaders start by using AI to search, summarize or draft emails, but many assume anything more sophisticated belongs to IT,” she says.

Her solution is deliberately hands-on. In her workshops, executives take a common business task, structure a prompt and turn it into a reusable workflow.

“That’s when the light bulbs go on,” Goddard says. “Leaders realize they’ve only been scratching the surface and that they’re entirely capable of using AI for far more complex work.”

The progression is incremental: Identify a recurring task, build a workflow, refine it through use and then tackle something more complex. Leaders end up learning best, she says, through experimentation and repetition.

“The fastest way for a leader to build an AI-capable team is to keep developing their own AI fluency first,” Goddard says.

Build the Context Layer Before the Agent

Paul Freeman, VP of AI and Strategic Intelligence at Test Rite Products, brings more than 25 years of experience across customer service, warehouse management, software development and global supply-chain operations. For him, the confidence gap becomes particularly dangerous when organizations confuse a fluent interface with a reliable system.

“This gap is well beyond leadership,” Freeman says. “It is dangerous at every level. A confident user who is not actually fluent will accept ungrounded answers, and a confident team will automate a workflow while the model quietly fills missing data.”

Freeman describes that as an AI fluency gap revealing a context gap.

“The people who are genuinely capable stay uneasy,” Freeman says. “They ask what data is captured, where it lives and whether the model has enough context to act.”

Freeman argues that the answer is not simply more agents. 

“Closing both gaps means putting a real context layer under the AI, not just adding agentic workflows,” he says. “Until that company context layer exists, more AI may just widen all the gaps.”

Democratize Experimentation

Manpinder Singh Panesar, Senior Solutions Architect at Amazon Web Services (AWS), helps enterprise customers design and scale cloud-native data, analytics and AI systems, including generative AI, agentic AI and responsible AI architectures.

Panesar sees AI fluency as an organizational pattern rather than an executive technical specialty. He compares today’s challenge with the earlier transition to cloud computing.

“I think this gap is real at every layer, not just leadership, and it isn’t unique to AI,” he says. “We saw the same with cloud: Adoption and productivity were two different things.”

That distinction matters for executives who worry they need to become AI engineers themselves. Panesar says they do not.

“A leader’s job isn’t to understand AI at the same depth as an SDE or AI engineer,” he says. “It is to steer, set guardrails and empower bottom-up decisions.”

The implication is that leaders should create conditions for experimentation rather than dictate a centralized catalog of AI use cases.

“Keep it democratized,” Panesar says. “Let teams closest to the problem experiment, make mistakes, learn and decide where AI truly improves productivity.”

That model also protects organizations from assuming one AI architecture will fit every function. Leadership establishes boundaries, accountability and goals while the people closest to the work identify where the technology actually helps.

Measure the Time AI Actually Returns

Sai Krishna Reddy Mudhiganti, Staff AI/ML Software Engineer at Samsung Semiconductor, works across AI strategy, product innovation, governance and enterprise adoption. He recommends changing the question leaders ask themselves.

“Instead of asking leaders, ‘How confident are you using AI?’ ask them, ‘How much time did AI save you last week, and where exactly did it save that time?’” he says.

That question introduces evidence into what can otherwise become a subjective self-assessment.

“Someone can feel confident after learning a few prompting techniques,” Mudhiganti says. “This question puts them on the spot because it requires better AI skills to give out the number.”

The point is not to dismiss email drafting or document summarization. Those uses can be valuable. But executives should use those gains as a baseline for asking what comes next.

“If the answer is writing an email faster or summarizing a document, that is still useful,” Mudhiganti says, “but it shows where their capability currently stands.”

“Business leaders must encourage their organizations to move past surface-level tools and embed data intelligence directly into high-impact strategic workflows.”

– Uttam Kumar, Engineering Manager at American Eagle Outfitters

SHARE IT

Move AI Into the Core Business

Uttam Kumar, Engineering Manager at American Eagle Outfitters, brings deep experience in retail technology, including point-of-sale systems, order management, inventory and customer relationship management.

Kumar argues that organizations can mistake administrative convenience for technology fluency.

“Many retail organizations mistake simple daily tasks like drafting emails or summarizing meeting notes for true technology fluency,” he says.

Those activities create comfort, but Kumar believes the larger opportunity comes when teams connect AI directly to the economics of the business.

“Real strategic capability emerges when teams apply smart systems directly to core operations like supply chain optimization and customer lifetime value,” he says.

Instead of asking employees to find another way to save five minutes, leaders can ask where better intelligence could influence revenue, customer retention, inventory or operational efficiency.

“In our teams, we closed this gap by creating structured, hands-on workshops that map automated intelligence directly to business revenue goals rather than personal productivity,” Kumar says.

For him, knowledge matters, but it matters more when it is applied to meaningful business outcomes.

“Business leaders must encourage their organizations to move past surface-level tools and embed data intelligence directly into high-impact strategic workflows,” he says.

Test Whether AI Changes the Decision

Dr. Aditya Vikram Kashyap, Vice President of Firmwide Innovation at Morgan Stanley, leads enterprise transformation and innovation initiatives spanning AI integration, governance, data strategy and emerging technologies in financial services. He describes the central problem as a gap in ambition.

“Searching, summarizing and drafting prove adoption; they do not prove transformation,” Kashyap says. “Fluency begins when leaders use AI to interrogate assumptions, model scenarios, challenge decisions and redesign workflows.”

That is a higher bar because it asks whether AI changes the quality of management rather than simply reducing the effort required to perform existing tasks.

Kashyap recommends that executives audit their own recent AI activity.

“Did it merely save you time, or did it change the quality of a decision?” he asks.

Then, he says, leaders should redesign one consequential workflow around human-AI collaboration. The goal is not to eliminate human judgment but to build AI into the architecture through which decisions are made.

“The organizations that pull ahead will not be those with the most AI users,” Kashyap says, “but those that move AI from the productivity layer into the decision architecture of the enterprise.”

Judge Fluency by Business Outcomes

Venkata Kondepati, Manager of Data Architecture and Engineering at Ascentt, has more than 24 years of experience in cloud engineering, data platforms and enterprise software development, including leadership of large-scale cloud and AI modernization initiatives. He sees the confidence-versus-capability gap frequently.

“Many leaders equate AI usage with AI fluency because they use it for search, summarization or content generation,” he says.

Those uses create productivity gains, but he argues that they rarely change how an organization operates.

“True AI fluency means redesigning decisions, workflows and business processes around AI while understanding its limitations, risks and governance requirements,” Kondepati says.

He recommends experimenting on real business problems rather than training in the abstract.

“The best way to close the gap is through hands-on experimentation with real business problems,” he says.

That also changes what leaders measure. Instead of counting licenses, prompts or AI projects, organizations should ask whether decisions are faster, customer experiences are better, quality has improved or new revenue has emerged.

“AI fluency is demonstrated through better business decisions, not better prompts,” Kondepati says.

Look Beyond the Prompt History

Goran Paun, Principal and Creative Director at ArtVersion, leads work at the intersection of design, technology and human-centered thinking for enterprises and other organizations.

Paun believes the question of fluency can itself be misleading if leaders are evaluated only by what they personally do with a chatbot.

“In mature industries, the most advanced AI is not visible in a leader’s prompt history,” he says. “It is embedded in forecasting, fraud detection, maintenance, pricing, logistics and other operating systems.”

That means executives do not need to personally prompt every model to demonstrate fluency.

“Fluency means knowing where AI creates an advantage, what data and controls it requires, how failure will be detected and who remains accountable.”

This is key as AI becomes embedded in existing systems. A leader may use a chatbot primarily for proofreading while overseeing an organization where AI is already making or informing far more consequential predictions.

“Leaders should be assessed by the systems they shape and the outcomes they can explain—not only by what they asked a chatbot last week,” Paun says.

“There are many things you don’t know about AI, and just keeping on top of what’s going on in the world of AI is a full-time job.”

– Edward Morris, CEO and Lead Prompt Engineer at Enigmatica

SHARE IT

Bring in People Who Specialize in AI

Edward Morris, CEO and Lead Prompt Engineer at Enigmatica, leads an AI consultancy focused on enterprise generative AI implementation, prompt engineering and AI systems. He takes a more pointed position: Executives should recognize when their own expertise is not enough.

“There is a huge gap,” Morris says. “I would beg business owners and people to put their ego to the side and hire a Forward Deployment Engineer or at least a Prompt Engineer.”

The reason, he argues, is that keeping up with the technology is itself a specialized job.

“There are many things you don’t know about AI,” he says, “and just keeping on top of what’s going on in the world of AI is a full-time job.”

Morris says organizations also need specialists who understand the economics of implementation, not merely whether an AI system can perform a task.

“You need people who can push AI to its limit while keeping costs down,” he says.

That includes optimizing models, workflows and token consumption rather than accepting an expensive first version of an AI agent.

“Anyone can create an AI Agent that burns through tokens to handle menial tasks,” Morris says, “but where you’ll see ROI is if that same person takes what they have made and optimizes it massively for token burn.”

His broader point is less about outsourcing AI responsibility than recognizing that specialized expertise can accelerate responsible execution.

“Simply saying ‘I’ll handle it!’ as management is arrogant and egotistical,” Morris says. “There are reasons why these specialists exist.”

How Leaders Can Close the AI Fluency Gap

• Redesign one consequential workflow every week. Move AI beyond drafting by using it to interrogate assumptions, simulate alternatives and improve decisions.
• Turn recurring tasks into repeatable AI workflows. Hands-on experimentation helps leaders build capability through repetition rather than simply accumulating prompting tricks.
• Build a strong context layer before adding more agents. Reliable AI depends on understanding the data, context and governance that surround the model.
• Democratize AI experimentation. Give teams closest to the work room to test, fail, learn and identify where AI genuinely improves productivity.
• Measure the time AI actually returns. Ask exactly where AI saved time and use the answer to identify the next level of capability.
• Connect AI projects to business economics. Strategic fluency emerges when AI is applied to areas such as supply chains, customer value, revenue and operational performance.
• Measure AI by decision quality, not adoption. The strongest indicator of fluency is whether AI changes the quality, speed or outcome of important decisions.
• Evaluate the systems leaders shape, not just their prompts. Enterprise fluency includes knowing where AI belongs, what controls it needs and who remains accountable.
• Use real business problems as the classroom. Experimentation on consequential work reveals AI’s limitations and opportunities faster than abstract training alone.
• Know when to bring in specialists. Prompt engineers, AI architects and other specialized practitioners can help organizations optimize systems, costs and implementation as the technology evolves.

What Fluency Really Looks Like

The real test of AI fluency may be surprisingly mundane: Look at the calendar, the operating plan or the last major decision and ask where AI actually changed the work. If the answer is nowhere, a leader may be using AI regularly without yet using it meaningfully.

That is not necessarily a failure. Most organizations are still figuring out what this technology is good at, where it introduces risk and which parts of the business are worth redesigning. The mistake is treating early convenience as the finish line. Drafting an email is easy to measure. Rethinking how a company forecasts demand, evaluates a market, serves customers or makes investment decisions is harder—and far more consequential.

For executives, that distinction creates an opportunity. Instead of trying to become the person in the room who knows the most about AI, become the person willing to ask what the technology makes possible that the organization could not—or would not—do before. That is where familiarity starts to become fluency.