AI Code Generation Is Faster. Is Software Delivery Better?

Writing code has always taken time. Now, with AI, developers can produce in minutes what might once have taken hours. For companies under pressure to move faster and do more with less, that sounds like a clear win. But getting code written is only one part of building software. Someone still has to review it, test it, make sure it’s secure and keep it running once it reaches customers.

So if teams are producing more code than ever, how do they know they’re actually getting more productive? Faster development doesn’t mean much if work piles up waiting for approval, bugs make their way into production or the cost of maintaining new software keeps climbing. And the more code a company produces, the more important it becomes to know what’s worth building in the first place.

Members of the Senior Executive AI Think Tank, a curated group of experts in machine learning, generative AI and enterprise AI applications, weigh in on what AI-driven productivity really looks like, where the biggest bottlenecks are emerging and how leaders can tell whether faster coding is translating into better business results.

“AI has accelerated code generation, while validation, security, testing and deployment still depend heavily on human judgment and processes that haven’t kept pace.”

– Steve Taplin, CEO of Sonatafy Technology, Inc.

SHARE IT

Measure the Value That Reaches Production

Steve Taplin, CEO of Sonatafy Technology, Inc., argues that the industry is entering a new era of software productivity—but one that can create a dangerous illusion of progress.

“AI has accelerated code generation, while validation, security, testing and deployment still depend heavily on human judgment and processes that haven’t kept pace.”

For executives, this requires changing what the organization rewards.

“The bottleneck is shifting from creation to verification,” he says, warning against celebrating lines of code, commits and pull requests as evidence of business impact.

Taplin recommends tracking lead time to production, change failure rates, escaped defects, security vulnerabilities and business outcomes. Leaders should fund, staff and govern engineering against those measures rather than raw activity.

“The question for every leader: When the evidence says AI output isn’t creating value, who has the authority to act on it?” he says.

Make Review and Rework Visible

Meghana Makhija, Senior Product Manager Tech at Amazon, approaches the challenge through the full software delivery process. She highlights an often-overlooked cost of AI adoption: the additional work required to turn generated code into an approved, dependable change.

“AI can improve software productivity, but faster code creation shifts the bottleneck toward verification and delivery,” Makhija says. “Leaders need to measure the full journey to reliable production outcomes.”

Rather than counting code reviews submitted by engineers, she recommends a different approach.

“Counting how many code reviews an engineer submits says little about productivity,” she says. “Track how many revision rounds each change needs for approval, time spent reviewing and reworking it, and how often it causes failures or incidents after deployment.”

These measures can reveal whether apparent speed gains are simply transferring effort to reviewers, testers and operations teams. Makhija recommends automating the metrics in dashboards, comparing similar work over time and including maintenance effort and security findings.

“Genuine productivity means useful software reaches customers sooner, with less total effort and without compromising reliability or security.”

Keep Quality Assurance Ahead of Output

Brock Murray, Co-Founder at seoplus+, sees the same principle applying beyond traditional software engineering teams: Scaling production is beneficial only when quality controls scale with it.

“More output doesn’t automatically mean greater productivity,” Murray says. “AI is making it easier to generate code at scale, but that creates new pressure on testing, security and deployment processes.”

For executives, that pressure is a reason to revisit quality assurance before expanding AI-assisted development across the organization. More generated code can increase the volume of changes that must be tested and approved, particularly when existing processes were designed around a slower development pace.

Murray recommends evaluating deployment frequency alongside defect rates, rework time and customer satisfaction. No single measure provides a complete picture: More frequent releases can indicate progress, but rising defects or declining customer satisfaction may reveal that speed has come at a cost.

“The real opportunity is using AI to improve efficiency across the entire development process while keeping people accountable for quality and business outcomes,” he says.

Count the Cost of Operating AI-Generated Code

Vivek Kumkar, Sr. GenAI Leader at Amazon Web Services (AWS), warns that the expense of creating software is only one part of its lifetime cost.

“AI made writing code cheap. It did nothing for the cost of owning it,” Kumkar says. “Every change still has to be run, patched and answered for in production long after it ships.”

He argues that verification alone does not capture the full bottleneck. Security patching, on-call demands and the maintenance burden of individual features can consume resources long after a change has passed review.

“On the services I run, writing the code is the cheap part and operating it is the expensive one,” he says. Shipping more software without changing how it is maintained can quietly expand an organization’s liabilities.

Kumkar recommends measuring operational load per unit of output. 

“If features ship faster while incidents, Sev-2s and patch latency hold flat or drop, the gain is real,” he says. “If output climbs and the pager climbs with it, you did not get more productive. You just moved the cost somewhere the velocity chart does not show yet.”

“Generative tools have turned code creation into a commodity, placing the real challenge squarely on verification and delivery.”

– Uttam Kumar, Engineering Manager at American Eagle Outfitters

SHARE IT

Prioritize Effectiveness Over Efficiency

Uttam Kumar, Engineering Manager at American Eagle Outfitters, emphasizes that the value of AI-assisted development must ultimately be judged by what customers and employees experience.

“Generative tools have turned code creation into a commodity, placing the real challenge squarely on verification and delivery,” he says. Drawing on Peter Drucker’s distinction between efficiency and effectiveness, Kumar urges leaders to focus on whether teams are building the right things, not simply building things faster.

“Leaders need to evaluate real productivity by examining production stability, security patch turnaround times and the predictability of their release cycles,” Kumar says. “When output spikes without a corresponding improvement in system resilience or user experience, you are only amplifying operational risk.”

For a retailer, the consequences are tangible. A feature that reaches store associates or customers quickly but disrupts a workflow, introduces a vulnerability or requires repeated fixes has not necessarily delivered value.

“True progress is measured by how seamlessly secure, thoroughly validated features reach the hands of store associates and customers without introducing new vulnerabilities.”

Build Trust Into the Development Lifecycle

Aishwarya Shah, Independent Researcher, argues that AI is exposing a constraint that software organizations have long struggled to address: trust in the systems they build and accountability for how those systems behave.

“AI is not eliminating the software productivity bottleneck; it is exposing where the real bottleneck has always existed: trust, verification and operational accountability,” Shah says. “Generating more code is not the same as delivering more business value.”

Her recommendation is to evaluate the end-to-end process, including deployment time, escaped defects, security vulnerabilities, recovery performance and measurable customer outcomes. Developer velocity, considered alone, cannot show whether the organization is producing dependable software.

“Companies will gain an advantage from integrating AI-generated development with automated testing, secure architecture and continuous governance,” Shah says.

That means verification should not be treated as a final checkpoint after code generation. Automated tests, security controls and governance requirements need to be incorporated into the workflow so teams can move quickly without bypassing essential safeguards.

Redesign the Entire Delivery Lifecycle

Venkata Kondepati, Manager of Data Architecture and Engineering at Ascentt, views AI as an opportunity to rethink the software delivery lifecycle rather than simply accelerate individual coding tasks.

“AI is creating a new era of software productivity, but the bottleneck is shifting from writing code to validating, securing and delivering it reliably,” Kondepati says. “More code does not automatically mean more business value.”

He recommends tracking time from idea to production, deployment frequency, change failure rates, security vulnerabilities, defect escape rates and customer impact. These measures connect the development process to the reliability and business outcomes that executives need to protect.

“If code generation accelerates while review queues, technical debt and incidents increase, productivity gains may be an illusion,” he says.

Kondepati’s prescription is to redesign the delivery lifecycle around AI, including automated testing, security, governance and deployment. This requires examining how work moves between teams and where manual approvals, fragile infrastructure or accumulated technical debt delay releases.

“The winning metric isn’t how much code AI produces, but how quickly organizations can deliver trusted software that creates measurable value,” he says.

“Count gains only when faster delivery or lower total cost persists without degrading reliability, security or maintainability.”

– Dr. Aditya Vikram Kashyap, Vice President of Firmwide Innovation at Morgan Stanley

SHARE IT

Prove Gains With Comparable Evidence

Dr. Aditya Vikram Kashyap, Vice President of Firmwide Innovation at Morgan Stanley, approaches AI productivity as an evidence and investment question. His focus is whether the organization can demonstrate that faster delivery produces more customer value at an acceptable level of risk.

“AI can improve productivity while shifting the constraint to verification and delivery,” Kashyap says. Citing DORA’s 2025 research, he notes that AI adoption can improve some aspects of development while creating challenges for delivery stability.

His proposal spans idea-to-production lead time, review queues, rework, escaped defects, security findings, change failure rates and recovery time. Crucially, it also includes engineering, review, tooling, infrastructure and incident costs.

“Compare similar work against a baseline or controlled rollout, adjusting for complexity and experience,” he says. That comparison helps leaders distinguish AI’s contribution from differences in project scope, team skill or existing workflow efficiency.

Kashyap also recommends investing in automated tests, policy checks and gradual releases. Those controls allow teams to capture potential speed gains while limiting the impact of defects.

“Count gains only when faster delivery or lower total cost persists without degrading reliability, security or maintainability,” he says. This approach gives executives a defensible basis for investment decisions rather than relying on anecdotal success stories or activity metrics.

Watch the Queue Between Completion and Release

Divya Parekh, Founder of executive coaching brand DivyaParekh.com, draws on her experience in regulated biopharmaceutical manufacturing to explain why completed work should not automatically count as productive output.

“In pharmaceutical manufacturing, a batch isn’t finished when it comes off the line,” Parekh says. “It sits in quarantine until quality releases it, and nobody counts it as output before then.”

She sees a direct parallel in AI-assisted software development. When coding accelerates but review capacity remains unchanged, the additional work accumulates rather than delivering value.

“When coding speeds up and review doesn’t, the code doesn’t disappear. It waits,” she says. Pull requests age in queues, test environments become congested and releases may be bundled into larger, riskier changes.

Parekh recommends measuring the volume of completed but unreleased work and the proportion of changes that reach production successfully the first time. Executives should also monitor rework and rollbacks after deployment.

“Is the pile of finished but unreleased work growing?” she asks. “What share of changes ship right the first time, without rework or a rollback a week later?”

The manufacturing analogy provides a useful management principle: Work awaiting validation is inventory, not delivered value. Increasing production capacity without increasing the capacity to verify and release work can make the organization look busier while slowing the flow of useful software.

Expand the Capacity to Make Better Decisions

As the cost of coding falls, Andre Shojaie, Founder of HumanLearn, argues companies can pursue more software ideas—but they do not automatically become better at choosing which ideas deserve investment.

“AI is not making software development cheaper; it is making software ambition cheaper,” Shojaie says. “As the cost of coding falls, organizations don’t build the same amount of software faster, they attempt far more than they could before.”

That expansion changes the nature of the bottleneck. Prioritization, governance, risk management, architecture and product strategy can become more important constraints than implementation capacity.

“The constraint is no longer writing code, but deciding what is worth building,” he says. Many companies already have more opportunities than they can effectively evaluate and execute.

Shojaie recommends measuring opportunity throughput: Are strategic ideas reaching customers faster? Are decisions made more quickly? Is experimentation increasing? Are teams learning and adapting faster?

These indicators connect AI investment to the organization’s ability to identify and capture value, rather than simply increasing the volume of development work.

“If AI expands an organization’s capacity to identify, prioritize and capture value, then the real revolution is not in software engineering, but in decision-making,” he says. Executives should therefore improve the systems that select, govern and evaluate projects alongside the systems that build them.

10 Strategies for Measuring AI Productivity

• Measure delivered value, not generated code. Track production lead time, escaped defects, security vulnerabilities and business outcomes to determine whether AI-generated work creates measurable value.
• Expose the cost of review and rework. Instrument review queues, revision rounds, approval time and post-deployment failures so downstream costs appear alongside development speed.
• Scale quality assurance with production. Monitor defect rates, customer satisfaction and deployment reliability as AI increases the volume of code entering the delivery pipeline.
• Include lifetime operating costs. Evaluate operational load per feature, incidents, patch latency, infrastructure expenses and maintenance effort to capture the full cost of ownership.
• Prioritize effectiveness over activity. Assess whether releases improve user experience, system resilience and the predictability of business-critical workflows.
• Make verification continuous. Integrate automated testing, secure architecture, policy checks and governance into development instead of relying solely on final-stage reviews.
• Redesign the delivery system. Address technical debt, review bottlenecks and fragmented processes rather than treating AI as an isolated coding assistant.
• Use controlled comparisons. Establish baselines, compare similar work and include all relevant costs before attributing performance improvements to AI.
• Treat unreleased work as inventory. Monitor the age and volume of changes awaiting testing, approval or deployment, alongside first-time release success and rollback rates.
• Improve opportunity selection. Measure how quickly teams make decisions, test strategic ideas and deliver customer value, not just how many projects they can start.

The Real Measure of AI Productivity Is What Gets Delivered

The temptation to equate more code with more progress is understandable, especially when AI makes development faster and the results are easy to see. But the real test comes later: when a release reaches customers, when a security flaw needs fixing or when a team has to maintain software long after the original developers have moved on. Those are the moments when the difference between producing more and accomplishing more becomes clear.

For executives, the challenge is no longer just figuring out how to put AI to work. It’s deciding what deserves to be built, making sure teams can validate and deliver it safely, and holding the organization accountable for the results. Companies that get this right will be the ones that know which work matters, get it into customers’ hands with confidence and can show that the investment made a difference.