Why AI coding agents aren’t production-ready: Brittle context windows, broken refactors, missing operational awareness
Remember this Quora comment (which also became a meme)?
(Source: Quora)
In the Stack Overflow era of the pre-large language model (LLM), the challenge was great which one code snippets to effectively adopt and adapt. While code generation has become trivially easy, the bigger challenge lies in reliably identifying and integrating high-quality, enterprise code into production environments.
This article examines the practical pitfalls and limitations observed when engineers use modern coding tools for real enterprise work, addressing the more complex issues surrounding integration, scalability, accessibility, evolving security practices, data privacy, and maintainability in live operating environments. We hope to balance the hype and provide a more technically sound view of the capabilities of AI coding tools.
Limited domain understanding and service limits
AI agents struggle significantly with designing scalable systems due to the explosion of choices and a crucial lack of business-specific context. To broadly describe the problem, large enterprise codebases and monorepos are often too large for agents to learn from directly, and critical knowledge can often be fragmented across internal documentation and individual expertise.
More specifically, many popular encoders face service limits that hinder their effectiveness in large-scale environments. Indexing functions may fail or degrade for repositories with more than 2,500 files, or due to memory limitations. Additionally, files larger than 500 KB are often excluded from indexing/searching, impacting established products with decades-old, larger code files (although newer projects may be less likely to experience this).
For complex tasks that involve extensive file contexts or refactoring, developers are expected to provide the relevant files while also explicitly defining the refactoring procedure and surrounding build/command sequences to validate the implementation without introducing feature regressions.
Lack of hardware context and usage
AI agents have shown a critical lack of awareness regarding OS machine, command line, and environment installations (conda/venv). This shortcoming can lead to frustrating experiences, such as the agent trying to run Linux commands on PowerShell, which can consistently result in “unrecognized command” errors. Additionally, agents often exhibit inconsistent “wait tolerance” when reading command output, prematurely declaring an inability to read the results (and resorting to retry/skipping) before a command has even completed, especially on slower machines.
This isn’t just about nitpick functions; rather, the devil is in these practical details. These gaps in the experience manifest as real friction points and require constant human vigilance to monitor the agent’s activity in real time. Otherwise, the agent ignores the initial information about the tool call and quits prematurely, or continues with a half-baked solution that involves undoing some/all changes, reactivating prompts, and wasting tokens. There’s no guarantee that you’ll submit a prompt on Friday evening and expect the code updates to run at the Monday morning check.
Hallucinations over repeated actions
Working with AI coders often involves a long-term challenge of hallucinations, or incorrect or incomplete pieces of information (such as small code snippets) within a larger set of changes that can be expected to be resolved by a developer with trivial to low effort. What becomes particularly problematic, however, is when improper behavior occurs repeated within a single thread, forcing users to start a new thread and re-provide all context, or manually intervene to ‘unblock’ the agent.
For example, while setting up some Python function code, an agent tasked with implementing complex changes to production readiness came across a file (see below) with special characters (brackets, period, star). These characters are very common to indicate in computer science software versions.
(Image created manually with standard code. Source: Microsoft Learn And Edit application host file (host.json) in the Azure portal)
The agent incorrectly flagged this as an unsafe or malicious value, halting the entire generation process. This misidentification of an enemy attack occurred four to five times, despite several attempts to restart or continue the change. This version format is actually standard and present in a Python HTTP trigger code template. The only successful solution was to instruct the officer not read the file and instead ask it to simply specify the configuration you want and assure it that the developer will manually add it to that file, confirm it and ask it to continue with the remaining code changes.
The inability to exit a repeatedly faulty agent execution loop within the same thread highlights a practical limitation that significantly wastes development time. Essentially, developers now tend to spend time debugging/refining AI-generated code instead of Stack Overflow code snippets or their own code snippets.
Lack of coding practices at the enterprise level
Security best practices: Encryption agents often use less secure authentication methods such as key-based authentication (client secrets) instead of modern identity-based solutions (such as Entra ID or federated credentials). This oversight can introduce significant vulnerabilities and increase maintenance overhead, as key management and rotation are complex tasks that are increasingly limited in enterprise environments.
Outdated SDKs and reinventing the wheel: Agents may not consistently use the latest SDK methods, but instead generate more elaborate and difficult to maintain deployments. As an example of the Azure Function example, agents executed code using the pre-existing v1 SDK for read/write operations, instead of the much cleaner and more maintainable v2 SDK code. Developers should research the latest best practices online to get a mental map of the dependencies and expected implementation that will ensure long-term maintenance and reduce upcoming technology migration efforts.
Limited intent recognition and repetitive code: Even for smaller, modular tasks (which are typically encouraged to minimize hallucinations or tracking downtime), such as expanding an existing function definition, agents can follow the instruction literal and produce logic that turns out to be virtually repetitive, without anticipating the upcoming or not articulated developer’s needs. That is, in these modular tasks, the agent may not automatically identify and refactor similar logic into shared functions or improve class definitions, leading to technical debt and harder-to-manage codebases, especially with vibe coding or lazy developers.
Simply put, those viral YouTube videos that demonstrate rapid zero-to-one app development from a one-sentence prompt simply fail to capture the nuanced challenges of production software, where security, scalability, maintainability, and future-proof design architectures are of paramount importance.
Confirming bias alignment
Confirmation bias is a major problem because LLMs often confirm users’ premises even when the user expresses doubt and asks the agent to refine their understanding or propose alternative ideas. This trend of models conforming to what they think the user wants to hear leads to reduced overall output quality, especially for more objective/technical tasks such as coding.
There is extensive literature to suggest that if a model starts outputting a claim like “You’re absolutely right!”, the rest of the output tokens tend to justify this claim.
Constant need to babysit
Despite the appeal of autonomous coding, the reality of AI agents in enterprise development often requires constant human vigilance. Occurrences such as an agent attempting to run Linux commands on PowerShell, false positive security flags, or introducing inaccuracies for domain-specific reasons highlight critical gaps; developers simply can’t step away. Instead, they must continuously monitor the reasoning process and understand code additions from multiple files to avoid wasting time with substandard answers.
The worst possible experience with agents is when a developer accepts code updates from multiple bug-ridden files, and then wastes time debugging because of how “pretty” the code apparently looks. This can even give rise to the sunk cost fallacy of hoping that the code will work after just a few fixes, especially when the updates involve multiple files in a complex/unknown codebase with connections to multiple independent services.
It’s like working with a 10-year-old prodigy who has memorized enough knowledge and even addresses every bit of user intent, but prioritizes demonstrating that knowledge over solving the actual problem, and lacks the foresight necessary for success in real-world situations.
This ‘babysitting’ requirement, coupled with the frustrating repetition of hallucinations, means that the time spent debugging AI-generated code can overshadow the time savings expected from using agents. Needless to say, developers in large companies need to be very purposeful and strategic when navigating modern agentic tools and use cases.
Conclusion
There’s no doubt that AI coding tools have been nothing short of revolutionary, accelerating prototyping, automating basic coding, and transforming the way developers build. The real challenge now isn’t generating code, but knowing what to ship, how to secure it, and where to scale it. Smart teams learn to filter the hype, deploy agents strategically, and double down on their technical judgment.
As CEO of GitHub Thomas Dohmke recently noted this: The most advanced developers have “moved from writing code to designing and verifying the implementation work performed by AI agents.” In the age of agents, success does not belong to those who can code, but to those who can design systems that last.
Rahul Raja is a software engineer at LinkedIn.
Advitya Gemawat is a machine learning (ML) engineer at Microsoft.
Editor’s Note: The opinions expressed in this article are the personal opinions of the authors and do not reflect the views of their employers.
