A fast browse of LinkedIn, DevTok, and X would lead you to imagine that nearly each developer has jumped on board the vibe coding hype practice with full gusto. And whereas it’s not that far-fetched, with 84% of builders confirming they’re presently utilizing (or planning to make use of) AI coding instruments of their every day workflows, a full give up to vibe coding autonomous brokers remains to be uncommon. Stack Overflow’s 2025 AI Survey revealed that the majority respondents (72%) usually are not (but) vibe coding. Nonetheless, adoption is trending upwards, and AI is presently producing 41% of all code, for higher or worse.
Instruments like Cursor and Windsurf signify the newest technology of AI coding assistants, every with a strong autonomous mode that may make selections independently based mostly on preset parameters. The velocity and productiveness beneficial properties are plain, however a worrying development is rising: many of those instruments are being deployed in enterprise environments, and these groups usually are not outfitted to handle the inherent safety points related to their use. Human governance is paramount, and too few safety leaders are making an effort to modernize their safety applications to adequately protect themselves from the chance of AI-generated code.
If the tech stack lacks instruments that oversee not solely developer safety proficiency, but additionally the trustworthiness of permitted AI coding companions every developer makes use of, then it’s seemingly that efforts to uplift the general safety program and the builders working inside it will likely be wanting the suitable knowledge insights to impact change.
AI and human governance must be a precedence
The drawing card of agentic fashions is their capability to work autonomously and make selections independently, and these being embedded into enterprise environments at scale with out acceptable human governance is inevitably going to introduce safety points that aren’t notably seen or simple to cease.
Lengthy-standing safety issues like delicate knowledge publicity and inadequate logging and monitoring stay, and rising threats like reminiscence poisoning and gear poisoning usually are not points to take flippantly. CISOs should take steps to scale back developer danger, and supply steady studying and expertise verification inside their safety applications in an effort to safely implement the assistance of agentic AI brokers.
Highly effective benchmarking lights your developer’s path
It’s very troublesome to make impactful, constructive enhancements to a safety program based mostly solely on anecdotal accounts, restricted suggestions, and different knowledge factors which are extra subjective in nature. Most of these knowledge, whereas useful in correcting extra obvious faults (comparable to a selected device repeatedly failing or personnel time being wasted on a low-value and irritating job), will do little to uplift this system to a brand new stage. Sadly, the “folks” a part of an enterprise safety (or, certainly, Safe by Design) initiative is notoriously tough to measure, and too usually uncared for as a bit of the puzzle that should be a precedence to unravel.
That is the place governance instruments that ship knowledge factors on particular person developer safety proficiency – categorized by language, framework and even trade – will be the distinction between executing one more flat coaching and observability train, versus correct developer danger administration, the place the instruments are working to gather the insights wanted to plug information gaps, filter security-proficient devs to probably the most delicate initiatives, and importantly, monitor and approve the instruments they use of their day, comparable to AI coding companions.
Evaluation of agentic AI coding instruments and LLMs
Three years on, we will confidently conclude that not all AI coding instruments are created equal. Extra research are rising that help in differentiating the strengths and weaknesses of every mannequin, for a wide range of purposes. Sonar’s current research on the coding personalities of every mannequin was fairly eye-opening, revealing the completely different traits of fashions like Claude Sonnet 4, OpenCoder-8B, Llama 3.2 90B, GPT-4o, and Claude Sonnet 3.7, with perception into how their particular person approaches to coding have an effect on code high quality and, subsequently, related safety danger. Semgrep’s deep dive into the capabilities of AI coding brokers for detecting vulnerabilities additionally yielded combined outcomes, with findings that usually demonstrated {that a} security-focused immediate can already establish actual vulnerabilities in actual purposes. Nevertheless, relying on the vulnerability class, a excessive quantity of false positives created noisy, much less precious outcomes.
Our personal distinctive benchmarking knowledge helps a lot of Semgrep’s findings. We have been capable of present that the most effective LLMs carry out comparably with proficient folks at a variety of restricted safe coding duties. Nevertheless, there’s a vital drop in consistency amongst LLMs throughout completely different levels of duties, languages, and vulnerability classes. Usually, high builders with safety proficiency outperform all LLMs, whereas common builders don’t.
With research like this in thoughts, we should not lose sight of what we as an trade are permitting into our codebases: AI coding brokers have growing autonomy, oversight and basic use, they usually should be handled like every other human with their arms on the instruments. This, in impact, requires cautious administration by way of assessing their safety proficiency, entry stage, commits and errors with the identical fervor because the human working them, with no exceptions. How reliable is the output of the device, and the way safety proficient is its operator?
If safety leaders can’t reply these questions and plan accordingly, the assault floor will proceed to develop by the day. If you happen to don’t know the place the code is coming from, ensure that it’s not moving into any repository, with no exceptions.
