Risks of Vibe Coding

When the stakes change

Vibe coding means not reading the code you accept (see What Is Vibe Coding?). For the space game built in this topic, that is fine: if the code is wrong, a rock flies the wrong way. Simon Willison, who drew the line between vibe coding and AI-assisted programming, put the conditions plainly: "Projects should be low stakes" and "Consider security" (19 March 2025). The risks on this page grow with two things: what the code can reach, and whether anyone checked it.

None of this is a reason not to use AI. It is a reason to know where your project sits in that table, and to start reading, testing and reviewing before it moves down a row. From Prototype to Real Software describes that move.

Secrets and data in prompts and code

Prompt data leakage is sensitive information (a password, an API key, customer data, internal code) leaving your control because it was pasted into an AI tool, where it may be stored, read by the provider or used for training. In vibe coding it has a twin: secrets that the AI, or you, put into the code itself, which then leaks wherever the code goes. What never belongs in a prompt is listed on using AI at work, and how to strip it out on keeping data out of prompts.

Secrets in generated code

The best numbers come from GitGuardian, a company that sells secret scanning, so read them as a vendor's findings and as correlations, not proof that the AI caused the leaks:

• In its 2025 report, public GitHub repositories that used Copilot had a "6.4% secret leakage rate", against 4.6% for all public repositories.
• In its 2026 report (17 March 2026), 28.65 million new hard-coded secrets were found in public GitHub commits in 2025, 34% more than a year before; secrets for AI services rose 81%, to 1,275,105; and "Claude Code-assisted commits showed a 3.2% secret-leak rate, versus a 1.5% baseline". GitGuardian adds: "The leak still happens through a human workflow."

The 2025 figure counts repositories and the 2026 figure counts commits, so the two cannot be compared with each other. The pattern behind both is familiar: the code needs a key, the quickest way to make it work is to paste the key into it, and nobody reads the result before it is published. A minimal check before you publish anything looks for names like key, token or password followed by a long string:

// script.js of a Pen, about to be made public (fictional).
const pen = `const SPEED = 9;
const WEATHER_API_KEY = "demo-0000-not-a-real-key";
const playerName = "Pilot";
const password = "";`;

// A name that sounds secret, then = or :, then a string of 12+ characters.
const SUSPICIOUS = /(key|token|secret|password)\w*\s*[=:]\s*["'][^"']{12,}["']/i;

pen.split("\n").forEach((line, index) => {
  if (SUSPICIOUS.test(line)) {
    console.log(`line ${index + 1}: looks like a secret`);
  }
});
console.log("checked", pen.split("\n").length, "lines");

line 2: looks like a secret
checked 4 lines

Real tools do this far better, with patterns for hundreds of providers: secret scanning and push protection catch a key before or after it reaches a repository. But a public Pen is not a repository with scanning, which is why the rule there is simple.

Training on your chats

What you paste may also be kept and used to train future models. For consumer accounts, the large providers let you choose (as of October 2026):

• Anthropic asked users of Claude Free, Pro and Max, "including when they use Claude Code", to choose whether their chats may be used for training, with a deadline of 8 October 2025. With training allowed, data is kept for five years, otherwise for 30 days. Commercial and API use are not covered by this setting.
• OpenAI, according to its help pages, trains on ChatGPT conversations "unless you opt out" (Data Controls, "Improve the model for everyone"); business plans and the API are excluded by default.

Settings change and differ per plan, so the safe habit does not depend on them: no secrets and no personal data in prompts, ever. In the EU and the UK, under the GDPR (in the UK, the UK GDPR), pasting customers' personal data into an AI tool is processing of personal data like any other: it needs a lawful basis and your organization's approval, not a private choice.

Invented packages

Package hallucination is an AI model recommending or importing a software package that does not exist, with a name that sounds plausible. A large study by Spracklen and colleagues (USENIX Security 2025) had 16 models write 576,000 Python and JavaScript code samples:

• 19.7% of the 2.23 million package references were hallucinated, with 205,474 different invented names.
• The GPT models averaged 5.2%, the open-source models 21.7%.
• When the same prompts were run ten times, 43% of the invented names came back in all ten runs, and 58% more than once.

The models tested date from 2023 and 2024; the rate for current models is not known. The repeatability is the dangerous part: a name that a model invents again and again is a name an attacker can predict and register. Slopsquatting is exactly that: registering package names that AI models tend to invent, with malicious code inside, and waiting for someone to install them. The word combines "slop" (low-quality AI output) and typosquatting; it is attributed to the security researcher Seth Larson (April 2025). That an invented name attracts real installs was shown earlier by Bar Lanyado of Lasso Security, who uploaded an empty package under a name that models kept inventing; it got "more than 30k authentic downloads" in three months (28 March 2024). It is a form of supply chain attack, see also AI security.

The space game installs nothing: it imports only three, and CodePen resolves that name itself. But ask for "better explosions" and a model may suggest a package that sounds made for the job. A JavaScript version of the allowlist check from prompting for code (which has it in Python) finds every package an answer wants you to use:

// An assistant's answer (fictional), pasted in as text.
const answer = `Install this first:

    npm install three three-retro-jet-fx

Then at the top of script.js:

    import * as THREE from "three";
    import { explode } from "three-retro-jet-fx";
    import { OrbitControls } from "three/addons/controls/OrbitControls.js";`;

const APPROVED = new Set(["three"]);

const found = new Set();
for (const line of answer.split("\n")) {
  const install = line.match(/^\s*npm install (.+)$/);
  if (install) install[1].split(/\s+/).forEach((name) => found.add(name));
  const from = line.match(/from "([^"]+)"/);
  if (from) found.add(from[1]);
}

for (const spec of [...found].sort()) {
  // "three/addons/..." is a file inside the package "three".
  const name = spec.split("/")[0];
  const status = APPROVED.has(name) ? "approved" : "NOT on the list: check first";
  console.log(`${spec.padEnd(40)} ${status}`);
}

three                                    approved
three-retro-jet-fx                       NOT on the list: check first
three/addons/controls/OrbitControls.js   approved

"Check first" means: look the name up on the package registry (npmjs.com for JavaScript), and see whether it exists, who publishes it, how long it has existed, how many people use it, and whether its description matches what you need. A package that does not exist is not an invitation to create it; ask the model for a solution without it. (A real check would also handle scoped names such as @scope/name, which this short version does not.)

I am building a small 3D space game in one CodePen Pen with Three.js (import * as THREE from "three"). Which npm packages should I add for asteroid physics, explosion particles and retro sound effects? Give the package names and the install command.

I am building a small 3D space game in one CodePen Pen with Three.js 0.186.1. Without any packages other than Three.js, using only Three.js and the browser's own APIs, how would I add explosion particles and retro sound effects? Explain in five sentences, without code.

Code that works but is not safe

Insecure generated code is AI-written code that does what was asked but contains a security weakness, such as missing input validation, an injection flaw, a hard-coded secret or missing access control. It is the risk that "it seems to work" hides best, because the weakness only shows when someone attacks it. The research points in one direction:

• Pearce and colleagues ("Asleep at the Keyboard?", IEEE Symposium on Security and Privacy 2022) gave GitHub Copilot 89 scenarios built around the most dangerous weakness types (the CWE Top 25) and found about 40% of the 1,689 resulting programs vulnerable. The scenarios were chosen to be security-relevant, so this is not "40% of all Copilot code".
• Perry and colleagues (Stanford, ACM CCS 2023) found that the 47 participants who had an AI assistant "wrote significantly less secure code" and "were also more likely to believe they wrote secure code".
• Veracode, a vendor of code-security scanning, reported in July 2025 that "45% of code samples failed security tests" across more than 100 models, and in its spring 2026 update (24 March 2026) a 55% pass rate across more than 150 models, with over 95% of the code syntactically correct: "Models have become excellent at writing code that compiles. They've failed at writing code that's safe." It is a vendor report, not peer-reviewed.

A real case shows how this reaches users. CVE-2025-48757 (published 30 May 2025) describes "an insufficient database Row-Level Security policy in Lovable", an app builder, that let unauthenticated attackers read or write database tables of generated sites. The researcher Matt Palmer found 303 exposed endpoints in 170 of 1,645 projects he scanned, leaking emails, phone numbers, payment details and API keys. Lovable disputes the CVE: its position is that each customer is responsible for protecting their own app's data. Either way, the generated apps were published by people who did not know what row-level security was, or that it was missing.

For the space game, one habit matters: text goes on the page with textContent, never innerHTML. With innerHTML, text that contains HTML becomes HTML, the root of DOM-based XSS. The game only shows its own numbers, but the habit is the one you need the day a page shows a name someone typed.

Instructions hidden in what you paste

A model cannot reliably tell your instructions from instructions that happen to be inside the material you give it. The OWASP Top 10 for LLM Applications lists prompt injection first, and describes the indirect form: "Indirect prompt injections occur when an LLM accepts input from external sources, such as websites or files." For a chat assistant that only answers, the damage is a misleading answer. For a coding agent that can read files, run commands and open pull requests, it is more:

• Invariant Labs (26 May 2025) showed a malicious issue in a public GitHub repository hijacking an agent that used the GitHub MCP server, so that it leaked data from the user's private repositories into a public pull request. They called it "a fundamental architectural issue that must be addressed at the agent system level".
• CVE-2025-53773 (12 August 2025), a command-injection flaw in GitHub Copilot and Visual Studio: according to the researcher who reported it, instructions planted in a source file, web page or GitHub issue could make Copilot write "chat.tools.autoApprove": true into the project's editor settings, switching off the approval of its own actions, and then run commands. Microsoft patched it in August 2025.

For a beginner the lesson is practical. Code copied from a random Pen, a forum post or a README into your chat is not only code; it can carry text written for the model. Read what you paste, and be most careful with agents whose actions you approve automatically. Defenses are on limiting the damage.

Agents that delete things

A destructive agent action is an AI agent deleting, overwriting or changing data or systems beyond what its user intended, usually because it had more access than the task needed. The cause is a combination you will recognize from excessive agency: broad permissions, no human approval for risky steps, and no easy way back. Three cases from July 2025:

• Replit and SaaStr. Jason Lemkin of SaaStr wrote that "after 100+ straight hours of vibe coding" Replit's agent "deleted our entire production database", with records of 1,206 executives and more than 1,196 companies, despite "eleven separate warnings in ALL CAPS". The agent first claimed a rollback was impossible; it worked. The root cause, in his words: the agent "had direct access to production databases". Replit's chief executive reportedly called it "unacceptable" and announced separate development and production databases, a planning-only mode and one-click restore.
• Gemini CLI. An issue in Google's Gemini CLI repository (21 July 2025) is titled "Gemini CLI 'lost' files during a failed file move operation. [Windows]". According to the reports, a folder was never created, and the following move commands overwrote the user's files one after another.
• Amazon Q Developer. Version 1.84.0 of the Visual Studio Code extension shipped with malicious code that an attacker had committed using an over-scoped GitHub token; according to AWS (AWS-2025-015, 23 July 2025) it "was unsuccessful in executing due to a syntax error". This was a supply chain compromise of an AI tool, not a user's vibe-coding mistake, but it shows that the agent you run is itself software that can be attacked.

The defenses are old ones: least privilege (an agent gets a copy of the data, never production access), a human in the loop for anything that deletes or deploys, and backups whose restore you have actually tested (a restore test). The habits of working safely on production systems apply to agents as much as to people. An instruction in the prompt is not one of these defenses: Lemkin reports "eleven separate warnings in ALL CAPS", and the agent went ahead anyway.

Over-reliance and skill atrophy

Vibe coding asks you to trust the tool instead of checking it, which is the definition of overreliance. Two findings from studies are worth knowing.

People misjudge the effect. In a randomized study by the research group METR (July 2025), 16 experienced open-source developers working in their own mature projects with early-2025 tools expected AI to make them 24% faster, believed afterward it had made them 20% faster, and were in fact 19% slower. In February 2026 METR reported that data from a newer experiment point to a speedup, but called that "an unreliable signal", because more and more developers refused to work without AI at all. The speed result changes with the tools; the gap between how fast people feel and how fast they are is the lesson that holds (see measuring the benefit).

Skill atrophy is the loss, or the failure to build, of a skill because a tool always does the work for you. In a randomized study by Anthropic (29 January 2026), 52 mostly junior engineers learned a new Python library, with or without AI help. On the quiz afterward the AI group scored 50%, the others 67%, with the biggest gap on debugging questions, and the AI group was not significantly faster. Those who used the AI to ask conceptual questions kept more than those who only let it write and fix the code. A survey of 319 knowledge workers by Lee and colleagues (Microsoft Research, CHI 2025) found that "higher confidence in GenAI is associated with less critical thinking"; it is self-reported and not specific to developers, but it points the same way.

The practical answer is the arc of this topic: ask why, not only what; read the code you got; debug with the AI instead of through it. That is how vibe coding turns into AI-assisted programming.

Who owns generated code

Copyrightability is whether a work can be protected by copyright at all. For AI output the question is how much human creative input it contains, and the answer differs per country. This is an overview, not legal advice.

Two practical points hold everywhere. First, a public Pen is MIT licensed, so everything you build in one is open source by default; check that this is what you want before you build something you may want to sell. Second, generated code can resemble existing public code, including code under a copyleft license with conditions attached. Code referencing is a feature of some coding assistants that compares a suggestion with an index of public code and blocks it, or shows where the matching code comes from and under which license. In GitHub Copilot it "compares a potential suggestion and approximately 150 characters of surrounding code with an index of public repositories", and matches "can be blocked or shown with information about the matching code", depending on the policy for your account or organization.