Vibe Coding vs. AI-Assisted Engineering: A Guardrails Cheatsheet

On February 2, 2025, Andrej Karpathy posted a description of a new way he was writing software: "There's a new kind of coding I call 'vibe coding', where you fully give in to the vibes, embrace exponentials, and forget that the code even exists." He accepted every change without reading the diffs, pasted error messages back without comment, and noted that the code grew beyond his usual comprehension. He also said where it belonged: "It's not too bad for throwaway weekend projects."

Within weeks the term was everywhere, and it started to mean "any coding with AI". That is a problem, because the two things need opposite habits. Vibe coding, as Simon Willison put it, is building software with an LLM without reviewing the code it writes. AI-assisted engineering is using the same tools while still owning every line that ships.

Meanwhile, the tools moved from autocomplete to agents. In the last three months alone, GitHub Copilot, Cursor, Anthropic and OpenAI all shipped agents that edit files across a repo and run commands in your terminal. That makes guardrails more important, not less. This cheatsheet is the line I draw between the two modes and the guardrails I use when the code matters.

1. Two modes, one set of tools

Willison's rule for the second column is the one I use: do not commit code you could not explain exactly to somebody else.

2. Where vibe coding is fine, and where it is not

The common failure is not vibe coding itself, it is the promotion of a vibe-coded prototype to production without changing modes. If a prototype survives, treat it as a first draft: read it, test it, and rewrite what you cannot explain.

3. The guardrails

Tests first

Tests are the contract the AI has to satisfy, and the only scalable way to check code you did not write.

• Write or review the tests before the implementation. Anthropic's own Claude Code best practices (April 2025) recommend this: have the agent write tests from expected input and output pairs, confirm they fail, commit them, then ask it to implement until they pass without modifying the tests.
• Watch for test tampering. Agents under pressure to make tests pass will sometimes weaken assertions, add skips or special-case inputs. Review test diffs more carefully than code diffs.
• Prefer behavior tests over mocks. A test that mocks everything the AI wrote proves nothing.

Small diffs

• One task per prompt, one concern per commit. "Add pagination to the orders endpoint" beats "improve the orders module".
• Plan before code. Ask the agent to read the relevant files and propose a plan, and do not let it write code until you agree.
• Commit often. A clean git state is your undo button. Agents make large, confident, wrong changes; small commits make them cheap to discard.
• Cap the diff size you review. If a change is too big to review properly, it is too big to merge. Ask for it in pieces.

DORA's 2024 report gives a reason to take this seriously: higher AI adoption was associated with lower delivery stability, and DORA's leading hypothesis is that AI makes change lists bigger.

Review every line

• Read it as if a new colleague wrote it, someone fast and confident who has never seen your system.
• Check what LLMs typically get wrong: invented APIs or parameters, outdated library versions, off-by-one and timezone bugs, swallowed exceptions, missing authorization checks, inconsistent error handling, duplicated helper functions that already exist in your codebase.
• Ask the AI to explain its own diff, then verify the explanation against the code. Mismatches are where the bugs live.
• Do not let a second AI be the only reviewer. AI review is a useful extra pass, not a substitute for a human who understands the system.

There is research behind the caution: in a Stanford user study (Perry et al., CCS 2023), participants with access to an AI assistant wrote significantly less secure code than those without, and were more likely to believe their code was secure.

Security

Slopsquatting is the newest of these. The term was coined by Seth Larson, the Python Software Foundation's security developer-in-residence, and popularized in April 2025. It names an attack made possible by a 2024 study (Spracklen et al., accepted to USENIX Security 2025): across 16 code-generating models and 576,000 code samples, 19.7% of recommended packages did not exist (5.2% for commercial models, 21.7% for open-source ones), producing 205,474 unique hallucinated package names. The names are often repeated across runs, which makes them predictable enough for an attacker to register.

A cheap guard: before adding a dependency the AI suggested, check that it exists and is not brand new.

"""Flag dependencies that do not exist on PyPI or are suspiciously new."""
import json
import sys
import urllib.error
import urllib.request
from datetime import datetime, timezone

MIN_AGE_DAYS = 90


def check(name: str) -> str | None:
    try:
        with urllib.request.urlopen(f"https://pypi.org/pypi/{name}/json", timeout=10) as r:
            data = json.load(r)
    except urllib.error.HTTPError as e:
        return f"{name}: not on PyPI (HTTP {e.code}). Hallucinated?"
    uploads = [
        datetime.fromisoformat(f["upload_time_iso_8601"].replace("Z", "+00:00"))
        for files in data["releases"].values()
        for f in files
    ]
    if not uploads:
        return f"{name}: exists but has no released files"
    age = (datetime.now(timezone.utc) - min(uploads)).days
    if age < MIN_AGE_DAYS:
        return f"{name}: first release only {age} days ago. Review before installing"
    return None


if __name__ == "__main__":
    problems = [p for p in map(check, sys.argv[1:]) if p]
    print("\n".join(problems) or "All packages look established.")
    sys.exit(1 if problems else 0)

python check_packages.py requests httpx some-ai-suggested-pkg exits non-zero if anything looks wrong. Existence and age are not proof of safety, but they catch the hallucinated-name case before pip install runs someone else's code.

CI gates

CI is where guardrails stop depending on discipline. The same gates apply to human and AI code, but AI code makes them non-negotiable.

# .github/workflows/ci.yml (excerpt)
jobs:
  checks:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v5
      - run: uv sync --locked                       # lockfile must match pyproject
      - run: uv run ruff check --output-format=github .
      - run: uv run ruff format --check .
      - run: uv run mypy src
      - run: uv run pytest --cov=src --cov-fail-under=80
      - run: uvx bandit -r src -ll                  # security lint
      - run: uvx pip-audit                          # known vulnerabilities

Add to that, at the repository level:

• Branch protection: no direct pushes to main, required reviews, required status checks.
• Secret scanning and push protection (GitHub, GitLab and others offer it; gitleaks or trufflehog in pre-commit).
• Dependency review on pull requests that change lockfiles.
• CODEOWNERS on sensitive paths (auth, billing, infra) so a human expert always reviews them.

Context files and rules

Agents follow your conventions much better when the conventions are written down where they look. Each tool has its own file:

What to put in them: how to build, test and lint; architecture boundaries ("handlers never call the database directly"); libraries to use and to avoid; security rules ("never log request bodies", "all SQL through the repository layer"); and the definition of done ("tests pass, ruff clean, no new dependencies without asking"). Keep them short and specific, and commit them so the whole team, and every agent, gets the same instructions.

Agent modes and permissions

The more autonomy you give an agent, the more the environment has to provide the safety.

Rules that hold across tools:

• Default to asking. Claude Code, by design, requests permission for anything that might modify your system; allowlist only what is safe or easy to undo (file edits, git commit), not git push or deploy commands.
• Full auto only in a sandbox. Anthropic's own guidance is to use --dangerously-skip-permissions in a container without internet access, because of the risk of data loss and exfiltration via prompt injection. VS Code warns that auto-approve removes your chance to cancel destructive actions.
• Always start from a clean, committed git tree. Codex CLI warns if you start an autonomous mode in a directory not tracked by git. Take the hint.
• Separate credentials. An agent should never run with your production cloud credentials loaded in the shell.

4. The tools landscape (late April 2025)

Agentic coding went from preview to mainstream in about three months. Snapshot as of April 29, 2025:

All four are moving weekly, so check the current docs before relying on a specific flag. The guardrails above do not depend on which one you pick.

5. A pre-merge checklist for AI-written code

• [ ] I can explain every changed line to a colleague.
• [ ] Tests describe the intended behavior, and I reviewed them before the implementation.
• [ ] No tests were deleted, skipped or weakened to make the change pass.
• [ ] The diff does one thing and is small enough to review properly.
• [ ] Every new dependency exists, is maintained, and is actually needed.
• [ ] No secrets, credentials or personal data in code, prompts, logs or fixtures.
• [ ] Inputs are validated; queries are parameterized; no eval or shell=True on external data.
• [ ] Lint, type checks, tests, security scans and dependency audit pass in CI.
• [ ] Sensitive paths (auth, billing, infra) were reviewed by their owners.
• [ ] The context file was updated if the agent repeatedly got a convention wrong.

Why this matters

• Speed without review is just faster debt. AI makes producing code nearly free; understanding it is still the expensive part, and it is the part that keeps production up.
• Bigger batches hurt delivery. DORA's 2024 data associates more AI adoption with lower delivery stability. Small diffs and tests are the mechanism that turns AI speed into shipped value.
• The supply chain has a new attack surface. Hallucinated packages are predictable, and slopsquatting turns them into an exploit. Verifying dependencies is now part of reviewing AI code.
• Agents act, not just suggest. Once a tool can run commands, permissions, sandboxes and credentials become part of your development environment design.
• Vibe coding is a feature, in its place. For prototypes and throwaway tools it is a genuine productivity gain. Knowing which mode you are in is the skill.

Final Thoughts

Karpathy's original post was honest about its scope: throwaway weekend projects, amusing, mostly works. The trouble starts when the same habits leak into code that has users, data and an on-call rotation.

The good news is that the guardrails are not new. Tests first, small diffs, real review, least privilege and strict CI are what good teams did before AI. Agents simply raise the price of skipping them. Vibe code your prototypes freely, then switch modes on purpose before anything ships.

Sources

• Andrej Karpathy, post on X introducing "vibe coding" (February 2, 2025). Full text reproduced in Simon Willison's post below.
• Simon Willison, Not all AI-assisted programming is vibe coding (but vibe coding rocks) (March 19, 2025): definition of vibe coding as building software without reviewing the code; the rule about only committing code you can explain.
• Anthropic, Claude 3.7 Sonnet and Claude Code (February 24, 2025): Claude Code as a limited research preview and its capabilities.
• Anthropic, Claude Code: Best practices for agentic coding (April 18, 2025): CLAUDE.md, permission allowlists, test-first workflow, --dangerously-skip-permissions only in a container without internet access.
• GitHub, GitHub Copilot: The agent awakens (February 6, 2025): agent mode preview in VS Code Insiders.
• GitHub, Vibe coding with GitHub Copilot: Agent mode and MCP support rolling out to all VS Code users (April 4, 2025).
• Visual Studio Code, March 2025 release (version 1.99): agent mode in Stable, experimental chat.tools.autoApprove and its warning, .github/copilot-instructions.md.
• Cursor, Changelog 0.45 (January 23, 2025): .cursor/rules; Changelog 0.46 (February 19, 2025): Agent as the default mode.
• OpenAI, Codex CLI on GitHub (released April 16, 2025): approval modes, sandboxing, codex.md project docs, experimental status. TechCrunch, OpenAI debuts Codex CLI, an open source coding tool for terminals (April 16, 2025).
• Spracklen et al., We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs (2024, revised March 2025; accepted to USENIX Security 2025): package hallucination rates.
• Socket, Slopsquatting: How AI Hallucinations Are Fueling a New Class of Supply Chain Attacks (April 8, 2025): origin of the term (Seth Larson) and repeat rates of hallucinated names.
• Perry, Srivastava, Kumar and Boneh, Do Users Write More Insecure Code with AI Assistants? (ACM CCS 2023).
• DORA, 2024 Accelerate State of DevOps Report (October 2024): AI adoption associated with a 7.2% reduction in delivery stability; batch size hypothesis.