The classic engineering interview loop was built for a world where writing a correct function from scratch, under time pressure, was a decent proxy for skill. That proxy is now weak. A take-home that used to fill an evening can now be largely done by a coding assistant in minutes, and a LeetCode-style screen can be solved by a tool that stays hidden from screen sharing.

You have two bad options if you do nothing: keep a loop that mostly measures who used AI most quietly, or ban AI everywhere and test a way of working your engineers no longer use. The better option is to decide, stage by stage, what you want to measure and whether AI should be part of it.

This post is a cheatsheet for engineering managers and hiring leads. It covers the AI policy options, what companies have published, the signals that still matter, a sample loop and rubric, fraud, fairness and calibration. It is not legal advice: hiring, recording, identity checks and accommodations are regulated differently in each country and state, so check local law with your HR and legal teams before you change the process.

1. The problem: what AI broke in the old loop

Stage What it used to measure What it measures now, without changes
Take-home exercise Ability to build something working, unsupervised Mostly whether the candidate can prompt an assistant and clean up the output. Useful only if AI use is expected and the follow-up is live.
Online algorithm screen (LeetCode-style) Data structures and algorithms under time pressure Easily solved by AI. Overlay tools exist specifically for this format.
Live coding on video Problem solving while thinking aloud Still useful, but vulnerable to hidden assistants and off-screen help

Three concrete signals that this is not hypothetical:

  • Cheating tools are a product category. CNBC reported in March 2025 on Interview Coder, a subscription tool built to generate solutions during virtual technical interviews while staying hidden from screen sharing. Its founder claimed it helped him get offers from several large tech companies. Similar tools exist.
  • Large employers changed process. In the same reporting, Google CEO Sundar Pichai suggested at a February 2025 town hall that "some fraction" of interviews be in person, Amazon asks candidates to acknowledge they will not use unauthorized tools, and Deloitte reinstated in-person interviews for some UK graduate hiring.
  • AI helps a minority of applicants lie, not only cheat. Gartner reported in July 2025 that 6% of surveyed candidates admitted to interview fraud (posing as someone else or having someone pose as them), and predicted that by 2028 one in four candidate profiles worldwide could be fake. Treat the prediction as a warning signal, not a measured rate.

2. Proxy candidates and deepfakes: the security angle

The cheating problem is a hiring-quality issue. The fraud problem is a security issue.

Warning What it says What it means for your loop
FBI IC3 PSA I-062822-PSA (June 2022) Complaints about deepfakes and stolen personal data used to apply for remote IT, programming, database and software roles, some with access to sensitive data. Lip movement and audio did not line up; background checks found stolen identities. Remote engineering roles are a known target. Video is not identity proof.
FBI IC3 PSA I-012325-PSA (January 2025) North Korean IT workers used AI and face-swapping during video interviews, then exfiltrated data and extorted employers. Recommends identity verification during interviewing, onboarding and employment, and conducting as much of hiring and onboarding in person as possible. Identity checks belong in the process, not only in onboarding.

Practical controls that do not punish honest candidates:

  • Verify identity once, properly, using your HR or background check process, and make sure the person you verified is the person in later rounds and on day one.
  • Keep one stage in person or on a proctored onsite for roles with access to production data, where feasible. Offer travel or a local office.
  • Drill into their own history ("walk me through the hardest bug in that project"). Proxies struggle with depth, not polish.
  • Same person, same camera, every round. Note mismatches, do not accuse in the moment, and escalate through a defined path.

3. Two policy options: AI-allowed vs. AI-prohibited stages

You do not need one policy for the whole loop. Decide per stage and tell the candidate in writing, before the interview.

AI-allowed stage AI-prohibited stage
Measures How the candidate works today: decomposition, directing tools, reviewing and verifying output, owning the result Unassisted fundamentals: reasoning, reading code, explaining trade-offs, debugging without a crutch
Best formats Realistic, ambiguous build or extend task in a real-ish codebase; take-home with live follow-up Code reading and review, debugging walkthrough, system design discussion, behavioral interview
Cheating risk Low by design: there is nothing to hide Higher on video; reduce with formats where AI help is slow or obvious (conversation, whiteboard, in person)
Problem size Larger, messier, more requirements than can be finished. AI makes small problems trivial. Small enough to reason about aloud
What fails candidates Accepting wrong AI output, not testing, not being able to explain their own code Not understanding the code or the system they discuss

A reasonable default for most teams: one AI-allowed practical stage, the rest AI-prohibited or AI-irrelevant (conversation formats where an assistant does not help much). Write it down like this:

Interview AI policy (sent with every invite)

- Stage 2, Practical build (90 min): AI assistants ALLOWED and expected.
  Use your own tool or our provided environment. We assess how you plan,
  direct, review and verify the work, not how much code you produce.
- Stage 3, Code review and debugging (60 min): no AI assistants.
  Official language docs are fine. Please share your full screen.
- Stages 4-5, System design and Behavioral: no AI assistants or scripts.
- Applications and preparation: AI help with polishing is fine. Experiences
  and claims must be your own.
- Need an accommodation? Reply to this email. It will not affect evaluation.

4. What companies have published

Public approaches as of March 2026, from primary posts where available (Google and Amazon are covered in section 1):

Company Approach Source type
Anthropic Encourages candidates to use Claude to refine applications and prepare. AI is not allowed in take-home assessments (unless stated) or live interviews, and must not be used to fabricate experience. Candidate AI guidance page (updated July 2025)
Canva Replaced the Computer Science Fundamentals round for backend, machine learning and frontend roles with an AI-Assisted Coding round in June 2025. Candidates are expected to use tools like Copilot, Cursor or Claude on realistic, ambiguous problems. Assessed on decomposition, technical decisions, and spotting and fixing flaws in AI output. Engineering blog (June 2025), interviewer guide (October 2025)
Meta Reported in July 2025 to be piloting AI-enabled coding interviews and recruiting employees as mock candidates. Internal rationale, per the reporting: more representative of how engineers work, and makes LLM-based cheating less effective. Press report of internal post, not an official policy page

Two lessons from Canva are worth copying even if you prohibit AI. First, make problems bigger and more ambiguous (an airport takeoff and landing control system instead of Conway's Game of Life), because AI makes small, well-specified problems trivial. Second, their interviewer guide names three failure patterns: showcasing AI instead of solving the problem, a "feature marathon" that trades quality for feature count, and hands-off over-delegation. Strong candidates planned first, used AI for suitable subtasks, and kept ownership of every technical decision.

5. Signals that still matter

Whatever the policy, these are the skills that separate engineers when code generation is cheap.

Signal What good looks like How to observe it
Problem decomposition Asks clarifying questions, states assumptions, splits the work into testable pieces, prioritizes Ambiguous practical task; first 10 minutes are the richest signal
Verification of AI output Writes or runs tests, checks edge cases, reads generated code before accepting it, notices hallucinated APIs AI-allowed stage. Seed the task with a spot where the obvious AI answer is subtly wrong.
Code reading and review Finds the real bug, ranks issues by severity, explains why, suggests a fix Give a 150-300 line PR with planted issues
Debugging Forms hypotheses, narrows down with evidence, uses logs and tests, does not guess randomly Failing test or production-style incident with logs
System design Clarifies requirements and scale, makes trade-offs explicit, identifies failure modes, knows what to measure Conversation plus diagram
Ownership Can defend every line they submitted, including AI-generated lines Follow-up questions on the practical or take-home

One rule ties these together: a candidate is responsible for every line they submit. "The AI wrote that" is not an answer in the interview, just as it is not an answer in a production incident review.

6. Structured interviews and rubrics: the basics

AI changes the format of the questions. It does not change what decades of selection research say about how to evaluate.

Finding Source Practical meaning
Structured interviews: validity about .51; unstructured: about .38; work samples: about .54 Schmidt and Hunter (1998) meta-analysis Structure was already known to beat free-form conversation.
Revised estimates: structured interviews about .42, top-ranked among common procedures; unstructured interviews about .19; most validities reduced by roughly .10-.20 after correcting for overcorrection of range restriction Sackett, Zhang, Berry and Lievens (2022) The ranking shifted toward structured interviews. The gap between structured and unstructured widened.

Treat these as correlations with later job performance across many jobs, not guarantees for your team. They still make the direction clear.

What "structured" means in practice:

Element Meaning Why it matters
Job analysis Write down the 4-6 competencies the role actually needs Prevents testing what is easy to test instead of what matters
Same questions Every candidate for the role gets the same core prompts, with planned follow-ups Makes candidates comparable
Anchored scale Each score (1-4) has a written description of observable behavior Two interviewers can agree on what a "3" is
Independent scoring Each interviewer submits scores and evidence before any debrief Avoids anchoring on the loudest voice
Evidence, not adjectives Notes record what the candidate said or did, not "smart" or "not a fit" Makes decisions reviewable and fairer
Decision rule Defined before the loop: e.g. no 1s on must-have competencies, average at least 3 Stops post-hoc rationalization

7. Sample loop

A senior backend engineer loop, about four and a half hours of candidate time, can be done in one day.

Stage Duration AI policy Competencies Format
0. Recruiter screen + identity step 30 min N/A Motivation, logistics, role fit Video, camera on. Explain the loop and the AI policy.
1. Experience deep-dive 45 min Prohibited Ownership, communication, technical depth Structured questions about one past project, with drill-down follow-ups
2. Practical build 90 min Allowed and expected Decomposition, verification, AI collaboration, code quality Extend a small service in a provided repo. Requirements exceed the time. Candidate narrates decisions.
3. Code review and debugging 60 min Prohibited Code reading, debugging, judgment Review a PR with planted issues, then fix one failing test using logs
4. System design 60 min Prohibited Design, trade-offs, failure modes Whiteboard or shared doc. In person if one stage must be onsite.
5. Behavioral and collaboration 45 min Prohibited Collaboration, feedback, conflict, learning Structured behavioral questions with a scoring guide
Debrief 30 min N/A Decision Scores submitted first, then discussion against the decision rule

If you keep a take-home, keep it short (2-3 hours maximum), allow AI explicitly, and make the live follow-up the part that counts: extend it, change a requirement, or ask the candidate to find a bug you introduced.

8. Sample rubric

For stage 2 (AI-allowed practical build). Score 1-4 per competency with evidence.

Competency 1: Concern 2: Below bar 3: At bar 4: Strong
Decomposition Starts coding or prompting immediately, no plan Plan exists but misses key requirements Clarifies requirements, states assumptions, splits work sensibly Also prioritizes explicitly and cuts scope with a stated reason
Verification Accepts AI output without reading or running it Runs it, but only on the happy path Tests edge cases, reads generated code, catches the planted flaw Writes tests first or alongside, explains why each matters
AI collaboration Delegates everything or cannot direct the tool Uses AI but loses control of the design Uses AI for well-defined subtasks, keeps the overall design Gives precise context, rejects bad suggestions with reasons
Code quality Does not run or is unreadable Works but hard to change Clear, idiomatic, appropriately simple Production-ready choices (errors, logging, naming) without gold-plating
Ownership Cannot explain parts of their own submission Explains what, not why Explains every decision and alternative Identifies weaknesses in their own solution unprompted

Hard fails, regardless of total: misrepresenting identity, using AI in a stage where it was prohibited after a clear policy, or being unable to explain submitted code at all. Store rubrics as data so they can be compared and audited:

RUBRIC = {
    "stage": "practical_build",
    "scale": (1, 4),
    "must_have": ["decomposition", "verification", "ownership"],
    "competencies": ["decomposition", "verification", "ai_collaboration",
                     "code_quality", "ownership"],
}


def decide(scores: dict[str, int], evidence: dict[str, str]) -> str:
    """Apply the pre-agreed decision rule. Evidence is required for each score."""
    missing = [c for c in RUBRIC["competencies"] if not evidence.get(c)]
    if missing:
        return f"incomplete: add evidence for {', '.join(missing)}"
    if any(scores[c] == 1 for c in RUBRIC["must_have"]):
        return "no hire (concern on a must-have)"
    avg = sum(scores[c] for c in RUBRIC["competencies"]) / len(RUBRIC["competencies"])
    return "hire signal" if avg >= 3.0 else "no hire signal"


print(decide(
    {"decomposition": 3, "verification": 4, "ai_collaboration": 3,
     "code_quality": 3, "ownership": 3},
    {"decomposition": "asked about load limits first",
     "verification": "test caught off-by-one in AI code",
     "ai_collaboration": "rewrote retry logic by hand",
     "code_quality": "clear modules, typed errors",
     "ownership": "defended queue choice"},
))  # hire signal

9. Fairness, accessibility and candidate experience

Topic Practice Why it matters
Clear policy upfront Send the AI policy with the invite, per stage Honest candidates should not be disadvantaged by guessing the rules
Equal tooling Provide the AI tool and environment in AI-allowed stages, or allow a defined list Paid tool access should not decide outcomes
Accommodations Offer them in every invite: extra time, captions, screen readers, breaks, alternative formats Some candidates rely on assistive technology. Separate it from the AI policy. Check your local legal duties.
No surprise surveillance Tell candidates what is recorded or monitored, and why Hidden monitoring erodes trust and may be regulated where you hire
Don't rely on AI detectors Use format design and follow-up questions instead Detection tools are unreliable; false accusations harm real people
AI used by you If you use AI to screen CVs or score interviews, disclose it and keep human decisions Gartner found only 26% of candidates trust AI to evaluate them fairly; some jurisdictions classify recruitment AI as high-risk (e.g. the EU AI Act)

10. Red flags

Each of these is a prompt for a follow-up question or escalation, not proof of anything by itself.

Red flag Possible meaning What to do
Long pauses then fluent, perfect answers; eyes tracking text off-screen Reading from a hidden assistant Switch to follow-ups: "Why that and not X?", "What breaks if the input doubles?"
Strong solution, cannot explain it or modify it Did not write or understand it Ask for a small change live
Audio and lip movement out of sync, odd artifacts on face movement Possible deepfake Ask to adjust camera or lighting; escalate via your fraud path
Different voice, face or style between rounds Proxy candidate Compare with identity step; escalate
Over-delegation in AI-allowed stage Weak judgment, not cheating Score it in the rubric, do not treat it as misconduct

Apply the same checks to every candidate. Red flags applied selectively become bias.

11. Calibrating interviewers

A rubric is only as good as the agreement between the people using it.

Step How Output
Train on the rubric Walk through each anchor with real (anonymized) examples of 1-4 Shared vocabulary
Mock interviews Run the new AI-allowed stage on internal engineers first, as Meta reportedly did Problem timing, planted flaws that actually get caught
Score independently, then compare Two interviewers score the same recorded or mock session Disagreement shows vague anchors
Measure agreement Percent agreement and Cohen's kappa (agreement corrected for chance) per competency Low kappa means rewrite the anchor, not retrain the person
Review outcomes Quarterly: pass rates per stage, score distributions per interviewer, offer acceptance, early performance of hires Find harsh or lenient interviewers and stages that predict nothing

Why this matters

  • Decide per stage, not per company. "AI allowed" and "AI prohibited" are both valid; mixing them deliberately gives you both signals.
  • Make the problem bigger, not the surveillance stronger. Realistic, ambiguous tasks and live follow-ups beat detection software.
  • Verification is the new core skill. Code generation is cheap; reading, testing and owning code is where engineers differ.
  • Structure is still the strongest lever. Same questions, anchored scales and independent scoring outperform gut feel, with or without AI.
  • Fraud is a security problem. Verify identity consistently and involve security for roles with access to sensitive systems.

Final Thoughts

The goal of an interview loop has not changed: predict who will do good work on your team, fairly and at a reasonable cost to candidates. What changed is that some old formats stopped measuring anything useful, and pretending otherwise rewards the candidates who hide their tools best.

Start small. Write the AI policy for each stage, redesign one practical stage around decomposition and verification, add anchors to your rubric, and calibrate on internal mock sessions before candidates see it. Then review the data every quarter and keep only the stages that earn their place.

Sources