Why the Course Looks Like This
This page explains the main design decisions in Vibe Coding 101 and the evidence behind each one. It's written for instructors adapting the course, for reviewers, and for students who want to know why they're being asked to do things this way. The evidence comes from the course's research review, completed in September 2026 (landscape report), its October 2026 update (update report), and the detailed research notes. The October 2026 update section below maps each change in the October revision to its evidence.
On this page: The short version · How strong is the evidence? · Decisions and evidence · October 2026 update · What we deliberately did not do · Open questions and how this course measures itself · How to update the course · Sources
The short version
Teach the loop, not the tool. University and industry courses on AI-assisted programming have converged on one shape: vibe-code and deploy something in week one, climb from chat to editor to agent, add specs, git, tests, review and security along the way, and grade students on explaining their code rather than on whether it runs (report). The research supports that shape. AI help raises task completion, but without structure it weakens debugging and code comprehension, and struggling students come away believing they understand more than they do (Prather et al.; Anthropic; Bastani et al.). Vibe-coded apps fail in predictable places: public database keys without access rules, leaked secrets, invented package names and destructive agent commands (report). And the free tools change monthly.
So the course teaches one tool-agnostic workflow, the Safe Loop; gives every tool two tested fallbacks; saves everything to GitHub; checks understanding hardest in the agent weeks; builds its security unit around real breaches; and keeps every product detail in one dated file, Tool Handout.
October 2026 adds friction on purpose. New controlled studies agree that AI-assisted novices finish more and remember less, and that what helps is making students explain or repair code before it counts (update report). So the Safe Loop gained two habits, check first and teach back; the safety contract gained a ninth rule; and the course became a 3-credit course with two studios a week, giving the extra practice room to happen (details).
How strong is the evidence?
Not all of it is equally strong, and the course is designed with that in mind.
- Randomized trials exist for a few questions: how AI help affects learning (Anthropic, 2026; Bastani et al., which studied high-school math, not coding; and a 2,379-student trial in which a course-integrated AI tutor lowered final grades, EdWorkingPapers 26-1598, whose abstract doesn't say what the tutor did or whether the course was in computing) and how it affects experienced developers' speed (METR).
- Small controlled studies from 2026 measure what happens after the task: recall two days later and claimed ownership (Bergh et al., 55 students), and maintenance with the AI switched off (Sankaranarayanan, 78 learners). Both are preprints.
- Observational studies and surveys describe what novices actually do when vibe coding (Geng et al.; Fawzy et al.) and how courses are assessed (Geng et al. syllabus study; ACM Task Force).
- Course reports such as CMU 15-113's are rich but self-reported, and its instructor notes that an LLM drafted most of the report before he revised it (Student Wisdom report). The Fall 2026 report was generated by an AI from about 630 anonymized answers with only light manual review; its quotes and statistics were checked by script, but its theme counts are an AI's reading (CMU F26 report).
- Practitioner consensus (vendor guides and well-known engineers) supports the workflow itself. It is strong agreement, not controlled evidence.
- Security figures come mostly from vendors with a commercial interest (Veracode; GitGuardian; UpGuard; Semgrep), though they point the same way, and one academic audit agrees (Deng et al.).
Where a decision rests on weaker evidence, the tables below say so.
Decisions and evidence
Sequence and pacing
| Decision | Evidence | Where it appears |
|---|---|---|
| Start with vibe coding in week 1, and deploy something that day | Across university and industry courses, students vibe-code and deploy a prototype in week one (report). DeepLearning.AI's zero-experience course follows the same describe → generate → customize loop (Build with Andrew). The course teaches Karpathy's original meaning and why the field moved on (Willison; Karpathy, 2026), because students will meet the term used both ways (Kim & Yegge). | Week 1 · Syllabus §1 |
| A ladder of tools: chat with preview → app builders → editor assistant → own server → agent | CMU 15-113 put its first project on browser chatbots so students would feel that workflow's limits before meeting an editor assistant (Student Wisdom report). Northeastern's CS 7180 climbs the same ladder explicitly (NEU CS 7180). Each rung comes after the habits it needs. | Syllabus §4 |
| Spec first, in week 3, with WHEN … THE APP SHALL … criteria | Vendor guides agree on spec or plan before code (GitHub Spec Kit; Codex best practices; Claude Code best practices). EARS-style criteria are learnable by non-programmers (Kiro). CMU students rated writing the spec as harder than managing agents (Student Wisdom report), so it is practiced early, before agents; in Fall 2026 CMU itself moved spec-driven development from homework 8 to week 5 (CMU F26 report). Writing skill predicts vibe-coding proficiency (Thorgeirsson et al.), and every spec now gets a Goldfish test (EGM). No study yet shows that spec-first helps novices learn (update report). | Week 3 · SPEC template · Goldfish test |
| Read, debug and own the code in week 4, with an AI-off Debug Clinic and "the AI plans, you write, the AI reviews" | Debugging is the skill AI use erodes most (Anthropic). CMU's highest understanding scores came on the assignment where students read and partly wrote the code (Student Wisdom report). CS1-LLM's final exam was mostly AI-free tracing, explaining, testing and debugging, and results largely matched pre-AI cohorts (CS1-LLM). Coding practice, not awareness, predicted who checked AI code (Fawzy et al.). Since October 2026, week 4 also has students write their first automated test, from a worked example, so that tests can be written before the code from then on: tests an AI writes after faulty code catch fewer faults (Konstantinou et al.). | Week 4 · Debug Clinic · check first |
| Deploy early and often: GitHub Pages in week 1, a dedicated deployment and secrets lab in week 5 | CMU students reported that deployment took more time than coding (Student Wisdom report). Deploying a small thing first makes the week 5 full-stack deploy a second attempt, not a first. | Week 1 · Week 5 |
| Hold agents until week 6 | UChicago and Northeastern both reserve autonomous agents for the second half (UChicago; NEU). DORA finds AI amplifies existing practice, and names strong version control and small batches as conditions for it to help (DORA), so git, specs and tests come first. At CMU, understanding and code-reading scores fell once agents took over (Student Wisdom report). | Week 6 |
| Open the agent week by "being the agent": change an app using only an agent's tools | Utah had students change a calculator using only an agent's tools (Utah calculator repo). Stanford's Fall 2026 course opens by showing students how an agent works inside (CS146S). Both show the loop at no cost before students hand it control. | Be the Agent |
| Security and review in week 7, built around an attack-your-own-database lab | The most instructive failures share one root cause: a public key plus missing authorization rules (Superblocks on Lovable; Wiz on Moltbook). In September 2026 the same root cause turned up in 16,326 Supabase databases, and tables that agents create through the API start without RLS (UpGuard). Newer models have not written more secure code (Veracode Spring 2026); working code was also secure only 54.8% of the time in one 2026 benchmark (Semgrep), so waiting for better models is not a plan. The attack page is now hosted on the course site, so the lab needs no extra repository (review H7). | Week 7 · RLS Attack Lab · case studies |
| Four projects, from toy to real stakes: home page → game → a tool you'd use → deployed capstone | The near-universal opening is a single-file page, then a game, which needs no backend and gives instant feedback (DLAI; Google codelab). Vanderbilt builds on learners' own tools and files (Coursera). The capstone follows CMU 15-113's deployed app with live URL, README, prompt log and demo video (CMU 15-113). | Projects |
Tools and cost
| Decision | Evidence | Where it appears |
|---|---|---|
| $0 required | The free-tool landscape can support an eight-week course (report). Michigan runs on free local models and CMU on free tools (UMich; CMU 15-113). Where a university plan ran short, UChicago students had to buy a paid plan (UChicago), and MOOC learners object to subscription costs (Coursera reviews). In October 2026, the new non-major courses mostly assumed students pay ($25–$100 a month at Stanford Continuing Studies), and 69% of CMU's cohort already paid for an AI plan (update report). With an AI tutor harming first-generation students most in a large trial (EdWorkingPapers), a free-tools course is an equity choice, provided it adds friction for everyone rather than more AI help. | Tool Handout · Syllabus §10 |
| A primary tool and two fallbacks per layer; switching is a planned drill | In CMU 15-113, free credits on several tools ran out mid-assignment for at least one student (Student Wisdom report). The report recommends teaching usage dashboards in week 1 and practicing the switch before it's urgent. In October 2026 the fallbacks, not the primaries, took the cuts (Gemini Free's default model, v0's credits, Replit's free plan, Antigravity's models), so the fallbacks are now the course's thinnest layer and get re-tested before each cohort (update report). | Tool Handout · I ran out of free credits · week 5 provider-swap drill |
| Two-layer materials: a concept layer that names no versions or limits, plus one dated tool handout | Nearly every tool a 2025 syllabus would have used changed its free tier, price, name or owner within fifteen months, and several disappeared (report). Recorded screen demos go out of date within months (VibeGym); Maven revises its curriculum every quarter (Maven). | CONTRIBUTING: two-layer rule · Tool Handout |
| Copilot in VS Code, in a Codespace, as the backbone from week 4 | It is the most dependable no-cost agent path: agent mode, MCP and a CLI on the free plan, all three operating systems, and no 18+ restriction (GitHub Docs); re-checked unchanged on 21 October 2026 (update report). A remote environment such as Codespaces is a recommended sandbox for agents (Willison). | GitHub Codespaces · Week 4 · Week 6 |
| Switch LLM providers by configuration, not code | Major free providers accept OpenAI's request format, so an app that reads the base URL from an environment variable can switch by changing one setting (report). | Week 5 |
| GitHub is the save point for everything, whatever tool made it | Builders disappear or expire: GitHub Spark and Firebase Studio were shut down, and Replit's free published apps expire, taking work with them (report). An agent's own checkpoints are "not a replacement for git" (Claude Code best practices). Strong version control is one of DORA's conditions for AI to help (DORA). | GitHub basics · git cheat sheet |
| Variant stacks for under-18 and EEA/UK/Swiss cohorts | Several tools are 18+ only, and the Gemini API's free tier may not serve users in the EEA, UK or Switzerland (report). Settle this before enrollment, not in week one. | Course variants |
Assessment
| Decision | Evidence | Where it appears |
|---|---|---|
| Oral walkthroughs in weeks 4, 6 and 8, clustered in the agent weeks | CMU 15-113 follows each project with a mock-job-interview check; UCSD can hold a grade until a student walks staff through the code; CS1-LLM required a video explaining one function (CMU 15-113; UCSD; CS1-LLM). 68% of surveyed educators had already changed their assessment, oral defenses among the changes (ACM Task Force). Understanding fell when agents took over at CMU, so checks increase in the agent weeks rather than fading out. CS1-LLM's 10–15 minutes per project gives a staffing estimate. A 2026 CS1 course with weekly oral code reviews kept exam scores steady while AI pasting rose (Fowles et al.), and a 2026 review proposes four "evidence obligations" (explain, verify, modify, account) that map onto the walkthroughs' four scoring areas (Mahmoud). | Oral Walkthroughs |
| Publish six skills students must show without AI, and require each to be shown once | 31.1% of CS1-LLM students were not confident which problems they should be able to solve without the AI, and the instructors said they should have spelled it out (CS1-LLM). The October review found the skills weren't really gated: exit tickets are completion-graded and walkthrough scores are compensatory (review A-09). Since October 2026 each skill must be shown at least once in a walkthrough, retake or follow-up, and all six are needed for a C or higher. | Six Skills You Must Show Without AI · walkthroughs |
| AI-free reflections and peer reviews, submitted on Canvas | CMU 17-316 bans AI from reflections, which must describe concrete moments when the tool surprised the student (CMU 17-316 HW1); UCSD bans AI from peer reviews (UCSD). Learners misjudge their own learning when AI helps (Prather et al.; Bastani et al.); writing without it is where that shows. Since October 2026 this graded personal writing goes on Canvas, not in public repositories (University of Idaho FSH 4700 F-6 and FSH 2600; review A-02–A-04). | REFLECTION template · CODE_REVIEW template · rubric |
Labs graded on effort, with cheap process logs (PROMPTS.md) | CMU 15-113 grades homework on effort; Utah grades in-class work on effort and fails assignments with missing AI logs; Michigan requires an AI log (CMU 15-113; Utah; UMich). Pasting share links into a file keeps logs cheap (NJIT). Effort grading plus oral checks avoids an arms race over detecting AI use. | rubric · PROMPTS template |
| Capstone graded on spec, tests, security and deployment; a working demo is required but not sufficient | AI makes brittle prototypes easy, so Utah gives credit only for edge cases, how features interact, and taste (Utah); Michigan's rubric is half spec and design, half implementation and tests (UMich). | capstone rubric · capstone brief |
| Participation counts (15%), across 16 studios | In the syllabus study, all conventionally graded courses counted participation, at a median weight of 17.5% (Geng et al.). | rubric |
Teaching format
| Decision | Evidence | Where it appears |
|---|---|---|
| Concept talks of 20 minutes or less; most studio time is lab | Learner reviews reward building and punish lecture-heavy courses (Coursera reviews). | Every week's slides.md, e.g. week 1 |
| Core and stretch homework tiers, with a "Minimum to keep up" box | Maven splits homework this way (Maven). Weaker students benefit least from AI (ACM Task Force; Prather et al.), so the core tier stays achievable and support concentrates there. Stated times must add up: CMU re-estimated its API and backend homeworks from 2–3 hours to about six (CMU F26 report), and the October review found several weeks over budget on paper (review A, F07, F14). | Every week's homework.md, e.g. week 1 |
| One workflow, the Safe Loop, in every lab from week 2, plus the two-strikes rule, check first and teach back | Vendor guides and practitioners describe nearly the same loop: plan, small task, verify with evidence, read the diff, commit (Claude Code; Codex; Osmani; Willison). Observed professionals plan first and check all agent output (Huang et al.). After two failed corrections, start fresh (Claude Code). Support for the loop itself is practitioner experience, not trials. The two habits added in October have more direct support: independent tests catch more faults (Konstantinou et al.), and an explanation gate cut AI-off maintenance failures from 77% to 39% (Sankaranarayanan). | The Safe Loop · prompt patterns |
| A hint-only course tutor, kept narrow: the only AI allowed in the Debug Clinic, and optional elsewhere | Unrestricted AI raised practice scores 48% but cut exam scores 17%; a hint-only tutor raised practice 127% with no significant exam harm (Bastani et al.). CS50's Duck and CodeAid follow the same idea in programming (Liu et al.; CodeAid). In October 2026, a 2,379-student trial found that a course-integrated AI tutor lowered final grades by 0.37 standard deviations and platform participation by 0.90, most for first-generation students (EdWorkingPapers), so the course does not add a course-wide tutor. Bastani studied math, the new trial's tutor design is not described, and this tutor is untested in this course. | Tutor mode · Week 4 |
| The Game in an Hour in week 2 | Adapted from CMU's one-hour game homework (Student Wisdom report). Games give instant visual feedback and need no backend. In Fall 2026 CMU turned it into a 30-minute in-class sprint followed by up to four hours at home, and satisfaction and confidence rose (CMU F26 report); this course keeps the hour in Studio A and continues the game in Studio B and the homework. Graded on effort, and anyone may do it untimed. | Week 2 |
| Code handoff and human-vs-AI review in week 7, the review done on a real pull request | CMU 15-113's HW7 has students swap unfinished games and finish each other's (Student Wisdom report); Stanford grades a comparison of a student's own review with an AI reviewer's (W7 spec). Pull requests were never taught although pairs had to merge through them, so week 6 Studio B now teaches them and the week 7 review is a pull-request review (review M4, H7/H9). | Week 7 · CODE_REVIEW template · branches and pull requests |
Safety, privacy and openness
| Decision | Evidence | Where it appears |
|---|---|---|
| A nine-rule safety contract, skimmed before Studio 1 and signed on Canvas in week 1, each rule with a plain gloss and the week it's taught | Each rule answers a real incident: destructive agents (Replit, Gemini CLI, Antigravity, PocketOS); poisoned rules files and skills (Pillar Security; HalluSquatting); invented and hijacked packages (arXiv 2605.17062; Socket); leaked secrets and stolen AI keys (GitGuardian; TechCrunch); agents that publish (PixelLeak); copyright (USCO). Signing it in week 0, framed by disaster stories and full of untaught terms, put students off (review F09). | Safety Contract · case studies |
| Privacy settings on day one | Several consumer AI plans train on chats and coding sessions by default unless users opt out (Anthropic; GitHub), and some let human reviewers read samples (Gemini privacy hub). | Privacy settings |
| Three project rules: no real payments, no real personal data about others, nothing you'd miss if an agent deleted it | Free hosting tiers prohibit commercial use (Vercel Hobby; GitHub Pages); free tiers train on inputs; agents have deleted real data (Fortune). | Syllabus §6 |
| The lethal trifecta and Rule of Two in the week 7 security checklist | Coding agents usually combine private data, untrusted content and a way to send data out (Willison); an agent should hold no more than two without human approval (Meta). Each 2026 prompt-injection case combined all three, and in each the vendor left the user as the last line of defense (Adversa; CSA): hence safety contract rule 9. | SECURITY_CHECKLIST template |
| An open license | CMU 17-316's openly licensed course was adapted at Memphis and NJIT within a semester, and outside adopters help keep materials current (Memphis; NJIT). | LICENSE · LICENSE-CODE · CONTRIBUTING |
October 2026 update
In October 2026 the course was revised from three inputs: decisions by the instructor (a 3-credit, eight-week course at the University of Idaho, for cohorts of five or more), a review of the course in four parts (pedagogy and workload, student experience, assessment and policy, technical accuracy), and the October research update. The tables below map each change to its evidence. "Review" means that course review. It isn't published, so each row says what the finding was; the codes in brackets (such as H1, F09 or S-01) are kept for the maintainers' records. The research report and notes are linked directly.
Credits and the weekly format
| Change | Why | Where |
|---|---|---|
| 3 credits, about 135 hours, in eight accelerated weeks | The old design was about 50 hours of student work, roughly one credit under the University's credit-hour definition (catalog Regulation D-1; review A-08). The instructor chose 3 credits, so the workload was raised to match rather than the credits lowered. 1- and 2-credit versions are kept as variants | Syllabus · variants |
| Two 3-hour studios a week: Studio A (learn and build) and Studio B (workshop), plus about 11 hours outside class | Labs were overloaded: stated 90–110 minutes, realistically 115–165 minutes for a first-time learner, with unbudgeted set-up and "first time ever" skills in expert-sized slots (review A). Skills the capstone needs were assumed but never taught: pull requests, charts, external APIs, the Supabase connection, a first automated test (review H8/H10, M4). Studio B gives them a slot | Syllabus §4 · every week's lab.md |
| "This week at a glance", Core and If-time parts, and a "Minimum to keep up" box | On a phone, one week's overview was 12.6 screens long, opened with the instructor's timetable, and didn't say what was required (review F03, F14). Each week needs the previous week's work, so students need to know the one thing they must have working (review M7) | Every week's README.md and homework.md |
| A recap slot and a delayed-recall exit-ticket question about last week's project | Two days after a task, AI users recalled 39% of what they'd done against 52% for web-search users (Bergh et al.). The recap was promised but had no slot (review M7) | Syllabus §4 · exit tickets |
| Doubled estimates for the week 5 and 7 homework, with work moved to stretch | CMU re-estimated similar homeworks from 2–3 hours to about six (CMU F26 report); week 5 was already over budget on paper (review H4, F07) | Weeks 5 and 7 |
Course-wide conventions
| Change | Why | Where |
|---|---|---|
| Check first: write the "done when" check, from week 4 an automated test, before asking for the code, in a separate prompt | Tests an AI wrote after faulty code caught 14% of faults, against 25% for independently written tests (Konstantinou et al.). Evidence about tools, not learners | Safe Loop |
Teach back: 2–3 sentences in PROMPTS.md before committing AI-written code, graded in the capstone Process row | With the AI switched off, 77% of unrestricted AI users failed a maintenance task, against 39% of those whose tool required an explanation first (Sankaranarayanan) | Safe Loop · capstone rubric |
An Assisted-by: line on AI-assisted commits (taught in week 2, required from week 6) | 48.8% of 281 open-source AI policies require disclosure (Hora et al.); LLVM, OpenStack, OpenTelemetry and Fedora use this line (All Things Open) | Safe Loop |
| Three habits from week 1: ask me questions first; show a reference; say what works | Students' own top practices in CMU's fall survey (CMU F26 report); the same habits in a practitioner discussion (HN) | Prompt patterns |
The Goldfish test for every spec and AGENTS.md (weeks 3 and 6) | Michigan's fresh-session test (EGM); writing skill predicts vibe-coding proficiency (Thorgeirsson et al.) | Pattern 18 |
| Personal AI-use rules, written in week 1 and revised in week 8 | Students who wrote their own rules did better in both arms of a seven-week pilot (Azimi) | Weeks 1 and 8 |
Agents: approvals on, never autopilot; AGENTS.md gets a Scope section, taught as advice, not enforcement | Removing a scope statement raised one agent's out-of-scope actions from 0% to 17.1%, and ask-before-continuing harnesses overstepped least (Qu et al.); an explicit scope line cut simulated attacks from 26/50 to 4/49, not to zero (UK AISI). Anthropic's docs call instruction files "context, not enforced configuration" (Claude Code docs). Autopilot exfiltration was ruled the user's responsibility (Adversa). Real enforcement is the Codespace, approvals, scoped tokens, spending caps and RLS: the report's distinction between rules and rails (conclusion) | AGENTS.md template · Week 6 |
A ninth safety rule ("agents don't publish, and untrusted content never shares a session with my secrets"), and extensions to rules 4–8: spending caps, scoped tokens, backups out of reach; tests are not a security check; installing runs code, pin versions; skills, plugins, hooks and received folders; Assisted-by: | Each addition answers a 2026 incident or study: PocketOS, stolen AI keys and hard caps (Willison); working-but-insecure code (Semgrep; Deng et al.); preinstall malware (Socket); the received-folder attack (Manifold); PixelLeak and autopilot exfiltration, which no old rule covered (report P2). Plain glosses and week tags respond to review F09 | Safety contract |
| Graded personal writing on Canvas, not in public repos; no classmates' names in repos | Reflections, the signed contract and peer reviews were public by default (review A-02–A-04), against University policy on posting graded work (FSH 4700 F-6) and student records (FSH 2600, which applies FERPA) | Syllabus §9 |
"Not every key is a secret": the Supabase publishable key in public/config.js; secret keys only in server environment variables; only publishable/secret keys taught | Week 5 said keys live in .env "and nowhere else" while week 7 put the publishable key in the browser and the starter's .env.example put it on the server (review H8/H10, B-03). Supabase deprecates its legacy keys by the end of 2026 (Supabase) | Troubleshooting · capstone starter |
| Supabase grants and RLS, always together | From 30 October 2026, new tables need explicit grants (Supabase changelog); a bare grant to anon recreates the 16,326-database exposure (UpGuard). The trap is the report's inference from the two sources | Troubleshooting · Week 7 |
| Vercel: submit the production URL and a screenshot; private organization repos can't deploy on the free plan | Vercel's free plan now deletes older deployments (Vercel changelog); the organization block applies to private repos only (Vercel Community) | Troubleshooting |
| AI-use labels always in words, square-bracket placeholders, one due-date wording, one walkthrough timing rule | The traffic lights differed only by color in 113 places (review F04); angle-bracket placeholders vanish when GitHub renders Markdown (review D, Low); due dates were worded three ways and walkthrough 3's timing differed across six files (review F03, M8, L3) | Everywhere |
| Six skills gated; the integrity process follows the University's Student Code of Conduct | The skills weren't really required (review A-09). FSH 2300 names reliance on prohibited AI tools as academic dishonesty and requires the instructor to notify the student and the Dean of Students before any grade outcome (review A-01) | Without-AI skills · Syllabus §11 |
| A screen reader and keyboard guide, text versions of every diagram, and a published self-paced path with a "Working alone?" line for each pair activity | ASCII diagrams read as garble and overflowed on phones; the only screen-reader advice was in a hidden instructor note (review F05, F06, F11, F20). Self-paced learners were pointed to an unpublished guide whose links the site hides (review F10, F21) | Accessibility guide · Self-paced path |
| A history-aware secret scan and committed lockfiles | The old scanner missed the course's own key formats and never read git history, yet its clean output counted as evidence; npm audit failed in every starter without a lockfile (review S-01, B-01, B-02) | Troubleshooting · starters |
| Node.js 22 LTS (minimum 20.12) | The starters need 20.12, and Node 20 has reached end of life (technical review) | Troubleshooting |
Week by week
| Week | Change | Why |
|---|---|---|
| 0 | Split into "Before Studio 1" (about 60–90 minutes) and "During week 1"; contract and self-assessment on Canvas in week 1; privacy advice and the week 1 lab agree about chat history | Week 0 asked for 2–3.5 hours and a signature on untaught terms; two pages contradicted each other (review F09, F15, F16) |
| 1 | The three habits; personal AI-use rules; an accessibility check of your page; free AI-fluency pre-work (Anthropic Academy) | Tool-agnostic judgment before tools (October notes); habits as above |
| 2 | Prompt makeover workshop; the Assisted-by: line; Willison's reading | Disclosure norms as above |
| 3 | The builder bake-off as a jigsaw; preview in a Codespace before committing, in Safe Loop order; the Goldfish test; a written peer spec review on Canvas | The bake-off overran the lab, and the old build loop committed before testing (review H1, H2) |
| 4 | git revert practice before walkthrough 1; "You write it" with feature cards (30 minutes); a first automated test from a worked example | Walkthrough 1 asked for a command not yet practiced; the first code students wrote got 20 minutes (review H11, M1); check first needs tests from week 4 |
| 5 | Part 1 folded into the demo; provider swap and the history-aware secret check in Studio B; a fill-in test skeleton; an external-API mini-lesson; production URL | Overloaded lab and homework; tests with no worked example (review H3, H4, F07, H10) |
| 6 | Capstone repo created before class; SPEC for one criterion; 50+ minutes for the first agent run with check first and teach back; branches and pull requests; a planted bug in agent-written code; feature clinic; the AGENTS.md trial | Week 6 crammed a new repo and a full spec into 20 minutes (review H5/H6); pull requests were never taught (review M4); planted bugs led to direct editing and better next attempts (Pădurean et al.); configure-then-test, as in CS146S |
| 7 | New case studies, with cases 12–14 as the default jigsaw; the hosted attack page; a Supabase connection worked example; a 20–30 minute AI-off maintenance task; the peer review as a pull-request review; a 7-minute injection demo; the handoff test as homework; student-facing Supabase and Vercel troubleshooting | The old cases were mostly from 2025 (report P4); the lab needed an extra repo and gave the database connection one sentence (review H7, H9, F08, F12, M6); AI-off maintenance is where the 77% vs 39% gap showed (Sankaranarayanan) |
| 8 | A known-issues section in the README; walkthrough 3 in Studio A or by appointment; revised AI-use rules; production URL, screenshot and a v1.0 release | Michigan grades "on evidence, not feature count", and saying what doesn't work is worth more than hoping nobody tries it (L08 slides) |
| All | Milestones defined once in the capstone brief; rescue starters for students who fall behind | Milestones disagreed across pages; each week needs the last week's artifact (review M7, M8) |
What the October evidence did not change
- The overall shape. Nothing published between 25 September and 21 October 2026 overturns the Safe Loop, the ladder of tools, the AI-free walkthroughs or the $0 stack (update report).
- No course-wide AI tutor. The new trial points the other way, so tutor mode stays narrow (EdWorkingPapers).
- The same two evidence gaps remain: no semester-long trial of teaching non-programmers through vibe coding, and no study of spec-first or test-first teaching on real learners (update report).
What we deliberately did not do
A course built around one vendor. Stanford's Fall 2025 course featured one partner tool per week. Its week 2 relied on a free year of Cursor Pro that has since closed to new sign-ups, so that week can no longer be reproduced as written (report). Vendor academies are free but tied to one tool, and some need a paid plan (Anthropic Academy). This course names products only where a step needs them and keeps the specifics in Tool Handout.
AI detectors. The course requires AI, so the question is never "did you use it?" but "can you explain what you shipped?" Educators are moving away from trying to make assignments AI-proof (ACM Task Force). Effort-graded labs, PROMPTS.md logs, AI-free writing and oral walkthroughs make detection unnecessary.
Required paid tools. The report found that if a course must require one paid tier, the evidence supports a cheap editor-assistant upgrade or one month of a frontier agent timed to weeks 6–8. We made both optional (Syllabus §10), because the free path works, and learners criticize courses that require paid subscriptions (Coursera reviews).
Agents in week 1. Agents amplify whatever habits a student already has (DORA), and understanding dropped when agents took over at CMU (Student Wisdom report). The destructive-agent incidents in the case studies show what an agent does without git, tests and containment. We borrowed the useful part of early agent teaching, showing how an agent works from the inside, but placed it in week 6 as Be the Agent, with no AI at all.
Cursor or Gemini CLI as the primary tools. Both were strong choices in 2025. By mid-2026, Cursor's free plan lacked MCP and its free student year had closed, and Gemini CLI's free Google login had ended, while its own README still advertised the old quota for months (report; dates in Tool Handout). Copilot Free kept agent mode, MCP and a CLI at no cost, with no age restriction. Since October 2026, Cursor's free plan has been folded into a social-network subscription bundle (dates in TOOLS.md).
Banning outside AI in favor of a course tutor only. CS50 allows only its own tutor (CS50). That suits a course whose goal is learning to program by hand. This course's goal is learning to direct AI tools safely, so AI is required, and the hint-only tutor is used only where the goal is the student's own thinking: the Debug Clinic.
A course-wide AI tutor. It would be easy to add one to every lab. In a 2026 trial with 2,379 students, a course-integrated tutor lowered final grades and participation, most for first-generation students (EdWorkingPapers). The course adds friction for everyone (check first, teach-backs, AI-off tasks, delayed recall) rather than more AI help for those who struggle.
Grading on whether the app runs. A working demo is required for the capstone but earns little on its own. Robustness, specs, tests, security and explanation carry the grade, following Utah and Michigan.
Open questions and how this course measures itself
What nobody knows yet
The report is explicit about the gaps (report; conclusion):
- No controlled study has compared course orderings. The shared shape of existing courses rests on design rationale and student surveys (Geng et al.).
- No semester-long randomized trial has tested teaching non-programmers through vibe coding. Studies of novices vibe coding are small (19 students in Geng et al.) or surveys. The longest retention measures in the 2026 studies are 48 hours and one week (Bergh et al.; ICER 2026).
- No study has tested whether writing specs or tests first helps novices learn. The case is practitioner consensus, plus one 2026 study about AI-written tests, not learners (Konstantinou et al.).
- No study shows how much programming knowledge non-programmers need to vibe-code safely. The six skills are an expert-consensus floor, not a measured one.
- The best evidence for hint-only tutors is from high-school math (Bastani et al.), and the largest 2026 trial found a course-integrated tutor harmful without describing its design (EdWorkingPapers). The course tutor is an untested adaptation.
- Productivity evidence is contested. METR's 2025 and 2026 studies point in different directions, METR calls its 2026 estimates unreliable (METR 2026), and it has published nothing since May 2026. Professionals' trust is conditional: in Stack Overflow's 2026 survey only 6.6% trusted AI for important decisions (Stack Overflow).
- How much non-programmers retain is the biggest unknown. The course's own delayed-recall questions and week 7 AI-off task can start producing that evidence (update report).
How the course checks itself
Because of those gaps, the course measures its own students' understanding before and after, as the report recommends.
| Measure | What it tells you | Where |
|---|---|---|
| Pre/post self-assessment against the eight learning outcomes | How students' confidence changes over the course | Self-Assessment: Before and After |
| Exit tickets, weekly and AI-free, with one delayed-recall question about last week's project | Which of the six skills are shaky, week by week, and what students still remember a week later | |
| The week 7 AI-off maintenance task on each student's own capstone | Whether students can change their own AI-built code without the AI: the measure on which the 2026 studies found the biggest gaps | Week 7 |
| Oral walkthrough results in weeks 4, 6 and 8 | Whether understanding holds up once agents arrive: the CMU warning sign | Oral Walkthroughs |
| Confidence vs explanation in each reflection: rate your confidence, then explain the part you're least sure of | Calibration: does felt understanding match demonstrated understanding? | REFLECTION template |
| A two-question survey after each studio, a mid-course survey, and a short survey after each project, as CMU 15-113 runs (Student Wisdom report; CMU F26 report) | Tool problems, workload, perceived understanding and code reading, early enough to act. With small cohorts, report nothing below five answers | |
| A log of tool failures and fallback switches | Whether the fallbacks in TOOLS.md actually work for a cohort | Pre-cohort checklist |
If you teach the course, compare self-assessment with walkthrough results. A gap between the two, especially in weeks 6–8, is the signal the research predicts. Please share what you find through an issue or discussion (CONTRIBUTING).
How to update the course
- Tool changes go in Tool Handout only. That is the point of the two-layer rule. If a concept page needs changing because a tool changed, the page probably named something it shouldn't have.
- Before each cohort, and again at mid-course, actually use every primary and fallback tool instead of trusting the docs; Google's own pages lagged its changes by months. The steps are in Pre-cohort checklist, and TOOLS.md keeps a calendar of announced changes (in late 2026: Supabase grants on 30 October, Antigravity's models on 2 November, Supabase's legacy keys at the end of the year; report P5).
- Check age and region rules for the cohort and pick a stack from Course variants before enrollment.
- New evidence goes into the research notes with a link, then into this page if it changes a decision. Keep every claim linked to a source.
- New incidents can be added to the case studies only with sources, and conflicting reports should be flagged.
- Report problems through the process in Contributing and Style Guide, and run its checks before sending a pull request.
Sources
- The full review: Teach the Loop, Not the Tool (September 2026)
- The October update: Free Tiers Shrink While Evidence Demands Friction (25 September – 21 October 2026)
- Research notes, with more detail and evidence labels:
- October 2026 notes: