Semester Project Guide

The semester project is where you do deep learning rather than study it: pose a question, build the experiment, run fair comparisons, present your results in a recorded video, and be ready to explain your own code. It is worth 35% of the course grade (350 points), in CS 4715 and CS 5715 alike.

Ground rules

  • Teams: 1–3 students (CS 5715: 1–2). Cross-listing sections may mix. Every member submits every deliverable on Canvas, and grades are per student. Every member must own an identifiable part of the code, named in the report’s Contributions section; a code defense can check this for any member.
  • Compute: use whatever compute you have access to; the Google Colab free tier is enough for a well-scoped project. Design small: the grade rewards experimental quality, not scale.
  • Framework: PyTorch (course standard). Existing libraries (HF transformers, torchvision, etc.) are allowed and encouraged where they serve the question.
  • AI policy: Tier 2 — open with disclosure. See AI Usage section below.

Fair comparison policy

No model family is required, favored, or penalized. What is graded is the comparison:

  • Every project must include at least one serious simple baseline (logistic regression, a small MLP, a fine-tuned small pretrained model, a majority-class or n-gram baseline — whatever is honestly strong for your task).
  • Baseline and proposed method get comparable tuning budgets. “We tuned our model for a week and ran the baseline once” is a rubric penalty, not a win.
  • A project where the baseline wins is a successful project if the experiment is well designed and the analysis explains the result. Negative and null results, honestly established, earn full credit.
  • Report variance where it matters (multiple seeds where runtime allows, or an explicit statement of why not), and use the uncertainty tools from reading 4.5.

Milestones and dates

Date Deliverable Points (of the project’s 350)
End of the S13 week Pre-proposal (≤1 page) 20
End of the S23 week Proposal: the full plan (a document) 70
End of the S27 week Final report + repository 155
Sunday before S28, 11:59 PM Pacific Recorded presentation (video) 105
S28 / S29 (class time) Code defense, only for students the instructor calls not scored separately

The late policy (12-hour grace, −10%/day, 60% floor) applies to the pre-proposal, proposal, report, and recorded presentation, and nothing is accepted after the last day of classes. A code defense cannot be late.

Pre-proposal (20 points)

≤1 page: team members; the question; the data (source, size, license — must be available now, not “we will collect”); the baseline and the method you will compare; the metric. The team writes one pre-proposal, and every member submits the same PDF. Graded on completion and feasibility; you will get written feedback within a week, before you write the proposal.

Proposal (70 points)

1–2 pages, a document only (no code): the question; the data; the baseline and the method; the metric and how you will estimate uncertainty; a compute estimate (total GPU-hours, and where they will run); per-member responsibilities, member by member: which code and which experiments each member owns, without overlap; what changed since the pre-proposal and what is at risk. The team writes one proposal, and every member submits the same PDF; it is graded per student. Graded on feasibility and clarity; written feedback within a week.

Final report (155 points)

Due at the end of the S27 week, before the recorded presentation: your presentation presents the results of the report you submitted.

Two parts in one PDF: the team report, which every member submits, and your own individual part, appended at the end.

The team report: 6–10 pages (plus references/appendix), any clean format. Required sections:

  1. Introduction & question
  2. Data (with license/provenance)
  3. Methods (baseline + proposed, tuning budgets for both)
  4. Results (with uncertainty; plots readable in grayscale)
  5. Analysis (error analysis or ablation — at least one)
  6. Limitations
  7. Model card (template below)
  8. AI Usage (requirements below)
  9. Contributions (who did what, per member)

The individual part (1–2 pages, written by you alone): the code you wrote (files and functions), the decisions you made and why, the results you obtained, and the problems you hit and how you handled them. It must match the Contributions section and the repository.

Repository requirements: README with exact reproduction commands, seeds fixed, dependencies pinned (requirements.txt), no data files over 50 MB committed (use download scripts).

Submitting code. The report goes in with its code, in one of two ways: a public GitHub repository, with its link and the full commit SHA at the top of the PDF’s first page; or a zip of the repository uploaded on Canvas together with the PDF (code, README, requirements.txt, download scripts; no datasets or model checkpoints). A private repository goes in as a zip.

Recorded presentation (105 points)

Every team records one 8–10 minute video, every member presents a part of it, and every member submits the same video on Canvas (a file upload, or a link the instructor can open without signing in; not a OneDrive link) by 11:59 PM Pacific on the Sunday before S28. Slides with your voice over them are enough; no live demo is needed. There are no in-class presentations. Structure: question → setup → comparison → what you learned. Cover:

  • the problem;
  • your baseline and your experimental design: what comparison you set up, and why it is fair;
  • results with error bars (seed spread, a bootstrap confidence interval: the S27 toolkit);
  • what you learned, including negative results. A careful experiment where the simple baseline wins is a successful project.

Code defense (only if you are called)

After reading the reports and repositories and watching the videos, the instructor calls some students, individually, for a code defense (a few may also be called at random). The list and the time slots are posted on Canvas before S28 and may be extended before S29. If you are called, come at your slot during S28 or S29 class time; Virtual Meeting and CdA students do theirs by video call at the same slot, and Online (Web) students arrange a video-call time with the instructor that week. If you cannot make your slot, contact the instructor before it to arrange another time that week. Students who are not called have no class meeting in S28 or S29.

In a defense you answer 2–3 questions from the instructor about code you wrote — why a design choice, what a specific block does, what happens if X changes. You may consult your own code while answering, and evaluation choices are fair game. The defense is not scored on its own: it checks that the code your Contributions section names as yours is code you understand. A student who cannot explain that code has their own project grade lowered; the rest of the team is not affected. This is where Tier 2 AI use meets accountability: using AI to build something you cannot explain will cost you here.

Model card (required template)

Include this table, filled honestly, for your final trained model:

Field Content
Model summary architecture, size, framework
Intended use what it is for; what it is not for
Training data source, size, known skews or gaps
Evaluation metrics, test sets, uncertainty estimates
Known limitations failure modes you observed (be specific)
Ethical considerations misuse potential, bias risks, licensing

See Mitchell et al., Model Cards for Model Reporting (FAT* 2019) and reading 4.6.

AI Usage section (required)

Your report must include a section stating: which AI tools you used; for what (ideation, code, debugging, writing); a concrete example of something an AI tool got wrong or you rejected (if applicable); and which parts of the work are entirely your own. “We did not use AI tools” is an acceptable entry if true. This section is not graded on how much you used AI — it is graded on completeness and honesty, and it is cross-checked in any code defense.

Rubric detail (report, 155 points)

The team report is worth 120 points, the same for every member:

Criterion Points
Question is answerable and answered 20
Baseline strength & fairness of comparison (equal budgets) 28
Experimental hygiene: seeds, splits, no leakage, uncertainty 28
Analysis depth (error analysis / ablation) 20
Writing clarity & figures 14
Model card + AI Usage completeness 10

Your individual part is worth 35 points, graded for you alone: concrete work of your own, decisions explained, and agreement with the Contributions section and the repository.

CS 5715 additionally: the report must engage at least one paper from the graduate reading list or a venue-published paper relevant to the method — a short “relation to prior work” paragraph with a real citation (this is folded into Analysis depth).

Topic seeds (optional)

You are free to propose anything meeting the rules. Ideas sized for the Colab free tier: compare LoRA ranks on a small pretrained LM for a niche classification task; does data augmentation close the gap between a small CNN and a fine-tuned ResNet on ≤10k images; distill a fine-tuned model into a tiny one and chart the quality/size frontier; diffusion on a structured 2D dataset vs. a VAE, evaluated beyond eyeballing; benchmark-contamination case study with a clean re-split; reproduce a published small-scale result and stress one assumption (5715-friendly).