CS 5715 Paper Reading List

Graduate students read one paper per module (four total; primary by default, alternate by prior arrangement) and submit a one-page structured response on Canvas. All four are due by Friday, December 4, 11:59 PM PT, the last day before No-Exam Week; the recommended time for each is the listed week, while the module is fresh and before its quiz. Worth 10% of the CS 5715 grade (2.5% each). Late responses follow the standard late policy.

Response format (one page, four parts)

  1. Claim — the paper’s central claim in your own words (2–3 sentences).
  2. Evidence — the experiment or argument that carries the claim, and how convincing you find it (one paragraph). Be specific: which table/figure, what would weaken it.
  3. Connection — one concrete link to a course session, reading section, or your project (one paragraph).
  4. Question — one non-trivial question you would ask the authors (not answerable by re-reading).

Graded 0/1/2 per paper: 2 = specific and critical; 1 = accurate summary but generic; 0 = missing/vague. AI policy Tier 1 applies (your reading, your words; concept questions to AI tools are fine, generated summaries are not).

Papers

Module 1 — recommended: the week of S08

  • Primary: Kingma & Ba, Adam: A Method for Stochastic Optimization, ICLR 2015. arXiv:1412.6980
    • Guiding: What exactly do the bias-correction terms fix, and what goes wrong at early steps without them? The paper’s convergence analysis was later shown to have gaps (Reddi et al. 2018) — does that change how you read the empirical sections? How does reading 1.6’s “Adam is not always best” experiment bear on the paper’s claims?
  • Alternate: Ioffe & Szegedy, Batch Normalization, ICML 2015. arXiv:1502.03167
    • Guiding: The paper’s mechanism story (“internal covariate shift”) is now disputed while the method endures — what evidence in the paper supports the mechanism vs. merely the benefit?

Module 2 — recommended: the week of S12

  • Primary: He, Zhang, Ren & Sun, Deep Residual Learning for Image Recognition, CVPR 2016. arXiv:1512.03385
    • Guiding: The paper distinguishes the degradation problem from overfitting — what is the evidence for that distinction? Why does identity shortcutting address it? Which claims does the reading 2.2 gradient experiment illustrate, and which does it not touch?
  • Alternate: Zeiler & Fergus, Visualizing and Understanding Convolutional Networks, ECCV 2014. arXiv:1311.2901
    • Guiding: How much of the paper’s influence is method vs. the specific observations? What would a skeptic say about deconv visualizations as evidence?

Module 3 — recommended: the week of S22

  • Primary: Vaswani et al., Attention Is All You Need, NeurIPS 2017. arXiv:1706.03762
    • Guiding: Which architectural components does the ablation (Table 3) actually justify, and which are taken on faith? The paper motivates self-attention partly by path length and parallelism — which motivation aged better? Where does reading 3.3’s pre-norm deviation from the paper come from?
  • Alternate: Hu et al., LoRA: Low-Rank Adaptation of Large Language Models, ICLR 2022. arXiv:2106.09685
    • Guiding: What is the intrinsic-rank hypothesis, and how directly do the experiments test it (vs. just showing LoRA works)? Relate it to the LoRA implementation walked through in the S19 reading.

Module 4 — recommended: the week of S27

  • Primary: Ho, Jain & Abbeel, Denoising Diffusion Probabilistic Models, NeurIPS 2020. arXiv:2006.11239
    • Guiding: The paper’s key simplification is the reweighted objective (L_simple) — what does the full ELBO say that L_simple ignores, and why might ignoring it help samples? How does the noise-schedule discussion in reading 4.3 connect to the paper’s choices?
  • Alternate: Ouyang et al., Training language models to follow instructions with human feedback, NeurIPS 2022. arXiv:2203.02155
    • Guiding: What exactly does the “1.3B preferred over 175B” comparison hold constant, and what doesn’t it? Which alignment-tax results survive scrutiny?

Logistics

  • Submit as PDF on Canvas (assignment per module). Online (Web) students: same deadlines; the window does not shift.
  • Choosing the alternate: message the instructor before you start writing.