Readings

For this module, choose one or more materials directly related to the module’s topic. You may select materials from the course reading list or use relevant materials found elsewhere.

As a general guide, read one or more articles or other written materials totaling approximately 10,000–12,000 words, listen to or watch at least 90 minutes of podcasts or videos, or complete a comparable combination of written and audiovisual materials. You may engage with additional materials.

Please submit all graded work via Canvas. Participation requirements and grading details are provided in Canvas.

Suggested Materials

• GeeksforGeeks: What are LLM Parameters?

• OpenAI: What are tokens and how to count them?

• IBM: What is a context window?

• Datasets for large language models: a comprehensive survey

• You may explore this visualization of an LLM workflow. It includes many technical details, so reviewing it is optional. You may simply explore the visualization to get a general sense of how LLM components connect.

• Jay Alammar: The Illustrated Transformer explains the transformer architecture in detail. Some parts may be technically challenging.

• AI Demystified: Introduction to large language models

• Andreas Stöffelbauer: How Large Language Models work? From zero to ChatGPT

• MIT Sloan Management Review: How LLMs Work: Top 10 Executive-Level Questions

• Drexel University: Dragons’ Guide to GAI

• Blog posts about Measuring LLM Performance

  • Data leakage is a major issue when measuring LLM performance
  • What LLM benchmarks get wrong about measuring model performance
  • Why the LMArena is a flawed approach to LLM evaluation

• Simon Willison: Understanding the recent criticism of the Chatbot Arena

• The Leaderboard Illusion

• Wikipedia: Language model benchmark

• Meta’s benchmarks for its new AI models are a bit misleading

• Evidently AI: 30 LLM evaluation benchmarks and how they work

• A Multimodal World

• GeeksforGeeks: Multimodal Large Language Models

• Google: What is agentic AI?

• Explaining Agentic AI: The Good, the Bad & the Ugly

• I asked ChatGPT if I should drive or walk to the car wash to get my car washed — and it struggled with basic logic

• A Definition of AGI

• Why language models hallucinate

• The AI revolution is running out of data. What can researchers do?

• Anthropic: A small number of samples can poison LLMs of any size

• Yann LeCun: We Won’t Reach AGI By Scaling Up LLMS

• Richard Sutton – Father of RL thinks LLMs are a dead end

• How close is AGI? What the experts say.

• Is AI Hiding Its Full Power? With Geoffrey Hinton