Introduction to Generative AI

Topics

Introduction to Generative AI

• Attention Is All You Need
• What Is a LLM?
• How LLMs Work?
• Evaluating LLMs
• Emerging directions
• Limitations and Open Challenges

Weekly Discussion: Trust and Decision-Making with Generative AI

Please submit all graded work via Canvas. Participation requirements and grading details are provided in Canvas.

This week we explored generative AI and large language models. These systems can generate text, summarize documents, answer questions, analyze images, and assist with complex tasks. In many situations, generative AI may be used not only for simple questions but also to support more complex decisions.

This discussion asks you to reflect on how much you would trust AI in a real-world scenario.

Step 1: Imagine a real-world scenario

Describe a situation in which someone might rely on generative AI to support an important decision. Choose a scenario that involves more than a simple question or short answer.

You may consider examples such as:

• medical information or health advice
• legal information or policy interpretation
• financial planning
• academic research
• workplace decision-making

Briefly describe:

• the situation
• what question or task the AI system would be asked to perform
• what kind of answer the system might generate

Step 2: Your evaluation

Explain whether you would rely on the AI’s response in this situation.

You may consider:

• Would you trust the AI enough to follow its recommendation?
• Would you verify the result before acting on it?
• Would you treat the AI as a primary source of information or only as a starting point?
• Would your level of trust change depending on the context?

Explain your reasoning.

Step 3: Practical and ethical considerations

Consider the potential risks and limitations of using AI in this scenario.

You may address:

• hallucination or incorrect information
• lack of transparency about how the system produces answers
• bias in training data
• accountability if the result is wrong

Explain why these concerns matter in your chosen situation.

Step 4: Role of human judgment

Finally, briefly respond to one of the following:

• In this scenario, what role should human expertise play?
• Should AI act mainly as an assistant, an advisor, or something else?
• Are there situations where people should avoid relying on AI entirely?

Expected outcome

Your post should demonstrate:

• Understanding of how generative AI may be used in decision contexts
• Ability to reflect on trust, reliability, and limitations of AI systems
• Awareness of the role of human judgment in AI-assisted decision-making

Reading

• GeeksforGeeks: What are LLM Parameters?
• OpenAI: What are tokens and how to count them?
• IBM: What is a context window?
• Datasets for large language models: a comprehensive survey
• You may explore this visualization of an LLM workflow. It includes many technical details, so reviewing it is optional. You may simply explore the visualization to get a general sense of how LLM components connect.
• Optional: Jay Alammar: The Illustrated Transformer explains the transformer architecture in detail. Some parts may be technically challenging.
• AI Demystified: Introduction to large language models
• Andreas Stöffelbauer: How Large Language Models work? From zero to ChatGPT
• MIT Sloan Management Review: How LLMs Work: Top 10 Executive-Level Questions
• Drexel University: Dragons' Guide to GAI
• Blog posts about Measuring LLM Performance
  • Data leakage is a major issue when measuring LLM performance
  • What LLM benchmarks get wrong about measuring model performance
  • Why the LMArena is a flawed approach to LLM evaluation
• Simon Willison: Understanding the recent criticism of the Chatbot Arena
• The Leaderboard Illusion
• Wikipedia: Language model benchmark
• Meta’s benchmarks for its new AI models are a bit misleading
• Evidently AI: 30 LLM evaluation benchmarks and how they work
• A Multimodal World
• GeeksforGeeks: Multimodal Large Language Models
• Google: What is agentic AI?
• Explaining Agentic AI: The Good, the Bad & the Ugly

• I asked ChatGPT if I should drive or walk to the car wash to get my car washed — and it struggled with basic logic

• A Definition of AGI

• Why language models hallucinate

• The AI revolution is running out of data. What can researchers do?
• Anthropic: A small number of samples can poison LLMs of any size
• Yann LeCun: We Won't Reach AGI By Scaling Up LLMS

• Richard Sutton – Father of RL thinks LLMs are a dead end

• How close is AGI? What the experts say.

• Is AI Hiding Its Full Power? With Geoffrey Hinton