Welcome to CS221, Artificial Intelligence at Stanford.
Instructors: Percy (AI and general instructor) and Dorsa (Robotics and robotic interactions).
Overview of the teaching team (CAs) with interests in NLP, machine learning, data mining, and computer vision.
Weekly sections covering review topics and advanced topics.
Homework 1 posted on the website, due next Tuesday at 11:00 PM on Gradescope (code on Piazza).
Importance of AI
AI is unavoidable in current news and media, with major impacts on various sectors including games, document reading, and medical imaging.
Success stories include AI systems playing games at superhuman levels and performing tasks like speech and face recognition.
AI perceived as transformative for future work and society, but also viewed with caution due to potential catastrophic consequences.
Historical context from 1956 to present: AI winters, evolution from early optimism to recent successes, and contributions to computer science (e.g., Lisp, time-sharing, inference modeling).
History of AI
Early Years (1956-1980s)
Dartmouth College workshop in 1956 marked the beginning of AI, aiming to simulate intelligence by machines.
Initial optimism with programs to prove theorems or play games; AI winter due to funding cuts.
Compute power and problem formulation issues hindered early progress.
AI led to contributions in computer science such as high-level programming languages and time-sharing.
Renewed Interest (1970s-1980s)
Focused on knowledge and expert systems, leading to real industry impacts.
Second AI winter due to challenges in maintaining deterministic rules and over-promising.
Return to Neural Networks (1950s-1980s)
Early work by McCulloch and Pitts on artificial neural networks; logical basis for neurons.
1969’s Perceptrons book stifled interest in neural networks temporarily.
1980s rediscovery of backpropagation, early success stories like Yann LeCun's convolutional neural network for handwritten digit recognition.
Modern AI and Deep Learning (2010s-2020s)
2012: AlexNet revolutionized computer vision, AlphaGo and other AI systems achieving major milestones.
Evolution of AI from logic-based to incorporating neural networks and deep learning techniques.
Conceptual Approach to AI
AI as Agents vs. AI as Tools
AI as Agents: Creating systems that mimic human capabilities like perception, language, reasoning, and learning.
AI as Tools: Using technology to benefit society (e.g., satellite imagery for poverty prediction, optimizing energy consumption in data centers).
AI systems must also address issues like security, bias, and fairness (e.g., adversarial examples, bias in translation models, fairness in criminal risk assessment).
Paradigms and Models
Modeling Inference and Learning Paradigm
Modeling: Simplifying real-world problems into mathematically precise models (e.g., finding shortest paths in graphs).
Inference: Asking questions and finding solutions within a defined model (e.g., shortest path computation).
Learning: Deriving model parameters from data using algorithms (e.g., learning edge weights in graphs from travel times).
Machine learning moves complexity from code to data, with a focus on generalization and formalization via probability theory.
Different Types of Models
Reflex Models
Fixed set of computations without feedback (e.g., linear classifiers, deep neural networks).
State-based Models
Model the world as a set of states and actions to move between states, useful in games, robotics, and motion planning.
Types: search problems, Markov decision processes (MDPs), and adversarial games (e.g., Pac-Man assignment and competition).
Variable-based Models
Focus on constraints and dependencies between variables (e.g., Bayesian networks, constraint satisfaction problems like Sudoku).
Logic-based Models
Systems capable of understanding and reasoning with logic (e.g., natural language understanding, reasoning over heterogeneous information).
Example of logic-based interaction system demonstrating knowledge inference.
Course Logistics
Prerequisites
Programming, discrete math, and probability (with recommended or equivalent courses).
Coursework
Eight homeworks, mixed written and programming, centered on specific applications and models.
Exam focused on problem-solving and model application, closed book with one-page notes.
Final project with milestones (proposal, progress report, poster session) and groups of up to three.
Submissions via Gradescope, seven total late days allowed.
Piazza for communication, extra credit for student assistance.
Honor Code
Collaboration encouraged but homework and code must be written independently.
No code copying or posting on GitHub; debugging discussions restricted to input-output behaviors.
Use of MOSS to detect code duplication.
Introduction to Optimization
Discrete Optimization
Finding best discrete objects (e.g., paths) using Dynamic Programming (DP).
Continuous Optimization
Finding best vectors of real numbers to minimize objective functions, often using gradient descent.
Example Problems
Edit Distance (Dynamic Programming)
Transforming one string into another using minimal edits (insert, delete, substitute).
Memoization to optimize the computational process.
Regression (Gradient Descent)
Fitting a line to data points to minimize least squares error.
Use of gradient descent to iteratively find the optimal slope (w).