Lecture on Future Advancements in AI: Talent, Compute, and Data
Housekeeping
All microphones are muted.
Use chat function for technical issues only.
Introduction
Moderator: Tim Wong, Senior Technology Fellow at The Institute for Progress.
Speaker: Micah, Research Analyst at Georgetown's CSET, focusing on Cyber AI project, disinformation, and more.
Key Discussion Points
Challenging Conventional Wisdom
Common beliefs in AI policy: compute power as the primary driver, biggest model is the best model.
Micah's goal: To challenge the belief that compute is the most important factor in AI advancement.
AI Policy Background
AI Triad: Identified by Ben Buchanan (CSET) - compute, data, and algorithms as three pillars of AI progress.
Talent: Underpins all three pillars.
Policy Focus: Recent policy focus heavily on compute (e.g., Chips and Science Act, export controls on GPUs). Less focus on Talent despite the importance.
Importance of Compute in Recent AI Progress
Significant increase in compute demands for AI models (e.g., language models like GPT-3, GPT-4).
Risks of over-indexing on compute: Unsustainable scaling, specificity to language models.
Survey on AI Researchers
Respondents: 533 AI researchers from academia and industry.
Findings:
Majority would prefer to spend additional budget on hiring talent rather than purchasing more compute.
Compute is less often cited as a constraint compared to talent and data.
Past progress driven by compute, but future progress expected to rely more on better algorithms.
Policy Implications
Variation in Compute Needs: Different AI subfields have varied compute needs.
Industry vs. Academia: Discrepancy in compute access between elite institutions/industry and broader academic field.
Future Focus: Need to balance compute with other factors like talent and efficient research.
Q&A Highlights
Strategic Importance of Language Models: Debate on whether focusing on compute-intensive language models is the best strategy.
Short-term vs. Long-term Policy Levers: Compute as a short-term lever, talent development as a medium-to-long-term strategy.
Interdependence of Compute and Talent: Access to compute can attract talent, but this requires supportive immigration policies.
Data as a Lever: Importance of well-curated datasets for specific applications, often best coordinated by government efforts.
Conclusion
Compute Policy: Valuable but not a substitute for talent-focused policies.
Talent Policy: Critical for sustainable AI advancement, includes immigration and education initiatives.
Holistic Approach: AI policy should balance compute, talent, data, and algorithms for a comprehensive strategy.
Final Notes
The conversation continues on how to best balance these factors in AI policy and research funding.
Encouragement to read the full survey for more detailed insights.